COUNCIL OF THE CITY OF PHILADELPHIA PUBLIC HEARING COMMITTEE ON PUBLIC SAFETY and TECHNOLOGY & INFORMATION SERVICES - - - - Room 400, City Hall Philadelphia, Pennsylvania Thursday, February 10, 2005 2:20 p.m. - - - - RESOLUTION 040993 - Resolution authorizing a Joint Hearing for both City Council's Committee on Public Safety and City Council's Committee on Technology and Information Systems to hold public hearings to investigate the malfunction of the City of Philadelphia's 800 Megahertz Digital Radio System which occurred five times... PRESENT: COUNCILWOMAN DONNA REED MILLER, Chair COUNCILMAN BRIAN O'NEILL, Chair COUNCILMAN FRANK RIZZO COUNCILWOMAN BLONDELL REYNOLDS BROWN COUNCILMAN MICHAEL NUTTER - - - - V A R A L L O Incorporated Litigation Support Services Eleven Penn Center 1835 Market Street, Suite 600 Philadelphia, Pennsylvania 19103 215.561.2220 215.567.2670 2 I N D E X RESOLUTION 040933 PHILIP GOLDSMITH, Managing Director... JOHN MCFADDEN, Motorola............... 6 7 8 9 10 11 12 13 14 15 16 17 18 19 3 02/10/05 - PUBLIC SAFETY - RES. 040933
Good afternoon. We're getting ready to call the Committee to order. The Committee on Public Safety and Technology and Information Services is now in session. For the record, in attendance is Councilman Frank Rizzo, Councilwoman Donna Reed Miller, Chair of Public Safety; Councilman Brian O'Neill, Chair of Information and technology; and Councilwoman Blondell Reynolds Brown. Will the Clerk please read the resolution?
Resolution 040993, resolution authorizing a Joint Hearing for both City Council's Committee on Public Safety and City Council's Committee on Technology and Information Systems to hold public hearings to investigate the malfunction of the City of Philadelphia's 800 Megahertz Digital Radio System which occurred five times: Once on March 25, 2004, at 8:59 p.m., then on May 18, 2004, at 9:04 p.m., again on July 9, 2004, at 6:30 p.m., and again on August 18, 2004, at 4 02/10/05 - PUBLIC SAFETY - RES. 040933 7:26 p.m., and finally on October 3, 2004, at 1:30 a.m.; and, in furtherance of such investigation, authorizing the issuance of subpoenas to compel the attendance of witnesses and the production of documents to the full extent authorized under Section 2-401 of the Home Rule Charter.
I would just like to put on the record, there has been some question about Verizon and their willingness to testify. I'll take full responsible for their not testifying today because I told the person from Verizon the other day that within the parameters of the hearings, as we set it up, there wouldn't be any testimony about who struck John and which finger was being pointed where. We're really looking at just a recap of the incident since our last hearing and looking forward particularly from the perspective of the City through the Law Department is there any place we can go there just as if you personally bought something, 5 02/10/05 - PUBLIC SAFETY - RES. 040933 had a warranty, were paying a maintenance fee and it wasn't working the way it was supposed to. In addition to finding out why it wasn't working, you'd still want to make sure you were somehow compensated in some way, whether it was a longer warranty added on to cover the period when it wasn't working the way it was supposed to or, as people have suggested, some other method whether it's reduction or elimination of the maintenance fee, something to so that $52 million investment that we may never find out really why it's not working as it should or hasn't worked as it should. But everybody agrees that it hasn't been working as it should and we paid $52 million for it and have a year and a half. Verizon is willing to come to any future meeting or hearing and participate. I take full responsibility for any confusion that occurred there. And I'll turn it back to the Chair. We've had many hearings. This is going to be our last one. That's the only thing we've agreed on, but we still want to 6 02/10/05 - PUBLIC SAFETY - RES. 040933 pursue this in more informal channels. Thank you.
Just to recap, we've mutually agreed that this will probably be our last public hearing. We see this a wrap-up, as an opportunity to move forward and continue working on the problem. And we understand Motorola has some updates to present today, and we want to get started unless -- Councilman Rizzo, do you have anything you want to say?
Nothing other than I appreciate the joint COMMITTEE'S efforts here.
We'd like to call up our first witness which is Phil Goldsmith.
I want to thank this Committee for their continual oversight and vigilance to what we believe is an important 7 02/10/05 - PUBLIC SAFETY - RES. 040933 issue for the people of Philadelphia in terms of public safety as well as our fire fighters and police officers as well as other City employees. I'm joined in this room by many experts, of which I am not one of them, that can certainly talk about the technological issues what has been done and what is being done and what will continue to be done to be ensure we have a communication process that's reliable and meets the 21st century needs of an urban city like Philadelphia. I think the question -- it's been almost two years I think. In fact, I remember it because it was just as I was assuming the position as Managing Director that we went live on this system. And I think that when you look at it from a two-year period of time, the system has worked well, which is not to say perfectly. And I continue to believe that the system that this City has invested money in is a better system and provides better public safety than the old antiquated system that we had. 8 02/10/05 - PUBLIC SAFETY - RES. 040933 I think the issue we have as a City and as a Council is very simple: Do we have a system that meets our public safety needs? Is it a system that's reliable that our men and women in our uniformed services can depend on and can rely on? And I think that as with any new system, there's always shake-out problems, there's always issues that have to be dealt with. I think over the past many months since we had a few issues bubble up, that the City, working with Motorola and subsequently Verizon, has addressed a lot of these issues, has tried to determine which of the issues are related to the system itself, to the actual Motorola system; which issues have been human error; what we have to do in order to improve our protocols, our testing and our cooperation among Motorola, Verizon, as well as the City of Philadelphia and make sure that we are communicating effectively and promptly with the men and women in our uniformed services so they know where we are and what's going on. I think I can say today that we've 9 02/10/05 - PUBLIC SAFETY - RES. 040933 had no -- and I say this somewhat nervously, we have not had any issues since the last hearing, that we have made enormous progress. I personally have met with Verizon, and we probably had of their representatives were 7 in my conference room pledging to do what 8 needs to be done and also making sure that 9 they are working effectively with Motorola. 10 This is an issue where there is 11 never one single problem or source of a 12 problem. It's where you have a lot of different parts. And where we have had issues, as I said, sometimes they've been a Motorola issue, sometimes they've been a Verizon issue, probably there's been times that they've been a City issue. And then we have an overriding issue we have on the national level of the amount of traffic that now comes into our City no one ever anticipated, and these are FCC issues that important issues to us. But they are issues that we as a City cannot solve by ourselves. We started some preventive testing, I think it was yesterday, with Motorola. 10 02/10/05 - PUBLIC SAFETY - RES. 040933 Verizon is going to be doing testing. And I think we can say that we have a much better comfort level in the system and our ability to be communicating with the people in the field, and the field back to headquarters, and in the field among themselves. We're beginning work with SEPTA. We're beginning work with the School District of Philadelphia to make sure that we can be operable among those agencies, and we have a project going on. It's funded by Home Land Security funds to allow us to be communicating with other counties and other counties with us.
So there has been a lot work, and I think that we all recognize this as an important issue and we're going to make sure that our processes are correct in our systems, both the protocols are proper and the communications among our different vendors and the City is running seamlessly. And I think we're a long way there, but we cannot get comfortable in what we've done; we've got to remain vigilant because it's an important 11 02/10/05 - PUBLIC SAFETY - RES. 040933 area. So I'd be glad to answer any questions and have any of the experts come up and help me out.
I have a question as it relates to human error. Certainly, you know the technology and all those other errors are things that -- I don't want to say we can't avoid, however, what is happening the area to correct the human error mistakes? Additional training? What are the needs?
Some of it is training. Again, I don't want to be here pointing fingers. But part of it happened with some of the other vendors. Part of it is training issue. Part of it is understanding that the human error took place first place as opposed to thinking the cause was over here, to understand that some cases someone was doing some maintenance upgrade and forgot to plug something back in. I mean, that's sort of my very simple explanation of what happened. 12 02/10/05 - PUBLIC SAFETY - RES. 040933 Like anyone of us that run large organizations, it becomes an issue of training and accountability and making sure that you have written protocols in place so people know what is expected of them and what has to be done. And I think in this case, both Motorola and Verizon have said that they are trying to tighten their end up, and we as a City are trying to do the same thing.
Mr. Goldsmith, it's been reported that there have been three issues that did not involve Motorola that were -- either two of them have been agreed upon that have been Verizon's responsibility and one that in fact there's still questions on what occurred there. The protocol that you mentioned earlier -- I understand that we have some equipment that's in a landlord tenant facility. Those protocols, could you comment in the event we have another issue with a hub or power outage and batteries going dead that 13 02/10/05 - PUBLIC SAFETY - RES. 040933 we now are in the situation and we have the ability not -- it was a boondoggle that incident that occurred where the system was off for -- there's a lot troubles with the system. Sometimes we say the system was off. It wasn't; it was in the backup mode. But obviously that's not as desirable as it is in its normal operating configuration. What have we done to be assured that the support that any vendor, including Verizon -- it's troubling to me that a power outage that lasted more than eight hours could have the effect that it did on the system, what we call a T1 line that is buried in the ground between here and one of the transmitter sites could become defective and take the system out. And that's why I was hoping that Verizon would be here today and be able to explain that. Maybe you can assure us all that what you refer to as protocol, communication between the monitoring center Verizon and any other vendor that supports this system, how confident are you that all these are now where 14 02/10/05 - PUBLIC SAFETY - RES. 040933 they need to be?
Well, I think I might ask Joe James to come up and help me with this. But let me say that we met with Verizon at our request. We told them we were concerned about some of these issues that had been occurring. They came in and we talked about these issues, and some of them they readily acknowledged and understood had to be resolved and fixed. We impressed upon them the importance and the urgency of this. Now, I think from my conversation with them that they understand. They understood and took responsibility for, in that particular case, what happened. I think they were chagrinned about it, and I think they understood that they had some systemic issues that they had to address. And I'm going under the assumption that those issues are being addressed or have been addressed, but I prefer for the Deputy Commissioner Joe James to talk about the specifics.
Thank you. Good afternoon, Councilman and Chair. 15 02/10/05 - PUBLIC SAFETY - RES. 040933 In regard to that particular incident, we have worked very closely with Verizon and Motorola together to unite the two service management teams together in an effort to avoid this type of human error or, as you describe, faux pas happening in the future. I think one of the things that was apparent once when we reviewed the cause analysis of that particular incident, there were some systemic issues in how they process, do their response to an outage. And it was clear in our review of it and what we needed to apply the fix was to make sure that the protocols that they were speaking in common nomenclature and the processes involved and oversight that was very similar to what we had instituted with Motorola when we started to experience those problems of an early alert system that got our response team on board the moment that we saw issues arise and not wait until the general experiences that there were two hours and the power wouldn't come back up and we don't have to dispatch tonight, we can wait until tomorrow. So we've gone through a 16 02/10/05 - PUBLIC SAFETY - RES. 040933 very exhaustive process with both teams to unite the teams along with our team to align it so that the processes and protocols are uniform, language is used in a uniform manner, and the alert system is systemic to how we do our business and how we make our responses when we have an outage.
Commissioner, do you feel confident that today that all the support, whatever vendor, Verizon, you name it, that all of the Achilles heal that support this system has been tested, quality assured that -- and maybe you can just take 30 seconds and explain how a single telephone line could cause you all of this confusion. It seems to me it is so vulnerable -- and I'm hoping that the redundancies are going to be in place or lines that we know aren't 50, 60 years old in the ground supporting the system. It's mind-boggling to me to think that a phone line could put us in a backup situation that easily.
Well, in response to this was one of the things the Managing 17 02/10/05 - PUBLIC SAFETY - RES. 040933 Director discussed very specifically with the Verizon team when we met. We wanted to look at an overall to process to install preventative maintenance process that would give us the kind of reassurance and the quality controls that every component of the system that they touched is being checked and checked on a somewhat regular basis to ensure that we didn't have to wait until we had a problem. And as you described, our radio system basically has to touch the ground at some point for communications to operate, and our dispatch consoles are connected using facilities that are provided by Verizon. If any one of those links were to go down would have an impact on the ability of the dispatcher to use their primary means of communication or affect the ability of a tower to have the coverage of a particular section 22 of the City. I think since our meeting and our efforts have been underway, one of the things that we're going through that's been started 18 02/10/05 - PUBLIC SAFETY - RES. 040933 is a preventative maintenance plan, a joint preventative maintenance plan between Verizon and Motorola and the City to go out to every site, touch every piece, verify every component, test every circuit and make sure that we have the kind of comfort level and the kind of quality controls in place. And I think we're in that process. We expect that to last about 30 days, and we can go into some further program testing.
Thank you. I'd like to say before we call up our other witnesses from Motorola that in future meetings when you're discussing this, we'd like you to make sure that this Committee knows about those meetings so that Councilmembers can be attendance and any other Members of this Committee.
Thank you. Thank you for your testimony. We'd like to call up John McFadden 19 02/10/05 - PUBLIC SAFETY - RES. 040933 from Motorola.
Good afternoon, Chairperson Donna Miller, Councilman Rizzo, O'Neill, Blondell Reynolds. Thank you very much for allowing us to come today and talk about the system. One of the things I'd like to talk about before we start is on July 14th I came before this Committee and testified and again on July 30th. At that time, I was the sales vice president for the Northern Division for Motorola. At this time, I'm a consultant to Motorola specifically for projects like Philadelphia that I was involved in the sale of. I also from Motorola have the authority to bind the company and to testify at the hearings for Motorola. I just want to make sure you understand my role. It has changed a little.
I hope we were helpful in that change. 20 02/10/05 - PUBLIC SAFETY - RES. 040933
Yes, you were. With me today is Brian Swan who is our systems manager and also Bob Janice who is our quality director for the Northern Divisio. We don't have a prepared statement.
We'd actually like to be updated. It's my understanding the failure hasn't occurred since October 8th or whatever date. I don't have my glasses on, so I can't find the date real quick. But we want to be updated on the status of -- what's been corrected? What still needs to be worked on? We heard the testimony from the Managing Director who says he's been working with Verizon and Motorola. And what do we have to look forward to moving into the future with this whole system?
That is correct, we are working with Verizon to make sure that this system is swept, that we are one team working on this together. Let's go back to the incidents. The incidents we had going back over the last year can basically be described as kind of 21 02/10/05 - PUBLIC SAFETY - RES. 040933 shake-out of a new system. There are 200 of these systems around the United States, cities like Cleveland, Baltimore, Washington, D.C. Each of the municipalities or cities has a different operating procedure, so they kind of overlay their operating procedure over the system. In Philadelphia's case, there were things that were -- so we had the basic system, and as we overlayed the operations, things appeared. As those things appeared, they were basically corrected by software changes over the last year. There hasn't been what we would consider any of those incidences probably since August. I think the Managing Director kind of indicated that the system is extremely stable at this time. But they were software modifications done to the system basically to optimize it for your operation. There were some issues that were actually parts that needed to be repaired or parts that had an issue and were replaced. So it was a combination of software upgrades that we made and parts that -- in one case, there were 22 02/10/05 - PUBLIC SAFETY - RES. 040933 timing boards that had a problem. We replaced them. We provided spares. So at this point, I think a lot of that stuff, in our minds, is behind us. We stand ready to work with the City and continue to monitor this system. This system, again, is monitored hours a day out of our 9 headquarters in Chicago. We have not left the 10 City of Philadelphia's as a customer we don't 11 intend to. We'll be here. We'll continue to 12 support whatever you want to do. That's kind 13 of been our commitment, and we intend to 14 continue to do that. 15
Well, we do 16 have a warranty. That means that you have to 17 do it anyway, correct? 18
Well, the warranty -- 19 I don't know the exact terms of the warranty, 20 but when you by a system from us, if there's 21 something wrong or something you don't like, 22 we don't leave. The warranty could be 23 expired; we don't leave. I'm not sure the 24 exact status of the warranty. I know we're in a maintenance mode at this point. So our 23 02/10/05 - PUBLIC SAFETY - RES. 040933 commitment is, regardless of what our warranty says, we're going to be here and put people on this system until you're comfortable and happy.
Thank you. Mr. McFadden, again, thanks for all your cooperation and your company's cooperation throughout this process. What I'd like to ask you is that we had three incidences, two definitely associated with Verizon; one we're not sure whether they were just implicated because of some human error or what. Why is this system -- Commissioner James explained this closer working relationship between the vendors that you are so very dependent upon to keep this system in an up status. Why is the Verizon component so critical? And what I'm trying to express is, why can these minor -- they sound just like minor hardwire telephone company issues -- put the system into this distress? What are you doing? And we heard 24 02/10/05 - PUBLIC SAFETY - RES. 040933 that they're checking everything that supports the system. You would have thought that that would have been done by the provider of these services and not had to be asked. But why are we so vulnerable here? And do you feel more confident now with this new relationship or this renewed relationship and the work that is going on, the quality assure? Because these issues are hurting the system and even though it's not Motorola's issue, it's still an issue that really affects our fire fighters, our police officers. They don't care whether it's a Verizon problem or Motorola problem. They know their radios go bonk or whatever.
These are network issues. There's redundancy built into the system. We always talk about the redundancy. And the redundancy in the system is if there was -- and the example everybody's using, suppose there was a terrorist attack and there were no phone lines. Suppose that there was a terrorist attack and there was no electricity for three days from the power company. This system has enough redundancy to stay on the 25 02/10/05 - PUBLIC SAFETY - RES. 040933 air. So when we talk about redundancy, there's multiple levels of redundancy. The issue we talked about, I guess, during December affected two consoles. And I don't know the details, quite frankly, of the phone lines and how they work or what happened with the phone lines. But we're willing to work with Verizon and keep working if that would help them and help ourselves and help the City, quite frankly. I mean, it's our goal to keep this system as reliable as possible. And that's the redundancy. So there's multiple levels of redundancy, telephone lines, multiple things that make up the system.
You say you have 200 of these systems throughout the United States. I know you even have it in foreign countries.
In the United States there are 200 systems of this release. There's others of other releases. So it's actually more, but 200 that look exactly like Philadelphia's system. 02/10/05 - PUBLIC SAFETY - RES. 040933
Since this system is so dependent on hardwire, is there anything Motorola could recommend to our communications people? Is it about money? Is there a better way to do this? Could our own internal network be a better alternative to -- how do I put it? A telephone company technician could go change one pair to another pair while he's installing a residential phone and again cause chaos to the system. Have you ever made recommendations to get away from this relationship and do it differently?
No, there really isn't in a major City -- the best way to do this is with T1 lines because -- there's other ways of doing it, but they're as reliable as T1 lines. There's always that issue that everything is 99.999 percent reliable. It's always that one piece that's not a hundred. In any kind of network that you have, there's always something that could affect the network or part of the network or a percentage of the network. So it's very difficult to say you 27 02/10/05 - PUBLIC SAFETY - RES. 040933 should be using something else versus phone lines. We find the phone lines of T1s to be the best way to do this in a major metropolitan area.
Thank you. And I want to thank you personally. It's refreshing to hear you say that you're with us and that this system is going to work well and you're not going to run from us even though you may not have any official responsibility, or at least that's what you implied.
Exactly. We appreciate you did not duck one little bit from this issue, and it's refreshing to see a company like yours participate in this hearing process. And I'm confident that you want to assure that name Motorola is serving the City well.
Can I ask one question? Have we already paid the 52 28 02/10/05 - PUBLIC SAFETY - RES. 040933 million, or is it in installments over years?
Thank you very 8 much for coming. 9 The next witness is Don Marino from 10 the Law Department and Michael Eichert. Good afternoon. State your name for the record.
Donald C. Marino. I'm Chair of the Litigation Group in the City's Law Department.
We're here today with question regarding our warranty, and that's basically what we want to know about.
Well, I think Motorola has pretty much indicated that -- the original warranty, I'm advised, has expired. But despite that, Motorola still has remained 29 02/10/05 - PUBLIC SAFETY - RES. 040933 behind and is working with the City to correct any problems that may arise.
How long was the original warranty? Couldn't have been very long. We haven't had the system very long.
Actually, I believe Commissioner James is better suited than I to answer that, but I think it expired in May of -- what? As I said, the Commissioner has more information on that than that I do. He may be suited to answer your specific questions with regard to the warranty.
Hyundai gives a 10-year warranty on a $10,000 car. Seems like we got a fairly short one. Whether the company's willing to stand it its stead or not, but I'm just talking about from an upfront perspective. I'd rather not count on their being nice. If I pay $62 million -- and maybe Commissioner James can just shed some light on that.
Councilman, it was done 30 02/10/05 - PUBLIC SAFETY - RES. 040933 in phases because we did it over a phase period of time, so different components came out of warranty at different points in time. It's a one-year warranty. When we negotiated the contract, what we had received assurance from Motorola was a 10-year commitment on the ability to get equipment and parts for the system, but we only signed up for a one-year warranty so that we stay within the bounds of looking at the warranty and be able to renegotiate it depending on what we think is appropriate for us to do.
Hindsight is beautiful. I love it. But you were there at the negotiating table. I'm sure there was trade-offs in how long the warranty would be and how much you'd pay and all that. Councilman Rizzo just mentioned it started at 52 and went up to 62 million. Where did the last 10 come in?
I'm not sure about the 62 million number. The original project was $52 million.
If Councilman 31 02/10/05 - PUBLIC SAFETY - RES. 040933 Rizzo says it's $62 million, he knows more about the system than anybody in the City, I thin, so I would defer to his numbers.
What I am interested in while you're here at the table, we have paid -- and you know the City Controller is doing an audit on this. The original price of the system was $52,767,380, which was amended in August 2004 to 54,811,406, then 3,597,713 in annual warranty and maintenance costs for three years. Now while we're having all these difficulties, do you really believe that -- I understand there are some things that break that aren't associated with our problems, but we're talking 3 and a half million dollars over a three-year period that we're paying in maintenance. Seems to me that we need to recover something here.
Well, as you recall when we had the discussion in the previous hearing, the City and Motorola at the time we were experiencing the problems, Motorola brought in additional resources that were beyond what we 32 02/10/05 - PUBLIC SAFETY - RES. 040933 considered was covered in the warranty at their cost. They also provided additional management of the system at their cost which was not part of the warranty, but they were doing that as part of their commitment to resolve the issues and the problems that we were experiencing. Again, I think they had a strong commitment to make sure that the system re-establish the credibility, that they were there, they were going to be here, if they need to bring additional people, staffing people at the police 911 headquarters, whatever they needed to do they did so at their cost not at our cost. We have not made a payment on the warranty since we started to experience the problems. We have held off making any more warranty payments and having Motorola fit the expense of this entirely on their on.
It would seem to me that whatever the warranty is -- you know, everybody would like to have a little longer warranty rather than the shorter warranty. Given that the warranty is a year, whether I like it or somebody else doesn't like it, it seems that the warranty would begin -- and this is for the lawyers probably rather than the Commissioner of Public Property. The warranty would kick in once everyone agrees that the system is running and is what you got what you paid for. If something isn't working from the day you got it the way both parties anticipated it working, it would seem the warranty would start whenever that point in time was where everybody said, "Yeah, we got it," whatever those shake-outs were, whatever those parts were that had to be replaced, we got it now and now your warranty starts. Then after that year if the company wants to stay on because of its own good name and its relationship with the City and other cities all that, that's something totally different than what they're 34 02/10/05 - PUBLIC SAFETY - RES. 040933 required to do. It seems like we're under warranty when we still don't have, I'll call it the finished product that we expected when we paid whatever number we paid, somewhere between 52 and 62 million.
I think the way the original contract was structured was as certain phases were up and accepted by the City, then you have a period of time on the warranty and then you start paying maintenance. So you have a year warranty and then after that you start paying maintenance on maintaining the system. The system was fully accepted by the City in about February of '03. The system was fully accepted. Everyone was fine. The problems really didn't start appearing until March was the first time there was an issue.
Everything was fine. Everything was working as spec. A year later after the system was fully accepted that we started to experience these problems.
Wouldn't shake-out occur in the first year rather than subsequent? When I hear it, it sounds very normal. You know, you start a new system, you have overlays and all that. Why does it work for a year and then all of a sudden --
That was part of our dilemma in trying to deal with some of the issues because the way this project was scheduled and phased is that we made sure that police were brought on last because we wanted 36 02/10/05 - PUBLIC SAFETY - RES. 040933 to make sure that there was a shake-out period and if problems would occur, they would occur when the municipal departments were up and using the system as opposed to fire and police on the system. So we brought the system up. The system performed as spec. The problems that did not occur in 2004 --
Councilman, can I add to the numbers so we can come up to $62 million.
Payments to RCC, the consultant, in the amount of 1,876.507. Tell us what that was about.
We've had RCC on the project going back to the designing of the RFP. They've been our technical engineering 37 02/10/05 - PUBLIC SAFETY - RES. 040933 firm who's been supporting us on a number components on the project. And their cost is the engineering services that they delivered to the project.
Payments all to management, the project manager in the amount of 1,648,758.
Right. Very similar to what we did when we constructed CJC, we hired a contract manager, you know, someone outside the City with the expertise to do the day-to-day contract management, and that's who we hired.
This one I should overlook, but it adds up to 62 million. V-Com for $24,500.
V-Com was a recent expenditure on the part of the Department, and this goes back into the later part of the fall of last year where because we were having those problems, there was a decision that maybe we ought to bring in an independent engineering firm just to take a look at everything we've done so far and go all the 38 02/10/05 - PUBLIC SAFETY - RES. 040933 way back to the original statement of work, look at the original documents, look at what was the design documents, look at all of the technical details. And we had a list of what we considered to be the known issues that we were experiencing problems and have an independent set of eyes kind of come in and take a look at it. It was a smaller purchase to kind of bring them in and for them to develop a report and kind of give us a road map for the future to say what we ought to do differently when we're looking at maintenance issues or we're looking at warranty issues or we're looking at enhancement as we plan to continue to enhance the system, what do we ought to be doing differently. And that's what the V-Com expenditure was for. But, again, the original -- my statement about the 52 million had to deal with what we paid Motorola for the system as part of the project. A lot of these, I did not count in.
In all due respect, though, the original price was 52, 39 02/10/05 - PUBLIC SAFETY - RES. 040933 almost 53. It was amended in August of 2004 to 55.
Does that mean we had the system with a year warranty, nothing happened during that warranty so we had to wind up paying more for additional warranty? Because the Councilman mentioned a $3 million item in August for three years of warranty. I don't understand that.
Payments for then 3,597,017 in annual warranty and maintenance costs for three years.
I'll have to get back to the Chair on that. I believe when we originally came to the project in May of 1999 before City Council, we were looking at a five-year program for maintenance as part of the approval process for the project. And I think maybe that's part of that maintenance payments over those five year periods. Let me respond back to the Chair with a direct answer to that.
I know this whole project precedes the current 40 02/10/05 - PUBLIC SAFETY - RES. 040933 Administration. That's been mentioned several times. Was it during the Rendell Administration where it began?
I know that from prior testimony that the operating departments pretty much, including Public Property, designed and worked as a team on the project that MOIS people weren't involved. I also know our IS Department was not of the same caliber 10 years ago or so go than it is now. So I'm not going to get into that. But has any point even since the first incident in March of '04, almost a year ago, have we brought in anybody from MOIS to help us or talk to us or participate, bounce things off, whatever you want to call it? I mean, these are our technical, you know, IT. This is all technology and it's pretty sophisticated stuff. They also have lightning quick communication with their peers and others in other cities who do the same thing. And I don't think I've said it at a hearing but I mentioned it at the FOP a month or so ago, a 41 02/10/05 - PUBLIC SAFETY - RES. 040933 lot of cities have combined the telecommunications and IT departments to recognize the changes that have occurred in the world recently. Whether it's by Charter restriction or because we just have decided not to do it yet, we haven't done that. But have they become at all part of the team, even added on in the last year? Or has there been any attempt to use that resource that we have to help us with this?
Well, for some of the particular problems that we've had on the radio system, we have not gone to MOIS for support on that issue. And I can't question their capabilities on the radio side. On the networking side, they have been involved in some of the issues that we've looked at in particular as they affect Verizon and the network. And the calls that -- if there are network issues, they do go through Verizon through the MOIS help desk. So from a network side, they are engaged in that particular aspect of the project. On the radio and the RF side, we have not engaged them on the 42 02/10/05 - PUBLIC SAFETY - RES. 040933 issues.
You say radio like it's real simple. This is a highly sophisticated system or we wouldn't be having the problems. The old analog we had no 7 problem with. I would suggest, just friendly advice, it doesn't cost you anything. They work with us, for us, for the Mayor that, that they at least be brought in to just bounce something off of them. It's not -- it's really looking forward not Monday morning quarterbacking going backwards. We didn't capacity there that we have today.
Are there any plans underway, any discussions pending that are designed to bring those two worlds together so that we can work more efficiently?
I'm not aware of any plans to merge the telecommunications and 43 02/10/05 - PUBLIC SAFETY - RES. 040933 communications division with MOIS at this present time. I don't know. There may be talks and discussions about it, I'm just not privy to or aware.
For my own understanding, internally when contracts of this scale come to the desk of the Commissioner, procedurally what happens internally within Public Property with regard to the final checkoff of a contract of this scale? Is there a step above the Commissioner level for review, for analysis, for authorization? Help me understand.
As far as the radio project, the radio project from its infancy was actually ran through the Procurement Department. And at the time Mayor Rendell had put together a team mainly run out of the Managing Director's Office of all the various operating departments to participate component in designing and drafting of the system. The system when it was finally -- the RFP, it went out and was again reviewed by this team. There were several teams whose job was focus 44 02/10/05 - PUBLIC SAFETY - RES. 040933 to on the technologies, a legal team look at the contractual issues. There was a number of different teams that were all part of this project. And at the final point, there was a recommendation that was sent to the Mayor at the time to say, you know, from all the teams "This is our recommendation on who we think the award should go to and these are what we thing are the issues." And once the Mayor agreed that was acceptable to him, at that time, it was brought to City Council for review. It was done by a City Council ordinance, and the contract and the proposals and the RFP were all amendments to the ordinance. It was, again, reviewed and we had hearings here in City Council and then it was finally approved by City Council. It wasn't done just by the Commissioner; it was a very exhaustive and very long process.
That's very, very helpful. So this particular contract has been in place since pre-1999.
In May 1999 I believe is when we had the Council hearing and it was 45 02/10/05 - PUBLIC SAFETY - RES. 040933 around the time that it was approved and it was shortly after that we got the contract finalized and went into construction.
Was this system paid for with tax dollars? Was it the fee that people pay on their phone bill for 911 service?
Yes, it was funded largely through the 911 surcharge and a component was paid through Water Fund and a component was paid by Aviation.
I don't know if you recall, but back then we were in a process where we had licenses in spectrum that was awarded by the FCC to new carriers, and they were required to reimburse the City for moving out of that spectrum. So we had a component of about $5 million that we used to cover the non-public safety agencies component of it. So it was kind of that component, the 911 46 02/10/05 - PUBLIC SAFETY - RES. 040933 Fund, and Water and Aviation Funds.
I believe that $5 million that we used from that covered the general obligation. We had actually accumulated about $15 million by that in our 911 fund specifically for that project. So we had kind of --
We get about $7 million that goes into Capital from the 911 surcharge. And then the remain which is like a 60/40 split between the money that comes in. The balance of it goes to paying the operating expense for police and fire for running the 911 dispatch centers.
Yes, for operations. 47 02/10/05 - PUBLIC SAFETY - RES. 040933
No, we have a capital component and then we have an operating fund, so we get about $7 million.
I think it's a million and a half maintenance on this system. Does that come out of the 911 Fund.
That's coming right not out of Operating. Public Property Operating, and there's a component that's reimbursed by Water and Aviation for their portion of it.
Thank you. I have no further questions. I think we're ready for the Law Department either publicly or privately.
We actually want the Law Department to let us know if you're ready to answer -- maybe you're not ready to answer today -- whether there is any recovery, financial recovery?
We can meet in private. This is something not usually discussed at public hearings. 48 02/10/05 - PUBLIC SAFETY - RES. 040933
But I do want to assure Council since your last hearings, we are in the loop. We have been meeting with representatives of the Managing Director's office, with Public Property, Mr. James. We've had meetings. Mike Eichert has been reviewing the paperwork, the contracts. So we're right there. And if and when the time comes, we'll be prepared to act accordingly in respect to this. As I said to Councilman Rizzo earlier, his office made a request of us with regard to information. We are preparing an answer for that and hopefully will have that to him within the next week.
Very good. We're just concerned that we get what we paid for. We all are consumers and whatever we purchase we want to make sure we get what we paid for. I have an air conditioning in on my office. And, you know, they're putting in brand-new -- you may not know, but I know that Joe James in Public Property knows. They're 49 02/10/05 - PUBLIC SAFETY - RES. 040933 putting in new air conditioners all through City Hall. And within a year, mine broke. And people keep coming to repair it. Don't we have a warranty that replaces it? I mean, that's the way I by things at home. So we just want to make sure, particularly in this time of fiscal tight-beltening that we get what we pay for. Anybody else?
I think the analogy is a good one because it's simple and it's something we personally did deal with. But I think the impact here as I see our uniformed officers in attendance, either in uniform or out of uniform, it's not just public safety that's affected but it's the safety of our own people. And I think that's the added dimension beyond the more simple get what you pay for warranty thing that we're hoping you're really involved in with them.
Just to finish up 50 02/10/05 - PUBLIC SAFETY - RES. 040933 and thank everyone. I think this process has really helped this radio problem immensely. But again, there's been Counselors there's than a significant amount of money spent by the City, Police Department, down in the radio room while there was testing going on. Lots and lots and lots of man hours put in to give support to try to come up with the solution to this problem. So there's a significant amount of time and effort. Sometimes people don't look at City employees' time is money. I do. I'm just hoping that the Law Department can goof, Motorola, Verizon, all these companies that have affected us. We really need to see what it has cost us and what the disposition is of the maintenance agreement. I don't truly understand where we have -- Commissioner James suggested that we stopped paying the maintenance agreement. But I'm sure that that's just not going to go away and you need to focus on what our obligation is to pay for a maintenance agreement or prorate a maintenance agreement during the course of these problems. 51 02/10/05 - PUBLIC SAFETY - RES. 040933
Councilman, your concerns are our concerns. And I assure you we are looking at all that and will act appropriately if and when the time comes.
We want to thank everyone for coming out. This is, I believe, the fourth hearing. And we look forward to just paying attention to what's going on. And hopefully, if the system needs improvements or continued monitoring, because we care about the safety of not just our uniformed officers but everyone. Thank you. Thank you for coming. This Committee is actually adjourned. (Council adjourned at 3:11 p.m.) - - - - 52 C E R T I F I C A T I O N I HEREBY CERTIFY that the foregoing proceedings of the Council of the City of Philadelphia of Thursday, February 10, 2005, were reported fully and accurately by me, and that this is a correct transcript of the same. RE: COMMITTEE ON PUBLIC SAFETY and TECHNOLOGY AND INFORMATION SERVICES _________________________ Lisa C. Bradley, RPR