Tacoma is paying millions more than expected and getting less than promised for the sophisticated new computer system it turned on 15 months ago.
What was billed as a $50 million state-of-the-art system will cost at least $74 million by the end of this year, city records and financial documents show. Instead of allowing the city and its utilities to shrink their work force, they added more than 60 employees in 2004 to run the complex SAP system and cope with problems it created.
It’s as if the city traded in the old Nissan for a high-performance, customized BMW. But nobody was clear that the luxury car would cost more to insure, maintain and gas up.
And many are disappointed in the new ride’s performance.
Never miss a local story.
Workers in some departments still fumble with the system and await expected features. Some frustrated City Hall employees bypass the expensive computers and do their calculations on Microsoft Excel spreadsheets or on paper. Many still need training and instruction manuals.
The officials in charge point to all of the technological tools now available, including mobile dispatching of work crews, online utility bill-paying and a network linking city departments.
“I think our overall assessment … is that it’s working well and that it has met the expectations that we set in the beginning,” project director Karen Larkin said in a recent interview.
As far as costs, Tacoma Public Utilities Director Mark Crisson says the $50 million covered only installation. Project leaders say they always expected ongoing maintenance and operations costs after the system launched.
But the City Council bought the SAP software system without knowing those costs. After start-up, council members were surprised by the need to hire more employees and for $2.5 million in additional contracts with TUI Consulting, which had already been paid $29.6 million to install the system.
“If we’d been more aggressive in hindsight about developing the budget and raising the flag (about the ongoing costs), we would have been better served,” Crisson said.
He, Larkin and City Manager Jim Walton see the system as a long-term investment requiring money and effort up front.
Crisson and Larkin dispute The News Tribune’s $74 million estimate. They put the costs of installation costs and the first two years of operation at $67 million. They also charge that the newspaper exaggerates the system’s problems and underreports its successes.
But some users disagree.
“It’s still a struggle … to get things right,” said one water utility employee who works with many system users. She asked not to be named for fear of being reprimanded. “All we see is a system that doesn’t work correctly, that can’t adapt to the way we do business. And they keep bringing the TUI folks in. It’s frightening. We have this huge system and no one knows how to run it.”
City Councilwoman Julie Anderson, tired of hearing complaints from “hardworking, dedicated employees,” wants a thorough assessment of the situation.
The system “is not good enough for a government that’s in a financial crisis. And it’s not good enough for a government that’s counting on efficiencies,” she said. “And it’s certainly not good enough for employees who are having to work harder to do their jobs a year after the system was implemented.”
THE $50 MILLION QUESTION
On the night the City Council approved the project in June 2002, Larkin promised a computer system that would set Tacoma apart from other cities. And $50 million was the bottom line, she said.
That price was favorable in comparison with what other municipalities spent on new systems, she said.
“We do not feel that we have underresourced this project,” she told council members before their unanimous vote to proceed. “We are not asking you for a $50 million project approval for a project we think will ultimately cost more.”
She did tell council members that city and utilities departments would need “to dedicate additional resources for training, for testing the systems, for converting data” and other tasks.
In presentations both before and after the vote, Larkin mentioned ongoing operating costs. But the millions the city would spend on system technicians and programmers, licensing fees and consulting contracts in the two years since the switches were flipped in late 2003, weren’t raised the night the council approved the $50 million project.
As a result, the $50 million figure became a key measure of the project’s success.
Mayor Bill Baarsma said recently that he’d never heard the $67 million number until The News Tribune asked him about it.
“I don’t recall that figure ever being told to me,” he said. “I don’t recall that figure ever being mentioned.”
During the 18 months of the installation and post-start-up period – from July 15, 2002, through Dec. 31, 2003 – Larkin repeatedly assured council members that the project was on time and within the $50 million budget, meeting minutes and tapes show.
But $50 million was not the entire cost of buying the new system.
To start with, the city paid $4 million for consultants, site visits and other expenses in a three-year vendor-selection and contract negotiation process. SAP software, customized and installed by TUI Consulting, won out over 27 other proposals.
The $4 million figure isn’t discussed in connection with the installation. It appears in only one place among thousands of pages of documents studied by The News Tribune. It was used as a parenthetical aside at the bottom of a script in an August 2002 presentation to employees by then-City Manager Ray Corpuz. “NOTE: Past costs 99-02 were $4 million if anyone asks.”
In a videotape of the Corpuz presentation at an employee meeting, the figure didn’t show up on the overhead screen. And no one asked.
The $50 million also doesn’t count training time, overtime and other expenses during the installation. More than 100 city employees, for example, served as liaisons between the project and their departments during implementation. When the project began in July 2002, its leaders estimated those liaisons would spend about a quarter of their time on the computer project over the next 15 to 18 months, though no records exist of how many hours they worked on it.
Their project work time was charged to their departments and not accounted for in the implementation budget, said Karen Jones, assistant manager of the Business Information Systems Department, which runs the computer system. She also served as the spokeswoman for the project.
The city didn’t keep a total of how much it paid in salaries to employees for the hours some 1,700 computer users spent in training. But utilities customer service manager Bill Schatz said his department spent nearly $300,000 on people to relieve workers who were learning to use the system during 2003.
Also not counted was the tab for new computers needed to replace existing units in some departments, equipment that was needed to run the powerful software. The city placed those costs in the capital budgets of individual departments.
It’s not unusual to leave “soft” costs – such as time spent in training and departmental overtime – out of a project budget, said city finance director Steve Marcotte. Nor is it unusual, he said, to have departments buy system hardware out of their own budgets.
Tom Miller, chief information officer for the State Department of Personnel, agreed that both training time and the purchase of computers could be charged off “as part of the cost of doing business.” But he added that if the purpose of the budget “was to truly represent the total cost of a project, then you’d want to factor them in.”
The city’s computer experts say the stabilization period, during which users become more comfortable with the system and unexpected problems are dealt with, lasts 18 to 36 months after start-up.
As the installation neared completion, project director Larkin, utilities director Crisson and others periodically gave upbeat progress reports to the City Council, stressing that the plan was on time and on budget. They didn’t tell council members or the public that it would cost $17 million more to maintain the system in 2004 and 2005, copies of reports, minutes and tapes of study sessions and meetings show.
But they knew it would take more money.
“We talked about not just initial costs but ongoing costs when we picked TUI/SAP,” Larkin said. “I think we probably could have done a better job of saying, ‘Now, remember, we don’t know what it’s going to look like, but we are going to have an ongoing cost of operations and maintenance.’”
In hindsight, both Crisson and City Manager Walton said a bigger implementation budget would have helped.
But they differ over whether it would have been wise to delay the go-live dates until both the system and city workers were ready – a tactic some project team members argued would be prudent.
Postponing the start-up three months would have cost roughly $6 million in worker salaries, consultant costs and other expenses, Larkin said.
As the project’s countdown clocks neared zero at City Hall and utilities headquarters, the team said, “‘We’re driving toward a deadline and we want to make it.’” Crisson recalled. And in some ways, people just needed to get on the system to learn how to use it, he added.
Walton believes spending more time and money on the installation might have prevented some of the problems that have plagued workers over the last 15 months.
“In retrospect, I wish we would have bought more time,” the city manager said. While that would have meant asking for more money, it would have “been better than what we’ve gone through.”
City Council members were surprised Nov. 4, 2003 – one day after the second of the two major system go-live dates – when they learned it would take dozens more people and millions more dollars to run the new system in 2004 than it took to maintain the old ones in 2003.
In one pointed exchange, Councilman Mike Lonergan grilled systems director Dave Otto on how a $7 million information technology budget for 2004 morphed into a $15 million spending plan. Lonergan said he understood how a budget estimate for the newly created department to run the SAP network might have been off, but he didn’t think such a discrepancy would be twice the estimate.
Otto replied that his department – then scheduled to increase from 37 to 97 workers – would also maintain Internet accounts, e-mail and a number of other computer functions not related to SAP.
Council members were told early on, he pointed out, that it could cost between $5 million and $8.9 million a year to maintain SAP once installed.
Lonergan told Otto to think of taxpayers and ratepayers as if they had “WIIFM” – “What’s in it for me?” – tattooed on their foreheads.
For a $50 million initial cost and $15 million a year in operations, “we better do a much, much better job” of telling taxpayers and ratepayers how they’ll benefit,” Lonergan added.
Budget numbers aside, Councilman Kevin Phelps wanted more than vague promises of efficiencies.
“Lacking (specific) goals … I can only be led to believe that we’re just increasing the size of government,” he said.
When Otto returned with a modified proposal two weeks later, Councilwoman Connie Ladenburg asked him: “How is hiring an additional 40 people going to save us money?”
The Business Information Systems department created to service the new network began 2004 with 92 employees – 55 more than the previous information technology department. Its $20.9 million first-year spending plan was $7.3 million higher than the department’s previous budget. But Otto said the net gain was only 27 employees because others transferred into the department.
The department’s 2005-2006 biennial budget is $31.6 million.
Although those costs include new licensing agreements and consulting contracts, pay and benefits comprise the bulk. Salaries range from $40,165 to $135,533 a year with an average of about $78,000.
It cost about $8 million more to maintain the new system in 2004 than it did to run the 102 separate systems it replaced in 2003, said Jones, the project spokeswoman. The budget for 2005 is about $9 million more, bringing the total cost of the system to about $67 million through the end of this year, she figures.
But the city’s accounting doesn’t include the 33 temporary employees Tacoma Public Utilities brought on to cut down on service delays caused by the new system. That cost the utilities $722,228 in 2004, said Schatz, the utilities customer service manager. Another $593,827 is budgeted for the 15 temps he expects to need in 2005.
The $50 million computer project budget also didn’t include $2.5 million paid to TUI Consulting for additional work in 2004, including:
• A $1.7 million “stabilization” contract to help nurture the system through its first year. The deal was approved by the City Council on Dec. 16, 2003 – a month after project leaders declared the conversion an on-time, on-budget success.
• A $472,500 contract awarded in October 2004 to make fixes and enhancements to the system, so it would serve customers faster.
• A $281,240 deal approved by the Public Utility Board in April 2004 to solve problems that brought hundreds of complaints and jammed phone lines to the customer billing system.
• A $50,000 deal to work on the city’s retirement system.
COST OF OWNERSHIP
Otto and Crisson both cited studies showing Tacoma’s maintenance and operations costs as being in line with those of other cities and businesses using “enterprise resource planning” (ERP) – software that integrates all of a business or a government’s operations.
Otto relied on one Forrester Research study that puts annual computer maintenance costs at about 20 percent of “upfront” costs. That would work out to roughly $10 million a year in Tacoma’s case, putting its expenses below the expected cost.
But a 1999 article in Computing Canada, a biweekly industry newspaper, “Beware of Costs Lurking in ERP,” uses the study to make the point that managers typically underestimate the ongoing costs of keeping a system running.
“It’s far easier to set up enterprise-wide applications on budget than it is to understand the costs that creep up long after someone throws the switch,” the article states.
The 20 percent figure “is reasonable” for such installations, said Paul Hamerman, vice president for enterprise applications at Cambridge, Mass.-based Forrester Research Inc. But the Forrester study Otto cited is out of date, Hamerman said.
A second study, quoted by both Otto and Crisson, concluded it costs many companies $53,320 per active computer user to install a system like Tacoma’s and run it for two years. Multiplying that yardstick times the city’s 1,700 users, the total would be more than $90 million, Crisson said. They credited the information to Meta Group, an information technology research company based in Stamford, Conn.
But the “total cost of ownership” measure, derived from adding system installation plus two years of maintenance and operation expenditures, “is rapidly losing relevance” for a couple of reasons, including an “inherent flaw in the TCO itself,” according to a more recent Meta Group report released Jan. 4.
Total cost of ownership is only half of a “viable (return on investment) equation,” the report said. It also pointed out that the ownership figure is based on a period far shorter than the 15- to 25-year lifespan of enterprise-wide systems.
Otto, who provided the research to Crisson, who then used it in meetings with The News Tribune, had not seen copies of either of the studies. He said he got the information from “Googling” on the Internet after questions arose about the system’s ongoing costs, nearly a year after the system went live.
LESS THAN SUCCESS
Many employees believe that while the project’s champions understate the costs of the new system, they also overstate its success. True, complaints from customers, vendors and workers died down since the first six months of the project, but The News Tribune has documented continued troubles that go beyond inconveniences.
• The system wouldn’t cough up crucial labor forecasts and spending reports as city officials prepared the biennial budget last summer. An embarrassed SAP sent a technological “SWAT team” to help get the job done after The News Tribune reported budgeting problems in August.
Council members finally got a proposed budget Oct. 23 – at least a month behind schedule. The delays shortened the amount of time they – and the public – had to consider the city’s $356 million general fund spending plan for 2005-2006. The delay also lessened the amount of time they had to weigh nearly $30 million in budget cuts.
• Complexities in the protocol for paying the city’s contractors continue to create problems for users – and delay customers’ payments. A year after project director Larkin told council members that the issue was among the “things that we saw immediately on go-live that have already been taken care of,” one contractor threatened to walk off the job because the city owed his company $1 million in back pay.
Larkin says the payment was delayed because the system’s rigorous standards for releasing money weren’t met – a sign the computer system is exercising good stewardship over the city’s money.
• Three fire department clerks spend a total of 11 hours every other weekend – on overtime – to calculate payroll by hand because the system can’t deal with firefighters’ varied schedules. The cost of that overtime since going live is about $19,000.
• The city recently spent nearly $200,000 to buy training software to help employees navigate the system better. Many workers still can’t comprehend the complex software and don’t have manuals to help them decipher it. The SAP software was so heavily customized for use in Tacoma, manuals would have to be written on site. Spokeswoman Jones says online training software takes the place of manuals, but some employees complain that’s not enough. The city is replacing that training software because it is inadequate.
• Efficiency is way down in the city’s building and land-use permits division, assistant public works director Craig Sivley recently reported. Only 5 percent of single-family permits were reviewed within two weeks of submittal during 2004 – down from 88 percent in 2003. Commercial plan reviews fell to 40 percent within eight weeks of being submitted during 2004, down from 86 percent in 2003. The labor-intensive SAP software takes longer to process the work, Sivley said. His report estimates $600,000 has been lost because of inefficiency.
• A 2003 financial audit of Tacoma Public Utilities light and water divisions was delayed several months because of problems with the system. Although auditors gave the utility a “clean” audit overall, they pointed out many cases in which the computer system or its confusing operation led to problems. They cited weak internal controls over the system – initially some employees couldn’t do their jobs because they didn’t have access to the system, and that later too many employees were given access. They noted the billing problems, unread meters and late fees that were charged incorrectly. They found cases in which employees weren’t properly trained to do their jobs. Many of the problems are either fixed or on track to be resolved soon, officials said.
Larkin and Crisson acknowledge the system is complex and challenging to learn and that they need to do a better job of training.
But it will grow more powerful, they promise.
Crisson points out that the labor-intensive steps required of some workers will pay off in the long run. Once a customer’s information goes into the system – in building and land use, in tax and license and in utilities – every worker who needs the data has access to it, he said. Those keystrokes need to be done only once – not three times – unless something changes.
And the SAP software is so powerful, the system will grow with the city’s needs over the next several years, he adds.
As more workers become comfortable with the software, project leaders say they’ll discover nuggets of information about how to do their jobs better and bring more efficiency to city government.
That’s already happening in some departments.
Line electrician Jan Chamberland loves that he can climb into his truck each morning, plug into his wireless laptop and get his orders for the day along the way. Before the new system, he had to go into the office to get his orders on paper. Now, they arrive throughout the day. When he finishes a job, he types in a few commands and a notice of completion goes into the system.
“With this system, I’ve done up to 35 locations (of utility lines) a day,” he said. He previously could do 12 to 20. “It’s made us more efficient,” he added.
But in other departments, where employees still struggle with the system to get their work done and wish they had more and better training, they believe the aging systems they gave up were superior to the “revolutionary” new network they got.
For example, the fleet services department that cares for some 1,400 city vehicles recently asked to opt out of the system and get specialized software because SAP couldn’t deliver the car maintenance and repair reports its workers needed to do their work.
Systems director Otto’s December status report made it clear that problems remain.
“This past year we have been very busy with system stabilization and support activities and focused our support in fixing as many problems as we can, as quick as we can,” Otto wrote.
He also reported that some work, including reports that must be filed with the Federal Energy Regulatory Commission, remains “very labor-intensive for finance staff.”
And some “drill-down activities” in utilities customer service were reduced to streamline workloads. The ability to “drill down” into computer data to give users more information about a customer was originally portrayed as one of SAP’s great assets.
“The fact is the project has yet to achieve its objectives and until it does it is not a success,” said Ian Campbell, president and CEO of Nucleus Research, a Wellesley, Mass., company that’s done extensive research into computer installations and SAP applications.
Mayor Baarsma says he’s frustrated – and that he doesn’t believe the system is living up to its billing.
“This whole project has been far more complex, difficult and more surreal than I imagined,” said Baarsma, who thought during budget negotiations last fall that he’d be able to “just pop onto the system and learn where we are.”
But the mayor admits he couldn’t do that. He had neither the training nor the patience to squeeze the figures he wanted from the complex system.
Baarsma also believes that the no-more-than-$50 million budget promise caused problems along the way.
“It’s very difficult to come back (to the council) and say, ‘Uh, oh, it’s going to cost more,’” he said. “But it would have been better, probably.”
Now, Baarsma said, the council and the city’s taxpayers want to see results, not promises.
“We were told … over time we were going to see miraculous savings and efficiencies,” he said. “The citizens are paying for it, and they are expecting to see more efficient delivery of services,” he added. “In some areas it’s working, but it’s got a long way to go.”
Jason Hagey: 253-597-8542
Kris Sherman: 253-597-8659