Special Reports

It works; no it doesn’t

Tacoma’s computer installation team talked up the project’s success even before the system was turned on.

After the switch was flipped, a stream of news releases proclaimed victory. Soon, city representatives popped up across the nation, touting great technological advances in America’s No. 1 Wired City.

“Tacoma has successfully completed the most complex and diverse … system implementation of any city in the world in only 18 months,” the project team announced Nov. 17, 2003, the day the last piece of the network went live. The word went out around the world.

But for many customers and workers at home, lurking behind that civic pride was a string of problems that scarred morale and marred credibility.

Glitches surfaced – then grew larger – in payroll, in tax and licensing, in finance, in utility billing, in accounts payable and in other departments. Bugs and problems cropped up across the city over the next few weeks.

Some vendors weren’t paid. Unions ranted about payroll errors and unpaid health insurance premiums. E-mails detailing mind-bogglingly difficult transactions flew around City Hall.

Inscrutable utility bills – and angry customers spilling out the doors of Tacoma Public Utilities’ headquarters – followed.

Project leaders’ public proclamations of success and presentations about gee-whiz technology belied the behind-the-scenes scrambling to get the city’s work done, the rising costs of added consultant contracts and growing numbers of hastily hired employees to solve customer service complaints.

City Manager Jim Walton and Utilities Director Mark Crisson, satisfied with TUI Consulting’s installation of SAP software under a $29.6 million contract, signed off on final payments totaling $1.9 million in November 2003 and January 2004.

“This is a huge achievement that all the city should be celebrating,” TUI CEO Ballu Khan said Nov. 17, 2003, two weeks after the last of two major system switch-overs. The issues remaining, Khan said, were “just tweaks.” On the same day, Crisson was in equally high spirits.

“We’re really pleased with the success of the project,“ he said. “We’re on target for schedule and budget.”

But Utility Board Chairman Jake Fey now compares those proclamations with President Bush’s triumphant speech aboard the USS Abraham Lincoln long before the hostilities in Iraq were over.

“They declared victory, but what condition are we in?” he asked.

Top city leaders such as Fey have heard repeated assurances only to be disappointed by reports of problems and cost escalations. He and others now support a thorough assessment of the system by an outside consultant.

‘Rage against the machine’

Many employees felt the start-up was made unnecessarily difficult by an implementation team that rushed to beat a deadline – posted on highly visible countdown clocks at City Hall and TPU headquarters.

“I cannot celebrate this as a success,” Finance Director Steve Marcotte wrote in an Oct. 10, 2003, e-mail in response to a report from computer project spokeswoman Karen Jones. Jones is now an assistant manager in the information systems department.

“The first SAP pay checks are on the way,” her citywide e-mail announced. “Yesterday, the BSIP (Business Systems Improvement Project) Human Resources team successfully ran the first SAP payroll and transferred the file to the bank.”

“Exhaustive testing of the system revealed few, if any, errors in payroll amounts,” Jones added. But she warned there might be discrepancies in vacation and sick leave accruals on as many as 2,400 pay stubs – about 73 percent of the roughly 3,300-check run.

Marcotte’s memo said “a huge clean-up” was needed to correct all the errors.

It took a full day to run a payroll that previously was done in 80 minutes, City Treasurer Morgan Jacobson wrote in an e-mail. To get it through, workers broke it down into about 66 batches of 50 pay stubs each.

As the days passed, more employees joined Marcotte and Jacobson in voicing their frustrations. “My customer service ‘id’ is bruised, battered and scarred and I am forever changed by the experience,” facilities maintenance and repair supervisor Ron Francis wrote following a Nov. 5, 2003, meeting with the computer project team concerning his department’s problems and concerns.

Computer installation decisions affecting his department were made by a group of people “who are by all indications completely competent in their own fields and yet completely clueless about mine,” he wrote.

In a postscript to his two-page memo, Francis added, “I intend to continue to rage against the machine.”

An instant success?

Twelve days after that meeting, on Nov. 17, 2003, TUI and the city issued announcements hailing “one of the most comprehensive municipal government and utility projects in the United States” as a success.

Complex payroll calculations, Internet bill-paying, wireless dispatching of utility crews and an array of other functions were working well across the city, officials assured. The problems cropping up, Crisson said, “are addressable and resolvable.”

As he spoke, slowdowns and backlogs were beginning to bring his utility customers grief.

By late November 2003, 132 customers wanted their tardy refund checks, money owed them for overpayments, deposit returns or other reasons.

Many called. Some had been waiting since October.

New-look utility bills, meanwhile, confused hundreds of customers. Loss of the familiar energy-usage bar graph steamed others.

Phone lines hummed, then overloaded. The complaints were exacerbated when customer service agents struggled to navigate accounts using the complex and unwieldy new software.

Over at Click! network, the utility’s cable division, tensions grew.

“Even my most dedicated and knowledgeable people are struggling,” Click! Network telecom manager Dana Toulson wrote in a Nov. 21 e-mail to her bosses. “We’re up to eight weeks behind making vendor payments – but not for lack of trying.”

Her staff didn’t have the knowledge to do their jobs, she said.

Many employees, stung by a “get on board” memo from Crisson, were unsure how to explain computer-caused delays to customers.

“If I learn that an employee used the new system as an excuse for poor service and/or was critical of the new system, then he (or she) and I, together with everyone in the chain of command, will have a meeting to discuss how we can do better,” Crisson wrote in a Nov. 19 e-mail to his management team. “That’s not a threat, it’s a promise.”

The missive, soon copied, passed around the utility and e-mailed across the city by employees, was written just two days after Crisson and Khan’s triumphant declarations of success.

At City Hall, things were equally strained in the Tax & License division.

“Yesterday we actually had to resort to using a typewriter so a customer could get their (business) license certificate so they could open a bank account,” operations manager Jodie Trueblood wrote in a Nov. 26, 2003, e-mail asking for technical help.

Just 13 days later, against this backdrop of customer complaints and employee cries for help, project director Karen Larkin gave an overall upbeat wrap-up to the City Council and the Utility Board.

Yes, there were challenges, she admitted. But many had been overcome. She backed her report up with impressive numbers: 10,285 vendors were paid $121 million within a few weeks; 1,326 permits were processed; 100 crews dispatched via wireless devices handled 700 work orders a day; 131,000 utility bills were mailed; 22,500 payroll checks were issued.

The report concluded with a Dr. Seuss-style poem that read in part: “Was it perfect? Not quite. But in the end Tacoma. You got it right!”

Successes and failures

Not everyone shared Larkin’s assessment of the system start-up.

 • One police department employee battled payroll for more than six weeks after the system overpaid her. A serpentine e-mail trail makes the saga difficult to follow. At one point, the overpayment is listed as $1,022; at another it adds up to $1,577. When one solution calls for taking $788 away from her Nov. 16 and Nov. 30, 2003, paychecks – a total of $1,577.64 – she complains that’s $555.46 more than she was originally overpaid.

“I have had nothing but problems with my paychecks since we went to SAP,” she wrote. “I have had my entire pay amount withdrawn from my checking account one full week after it was deposited. Now you will be taking out a total of $500 more than you should, so I will have to transfer money from my savings to meet my financial obligations.”

 • In fleet services, which cares for about 1,400 city vehicles, employees grew increasingly worried about an unpaid fuel bill estimated between $400,000 and $500,000. “Our fuel vendor has not been paid since October,” general services director Leslie Rowen wrote to Larkin on Nov. 21, 2003. “I am very concerned that the city’s failure to pay this bill will put our fuel supply at risk.” The problem persisted into January when the city owed PetroCard of Kent $415,889 – $260,083 of which was overdue.

 • By Dec. 17, phone lines from utilities headquarters to City Hall were overwhelmed with calls from angry customers who wanted to talk to somebody about the bills they couldn’t understand.

“This sure looks like we’re testing in a live environment,” utilities customer service boss Bill Schatz replied to a December e-mail about a problem with past-due notices. “Wasn’t this tested before?? Pretty risky!!”

 • And on Christmas Eve 2003, Finance Director Marcotte wrote to Larkin that the city was overdrawn at the bank for the third time since going live with the system. At one point, the city’s account was $4.5 million in the red.

“Karen, it is abundantly clear to me that the go-live process was seriously flawed,” Marcotte wrote. “These systems were NOT ready when we turned the old ones off. Problems on this scale should have been readily detectable by any honest process intended to ensure readiness.”

Larkin and others disagreed.

When she went to the City Council in February 2004 for an additional $651,600 to wrap up the final project cost, Larkin acknowledged there had been some rocky days, but she said that would change once users adjusted.

“We’ll really be able to step back and look at the power of the tool we’ve created,” she told council members.

But as system officials talked publicly about a few glitches and minor problems, they wrote multiple e-mails about issues that needed work. Some were concerned about disappointing City Council members.

Systems Director Dave Otto warned retirement systems director Patricia Pabst in January 2004 about hitting the $50,000 mark on a consulting contract with TUI. If the bill reached $50,000, she would need to seek City Council approval, he reminded her. That’s something “I don’t think we want to do,” he added. “So watch the (consultant’s) time closely.”

Despite growing costs and continuing concerns, City Manager Walton and Utilities Director Crisson approved the consultant’s final payment in January 2004, about 45 days after the systems were turned on.

Their approval came over the objections of Finance Director Marcotte, who warned them in a memo he couldn’t fulfill his legal obligation to certify the payment as a “‘just, due and payable’ amount when we still have long lists of uncompleted tasks and system functionality remains below expectations.”

He worried that signing off would limit the city’s legal options.

Marcotte ordered the money released but made it clear he wanted further discussion of what was promised in the contract and what was delivered. He also asked accounts payable supervisor Jeff Christensen to keep a copy of the e-mail documenting his misgivings with the payment information.

Walton had no qualms “whatsoever” in signing off, he said recently.

On ‘triage’ and praise

Larkin wasn’t worried, either. The project manager returned to her assistant public works post in February 2004, a month after the installation phase officially concluded.

She worked on a series of presentations touting Tacoma’s success for trade groups across the nation. She conducted telephone interviews with prospective SAP and TUI customers and had e-mail exchanges with governments wanting to know how the city achieved its technological advances, e-mails and other city documents show.

By Jan. 28, she was confirmed as a speaker at the May 4-7 CIS Conference, a trade meeting in Miami for customer service professionals. Her mission for the taxpayer-funded trip: Tell Tacoma’s success story, with details about “enhanced business processes and productivity.”

Meanwhile, SAP’s Tom Bettes coached her on how she might help the software vendor land a contract with the state of New Mexico by allaying some state officials’ concerns about the cost and complexity of the software.

Preparations for a string of more than a dozen similar presentations – some at home, some on the road – by Larkin and others went on behind the scenes even as some city officials became increasingly worried about problems created by the new system.

Tacoma Power energy services manager Steve Hatcher detailed his department’s troubles in an e-mail to Otto, the systems director, on Jan. 26, 2004.

Hatcher’s department was in critical need of the “ability to generate timely, accurate bills” and to produce reports needed to fulfill Tacoma Power’s legal obligations, his memo said.

A draft “Tacoma Power SAP Triage Report” dated Jan. 30, 2004, detailed several issues.

“Tacoma Power’s revenue and reputation are at risk with the current billing situation,” said the report, written by a group of 10 employees and managers.

It listed four bulleted items.

 • “90 percent of commercial accounts reviewed to date are inaccurate, equating to approximately $4 million of revenue at risk.”

 • “Tacoma Power’s previous stellar reputation is tarnished daily with long lines in the lobby, extended ‘on-hold’ situations via telephone and repeat billing problems.

 • “Employees are stressed, stretched and not feeling empowered.

 • “Perception by employees is that billing issues are increasing, not decreasing and that no one sees the entire systemic dilemma.”

An $8,500 Pat on the back

As frustrations grew, project leaders decided it would be a good idea to have a return-on-investment study done to show how much SAP would increase efficiency.

Larkin wrote SAP vice president Klaus Heimann on Feb. 5, telling him she was “convinced our exhaustive … selection process paid off with the choice of SAP for our business solutions.”

The city was already telling its success story to other cities, Larkin said. The city would be willing “to have SAP do such an analysis and would ensure participation in the process,”

Learning of Larkin’s plan, Marcotte shot off his own memo to customer service manager Schatz and Systems Director Otto, saying he was worried such a study would be “highly negative” and that the system wouldn’t stand up to close scrutiny,

“We invested a very large amount of money – much more than is admitted – and so far we have degraded systems,” he wrote.

Larkin said problems were easing, employees and vendors were being paid and the city’s business was getting done.

As testimony to their contentment, Walton and Crisson gave Larkin a favorable review in March, awarding her $8,500 in incentive pay for completing the project on time and within $250,000 – according to Larkin’s figures – of the $50 million budget.

“With the possible exception of customer relations, Karen met or exceeded expectations in every category,” Crisson said in the review.

They were pleased with TUI’s work, too.

The consultant got three contracts totaling about $2.5 million for follow-up work on the system in 2004. The work was awarded without seeking formal bids from other vendors, contrary to standard practice with city contracts, because officials believed TUI was the best company for the jobs.

“I know they put in a lot of hours that weren’t accounted for,” Crisson said of TUI.

Troubles continue

As the year moved along, problems were solved on one front, only to appear on another.

A problem with overdue bills escalated as the months passed. By July, power, water, sewer and garbage accounts more than 90 days old totaled about $1.4 million – a sum more than $1 million higher than in July 2003.

Many employees not only were fed up with the system, but also were tired of being blamed for its problems.

“One issue that irritates me is the BIS (Business Information Systems) department’s insinuation that a significant part of the problems with SAP is due to employee resistance to change,” one worker wrote in the anonymous open-comment portion of a customer satisfaction survey published in August. “For my 20-plus years with the utility, I have found the organization and its employees to be very dynamic and quick to adopt new processes. SAP appears to be a system that was designed more for a manufacturing business,” the employee concluded.

In August, council members approved a labor agreement with 284 Teamsters without numbers showing the contract’s cost. The “fiscal note” normally accompanying contract resolutions was postponed by the prolonged number-crunching problems that also delayed the council’s budget deliberations.

A few weeks later, project leaders threw an Oktoberfest celebration, featuring a potluck, cake, balloons and tricycle races, to mark the German-made system’s first anniversary.

Not everyone thinks there was reason to celebrate.

“I didn’t get what I asked for. And I still don’t have what I need,” Francis, the facilities maintenance supervisor, said last week. He’s still raging against the machine. “The system works fine for some people,” he added. For others “it’s almost unusable.

“Parts of it appear to work really well, and there are people who use it all the time and swear by it.” Francis said. “I think we just didn’t implement it right.”

Tacoma resident Rosemarie Shook, meanwhile, is still bedeviled by utility account problems that began last March.

Instead of the $98 she was accustomed to seeing on her utility bill, the total was more than $300. Shook, who lives on a fixed income, couldn’t pay it. When her account became past due, the utility kicked her off its budget billing plan. As the weather grew colder and her bills grew higher late last year, she worried about how she’d stay warm through the winter.

“To me, it’s like I was getting punished for them having a new computer system,” Shook said. “In 20 years, I was never late. This year, I’m having all these problems.”