Colorado DMV wtf's in the news



  • URL: http://www.rockymountainnews.com/drmn/local/article/0,1299,DRMN_15_5538977,00.html
    Steven R. Nickerson © The Rocky

    New system to register motor vehicles just the latest to misfire for state
    By Ann Imse, Rocky Mountain News
    May 16, 2007

    Colorado officials had plenty of warning that their new computer system for registering motor vehicles had careened off track.

    Outside auditors, county clerks and state employees who issue license plates all raised the alarm. The state even briefly suspended the development contract in 2004.

    But six years passed and nearly $8 million went out the door before a new Department of Revenue director halted the system's -most recent trial run last month and brought in another consultant to see if it's salvageable.

    State and county motor vehicle staff members say they poured months into helping plan a modern computer system, only to see their efforts dropped by state officials and the computer contractor Avanade.

    "It's like you were having a baby, and it turned out to be ugly," said P.J. Taylor, head of Broomfield's motor vehicle division.

    The computer system known as CSTARS was plagued from the beginning by state and Avanade officials, who:

    • Ignored the state and county staffs that would use the new system and fired the subcontractor in charge of seeking their advice.

    • Wrote computer code before they were given detailed plans for what that code should do.

    • Disagreed over who should make key decisions, including which side was responsible for compiling detailed plans.

    • Ignored monthly reports on the mounting problems filed by SysTest, a Denver company hired by the state to monitor the project.

    The bottom line is that CSTARS doesn't work.

    Not yet. And apparently not anytime soon.

    CSTARS is Colorado's fifth problem computer contract. All five are ventures dating to the administration of former Gov. Bill Owens.

    The five deals total $325 million, with CSTARS contracted at $10.3 million. Other new systems have failed to pay road workers and welfare recipients accurately and have failed to track unemployment benefits and voter registration.

    "It's number five - something is broken at the state," said Don McCubbrey, a professor of information technology at the University of Denver.

    "As a taxpayer, it's really disheartening," said DU management professor Cindi Fukami, who with McCubbrey has studied the state's largest computer boondoggle, CBMS, which handles welfare benefits. They are using the tortured CBMS story to teach students how not to manage.

    But M. Michael Cooke, the Department of Revenue director in the Owens administration who oversaw CSTARS, says it's not comparable to the state's other problematic computer systems, which either never got off the ground or were deployed and fouled up state and county operations. CSTARS was never deployed to the counties and was being used in a trial run - parallel with the old computer system.

    CSTARS "hasn't failed," Cooke said. "It's in test."

    Problems from the start

    Interviews and records show CSTARS was plagued from the beginning by state officials and Avanade employees, who, to put it plainly, didn't listen.

    The record reflects a history of advice unheeded, problems and warnings ignored. Even the language of the signed contract was in dispute.

    Four years ago, SysTest reported nine critical and 47 major problems. On each one, it said, "We brought this to (state officials') attention and they chose to do nothing about it." During the next months, the warning was repeated again and again.

    Avanade declined to comment in detail for this story, saying it was working with the state to complete CSTARS. But Avanade did say it thought CSTARS was finished and tested, when the state came up with 27 new requirements.

    CSTARS started with high hopes in the late 1990s, when state and county Division of Motor Vehicles staff members wrote a wish list for a new computer system. It ran to 700 requirements. "It was everything the system had to do," said Sharon Wheat, supervisor of Jefferson County's motor vehicle office in Lakewood.

    That list was in the state's request for proposals in 2001. But as work proceeded, "There were a lot of requirements taken out," Wheat said. "I don't know how or why." As a result, the new computer system "wasn't what the counties wanted," she said.

    The state initially wanted to buy an existing DMV system and modify it. But instead it chose Avanade, which had never built a DMV system, over six companies that had. Many of the other six, however, ignored key requirements in their bids.

    Colorado did make it clear that CSTARS would be no easy task. Its bid request said the old computer system had been altered repeatedly over 24 years, often to allow cities and counties to add procedures unique to them. Programmers were 15,000 hours behind on program changes to the old system.

    The new system would have to deal with so many vehicle registration laws that the list alone ran 32 pages.

    Less than a year after the contract was signed, SysTest began sounding alarms in the summer of 2003. SysTest declined to comment for this story, but its reports are clear:

    • Insufficient state staffing had been assigned to the project. The state project manager, Patrick Chase, was "overburdened." Twice, managers were installed over Chase's head, and twice they departed.

    • "The project will be in jeopardy" if the state does not settle on what it wants the new system to do.

    • The state had accepted four design documents over SysTest's objections.

    • The state needed to decide who would approve Avanade's work - a changing cavalcade of project managers and chief information officers, the Project Review Board or Cooke herself.

    Cooke said SysTest's concerns were dealt with.

    Bid partner fired

    In November 2003, Avanade fired its partner on the bid, Attain Technologies of Englewood. A state letter says Attain was fired because it didn't have the working knowledge of the Colorado DMV that it claimed - the reason it was brought on board in the first place.

    Attain says that's not true.

    Broomfield's Taylor said a team of county officials had been working with Attain's -Jose Albino to detail what they needed. But their work went out the door when he did. "Not only is Jose gone, but so is the documentation."

    Jim Freeman, of Attain, said the dismissal came after "we tried to represent the state of Colorado and the county clerks. (Avanade) didn't want to hear our opinion. We felt like they used us to win that large opportunity and then had absolutely no idea what they needed to do."

    At the end of 2003, the state DMV staff went over the head of the project manager to Cooke with a list of 120 problems in CSTARS. SysTest, the outside auditor, responded that the list was relevant - but way too late - and that it was a sign of huge communications problems that could derail CSTARS.

    SysTest warned it "has never seen a successful project where the stakeholder and the project management group were at odds with each other."

    SysTest also warned that Avanade could not do its job with users disagreeing on what they needed.

    Avanade later wrote that most of the 120 issues were dropped and called this proof that they were unimportant. But Taylor, a county representative on the Project Review Board, said the complaints were serious. She was overruled when Cooke took the decision out of the board's hands, she said.

    "I thought, 'Wait a minute, she can't just ignore this' - but she did," Taylor said.

    Cooke said she did listen to the users. "If I had not paid attention to the issues raised, we would have launched 20 months ago," she said.

    In March 2004, SysTest got tough: It warned that without more time and money, the users' needs would not be part of CSTARS. Avanade estimated the cost: another $3.8 million and 11 months.

    Revenue director Cooke not only refused, she suspended the contract and insisted Avanade deliver, on time and on budget.

    In a series of blistering letters, the Department of Revenue and Avanade accused each other of fouling up the project. In fact, they disagreed on the core of the contract.

    Each insisted the other was responsible for the first step of the design: Writing the "business processes." For example, someone had to write down every step necessary to issue a license plate. No one was doing it.

    Warning audit in 2002

    A 2002 state audit had warned that DMV business processes were confusing and error-prone, and they had to be cleaned up before creating CSTARS.

    The letters show the state thought that was Avanade's job. Avanade said it wasn't.

    But, in fact, Avanade had volunteered to do this job, in its proposal. And the state made that promise part of the contract.

    Avanade accused the state of expecting a "turnkey" system, without the involvement of those who would be using it.

    "That's way out of bounds," says DU's McCubbrey. "It's like telling an architect, 'Here's $3 million, and you do it.' "

    "Most systems fail because there is inadequate understanding of the business processes," McCubbrey said. Managers must deal with these problems as they come up. "If you sweep them under the rug, they just stay there and continue to grow."

    After the rocky spring of 2004, Revenue and Avanade made up, with both promising more staffing. The new contract said the state would "verify" the design, not create it.

    "We worked cooperatively with them from that point forward," Cooke said.

    But SysTest was soon warning of dangerous communications problems again, with team members afraid to raise their concerns.

    Cooke acknowledged that the project didn't work easily. "It became abundantly clear that one size doesn't fit all" when it comes to issuing Colorado license plates, Cooke said. "What we learned is every county is different," she said. "Adams County signed off on a system that didn't work in Jefferson County."

    Weld County Clerk Steve Moreno took on the test of the new CSTARS computer system in August 2005. "It couldn't even process a license plate," Moreno said.

    According to SysTest reports, the system mangled sales taxes and fees. It took two minutes to move from one step to the next. It would process a transaction then lose all record of it. After 10 weeks, testers were still finding dozens of new problems every week. The test halted, and Avanade went back to work.

    "It was meant to be a test," Cooke said. "That doesn't mean it's a failure."

    A year later, in August 2006, the state DMV switched on CSTARS, but the counties stayed on the old system while bugs were worked out.

    Operating out of two systems soon turned into trouble, with at least four cases of motorists pulled over by police and being told - incorrectly - that their license plates didn't match their registration.

    The new Department of Revenue director, Roxy Huber, pulled the plug on CSTARS on April 2. She hired another outside company to determine whether the system can be fixed to "do what the counties need it to do."

    Even before that evaluation, the state had listed 27 changes it needed. Avanade estimates that will take 5,000 hours of work.

    And Weld County Clerk Moreno, co-chairman of the CSTARS advisory committee, says Avanade's system hasn't been tested at the county level since that disastrous -trial in 2005. New testing likely will find new problems.

    Huber also asked for an analysis of what went wrong. "A lot of folks that launched this ship aren't with us," she noted.

    Former Adams County Clerk Carol Snyder, who declined to participate in the 2005 CSTARS test, said she hopes the system can be fixed, given the huge amount of time that state and county staff members have invested in it.

    "It's been almost done for three years," Snyder said. "I hope they can work it out."

    Other state computer problems

    • CBMS: Colorado Benefits Management System, developed by EDS at a cost of $223 million. It fouled up welfare benefits and left poor people without critical cash.The federal government has demanded an $11 million sanction for money that CBMS incorrectly paid out in food stamps.

    • Genesis: Developed by Accenture under a $40.8 million contract with the state Department of Labor to track unemployment insurance. The state said the system had a 20 percent error rate. The two sides agreed to cancel the contract. Colorado paid Accenture $27 million.

    • SCORE: Statewide Colorado Registration and Election system, developed by Accenture under a $10.5 million contract. When Accenture missed a federal deadline for completion of the new statewide voter registration database, the secretary of state canceled the contract and hired a different company to do the job.

    • ERP: Enterprise Resource Planning system was developed by SAP at a cost of $38 million to track Colorado Department of Transportation finances, personnel and project management. About 3,300 employees had their payroll checks messed up up last winter. The system continues to be altered and staff are getting extra training.

    Issuing a Colorado license plate

    • Step 1: Who are you? The computer must determine from the address of a buyer what percentage of the sales tax goes to the state, county, city, RTD and school, fire and sewer districts.

    • Step 2: Where are you? Tax districts have made their own rules, too. One Jefferson County district, for example, requires sales tax only if the seller and the buyer live in its boundaries.

    • Step 3: What are your qualifications? For some specialty plates, the staff must check whether the applicant qualifies. A disabled veteran plate and a Bronze Star plate require proof, but the list of vehicles that can carry these two plates is different. For some specialty plates, a share of the fee may go to a charity.

    • Survey says: The process is so complex that a 2002 state audit of the existing computer system found errors in 35 percent of the transactions.

    imsea@RockyMountainNews.com or 303-954-5438

    Copyright 2007, Rocky Mountain News. All Rights Reserved.





  • I, for one, am proud to see my tax dollars give me entertainment on WorseThanFailure.com.



  • I think the biggest WTF in that article is that Accenture was hired a second time after having already fucked up once!  I mean, seriously - if the $48 million project was the one they fucked up first, that's just jaw-dropping.



  • That has to be the longest sidebar ever--if not, it's close.  And it was worth reading every word.



  • An instant classic.

    I'm not sure who is to blame in all that, but it's clear one or more persons clung fervently to the "good enough for government work" approach. In general, the "just good enough to not get us sued" attitude prevailed.

    If I were an Avanade employee involved with this, you can be sure I'd be calling the press (probably on condition of anonymity) to tell them about the circus of apathy that's been going on behind the scenes.



  • Bonus points for using the word "boondoggle".



  • @Morbii said:

    I think the biggest WTF in that article is that Accenture was hired a second time after having already fucked up once!  I mean, seriously - if the $48 million project was the one they fucked up first, that's just jaw-dropping.

    You have to realise that Accenture's specialisation is "winning contract bidding processes". They don't even employ people who are really good at developing systems, they just focus on winning the bids. In the field they are in, you do not get paid based on the quality of your product (because at the point when you get paid, the product does not exist) - you get paid based on the quality of your bid. So Accenture are infamous for optimising this to its ultimate form, and putting all their energy into the bidding process, and to hell with the rest of it.

    They almost never deliver anything that works. Their projects are usually cancelled or handed over to a third party for completion.

    Winning the same contract twice is impressive even for them - but it is the field that they focus on. They probably delivered a really impressive bid for the second round.

    (Yes, governments are usually buying promises of software instead of actual software. This is not supposed to make sense, it's supposed to funnel tax money into the pockets of political allies. The product is not the point.)
     



  • You could just put the link there



  • I have no comment on the story (still reading it) but: 


    "It's like you were having a baby, and it turned out to be ugly," said P.J. Taylor, head of Broomfield's motor vehicle division.

    Obviously PJ Taylor has never had a kid. One's kids are never ugly.
     



  • The Real WTF is this isn't on the front page.



  • Has anybody here ever worked on a multi-million dollar government contract that went perfectly?  Or even close?  Government software project handling seems to guarantee everything will end up in a mire.  This is the fault of everyone involved: the bidders, the RFP writers, the gov-side oversight (if any), the primes, the subs, everybody.  No one wants to talk, or at least no one wants to listen to each other. I don't know what it is that makes it more likely that private companies requesting similar levels of project (ERP?  Complex, but understood) will succeed and while governments are doomed.

    I've done my share of state and municipal contracts.  I'll take even antagonistic private ones anyday over those. 



  • @aquanight said:

    The Real WTF is this isn't on the front page.

    Could you imagine the war cry of the masses?  "Alex, today's WTF is way too long!  The new name sucks!  I'm never visiting again!"

    Perhaps it could have been a "Virtudyne 2", another classic multi-day WTF.  Too bad we already know the story!




  • @aquanight said:

    The Real WTF is this isn't on the front page.


    The real WTF will become apparent when the Rocky Mountain News issues a takedown notice under the DMCA. 

    WTF does the poster think hyperlinks are for? Decoration?



  • The newspaper has a WTF of their own: a large portion of their article criticized Avanade, yet there are no quotes from any of the consulting companies or even the obligatory 'they did not comment on the story'. It sounds like the Colorado DMV played a significant role in the project failure as well through conflicting designs, etc.



  • @poochner said:

    Has anybody here ever worked on a multi-million dollar government contract that went perfectly?

     

    Yes.  I have. 



  • Best Quote: "It was meant to be a test," Cooke said. "That doesn't mean it's a failure." 

    This is directly after a paragraph saying stuff like "It would process a transaction then lose all record of it."

    This is truely worse than failure.  And I agree, its front-page worthy.



  • @kaekae said:

    Best Quote: "It was meant to be a test," Cooke said. "That doesn't mean it's a failure." 

    This is directly after a paragraph saying stuff like "It would process a transaction then lose all record of it."

    This is truely worse than failure.  And I agree, its front-page worthy.

    If there was a worst WTF award, this would win the one for the year. & yes, this should've been on the front page, now if he only sends it in to Alex.


Log in to reply