By Erin Sherbert
By Howard Cole
By Erin Sherbert
By Erin Sherbert
By Leif Haven
By Erin Sherbert
By Chris Roberts
By Kate Conger
It is about two weeks before election day, and San Francisco School Board President (and re-election candidate) Mary Hernandez has just explained why she is too busy to participate in a long-scheduled interview, leaving Cathi Vogel to explain everything else. And Vogel, the school district's chief financial officer, is doing her dead-level best to put a fresh face on an ugly reality: After five years of trying to install computer software meant to seamlessly integrate all of the San Francisco school district's financial and human resources information, nothing coming out of district computers can be trusted.
Vogel looks as if she has smelled something foul, but is too polite to mention it. "The focus I've taken is not to look back on the past," she says, "but to do an assessment of where we are right now, and what we can do about it."
Right now, the district is essentially nowhere, and it's entirely unclear what can be done.
Read the feature story sidebar: Software Hard Times.
"Look," Vogel explains, "if you go into the system and type in something as simple as the district's name, the district's name could be in there 25 different ways. That similarly could occur if you go in and look up my name. I could be under "Cathi Vogel,' under "C. Vogel.' I could be under "Anne Catherine Vogel' -- and it thinks those are all different people."
The school district is not the first client that bought PeopleSoft Inc. software and then struggled terribly with an installation. Far from it. In various incarnations, PeopleSoft products have helped failing college students escape mandatory expulsions, forced a large corporation entirely unconnected to the Disney Co. to cut regular paychecks to Mickey Mouse and Donald Duck, and mangled one public school district's payroll so spectacularly that the local teachers' union picketed in response. PeopleSoft has even been sued under federal racketeering law over arrangements surrounding the installation of its software.
But the San Francisco school district's computer mess is not just another in a line of PeopleSoft follies. Even if PeopleSoft's troubled implementation programs often exceed deadlines and budgets, most of them have eventually resulted in software that is at least partly functional and somewhat dependable.
While public records show that the San Francisco Unified School District spent five years and more than $5 million, it now has a system that is all but useless, because, in the SFUSD, PeopleSoft found a client with the muddled, conflicting leadership and indifferent, unquestioning oversight necessary to grab onto PeopleSoft flaws -- and magnify them tenfold.
Pleasanton-based PeopleSoft, founded as a human resources software maker in 1987, is today considered -- along with Oracle Corp., SAP A.G., Baan Co., and J.D. Edwards & Co. -- one of the leaders in the business software industry. More specifically, PeopleSoft specializes in "enterprise-resource planning" via complex packages of software designed to automate and integrate traditional "back office" functions, such as accounting, finance, and human resources. When functioning properly, these systems can make a business more efficient. That said, because all businesses are organized differently, an ERP software package usually has to be custom-tailored to each business that installs it, often by consultants. Because of that, ERP vendors usually acknowledge that their software is only as good as the way it is implemented.
PeopleSoft has built a huge business in ERP applications. It employs more than 7,000 people and reported $1.4 billion in revenues last year. And as the initial ERP market -- Fortune 1000-class corporations -- has become saturated, the company has developed applications to manage other enterprises, including college campuses, offices in the public sector, and health care providers, to name a few. Buzz over the company's latest release, PeopleSoft 8, pushed its stock prices up to $50 a share earlier this month, its highest total since early 1998. (The stock was at $38.81 at press time.)
In October 1995, however, the San Francisco school district bought the earlier and much more troubled PeopleSoft 6.0.
"The logic behind buying a new computer system was sound," recalls Jill Wynns, a school board member who was part of the unanimous board vote to accept former Superintendent Waldemar "Bill" Rojas' recommendation to purchase software from PeopleSoft. Indeed, the district had little choice but to shop for a new system; it was then running a system it built during the 1970s that was not Y2K compliant. But the board apparently was not told that the $283,000 human resources/payroll software package Rojas recommended had been designed with commercial enterprises, rather than public school districts, in mind.
Wynns says the board asked "the right questions" when Rojas' office pitched PeopleSoft to it, but didn't get honest answers. Specifically, she contends, Rojas and his chief financial officer, Bill Coleman, didn't disclose that the software would require extensive consulting costs to implement and that PeopleSoft had no legal responsibility to make sure it worked properly. (In August, a district technology consultant described this arrangement as "drop and run" -- which apparently marked the first time the school board was informed of the one-sided nature of the PeopleSoft contract. Repeated attempts to contact Rojas and Coleman for comment were unsuccessful.)
"What that means," says Wynns, the board's most outspoken -- some say only -- critic of wild spending during the Rojas regime, "is that you buy this software, and the people that sell it to you have no obligation to make it work. They tell you when you buy it. And Coleman used to say, "We've known all along that we'd need consultants to make this work, it's part of the package.' But we were never told that in the beginning. So, when we approved that one contract, we were actually committing ourselves to something like $4 million.
"Maybe Coleman and Rojas thought we had the expertise in-house to install it, but that would have been pretty ridiculous, considering the level of technology we had."
Actually, consultants weren't used "all along." Coleman and Rojas let district staff spend 1996 trying to install the PeopleSoft package. By the beginning of 1997, however, the district was in the market for a consulting firm to help it with the implementation.
The firm chosen, the Sacramento-based Carrera Consulting Group Inc., is a PeopleSoft "certified Alliance Partner," which means it is on a roster of consulting firms that PeopleSoft endorses to install and modify its software. But these "partnerships" -- the subject of a major lawsuit PeopleSoft settled last month -- have not always ensured that the consultants who show up to install the software are experienced and qualified to do so.
In June, PeopleSoft sent a team of six of its own consultants to assess the state of the San Francisco school district's PeopleSoft systems, free of charge. What the team found, according to a sharply worded report, was a system in shambles.
"Currently the district is barely able to use the data, despite spending over five million dollars in software and implementation costs," the PeopleSoft consultants wrote. "In 1998-99, the district was unable to close its books in a timely and accurate manner [because of software problems], and the 1999-2000 financial records are in a state of chaos. ... Few modules [of the system] are working as intended, and a significant portion of the district's financial records are being compiled and managed on other databases."
Not only did the report acknowledge the software's failings, it also referred to the people allegedly responsible for those failings: the consultants who implemented it, and the district administrators who worked with them.
As it turns out, the consulting group PeopleSoft criticized, but did not name --Carrera Consulting -- was recommended to the SFUSD by PeopleSoft through a marketing arrangement based in part on payments that Carrera was to pass on to PeopleSoft. Carrera, which specializes in public-sector software installations, remains a "certified PeopleSoft Alliance Partner," according to the software maker's Web site. That distinction was a prominent aspect of Carrera's bid to the school district, which, at $1.3 million, was the second-cheapest of three submitted to install the personnel and payroll applications. But in a Sept. 17, 1997, memo, Glenston Thompson, an official in the school district's business office who headed up the disastrous financial software section of the PeopleSoft project, recommended Carrera's bid as best, referring to Carrera's "alliance" with PeopleSoft in the very first paragraph. The board approved Thompson's recommendation with (as was often the case in an administration that once spent $35 million on consultants in a single fiscal year) very little discussion.
And so a brutal relationship began: During the 18-month duration of its three contracts to install PeopleSoft applications at the school district, Carrera collected about $2 million and walked away from the installation at a juncture when district officials -- their computers badly malfunctioning -- insisted it was unfinished.
And then, in what can reasonably be called an unusual turn of the screw, the consulting firm sued the school district for more than $610,000.
In its lawsuit, Carrera complains that the district unjustly ended two of its contracts in January 1999, almost three months after the firm stopped working on district projects. "SFUSD did not give Carrera an opportunity to cure the alleged failure to complete its work," the lawsuit reads.
Now, more than two months after the suit was filed, the district still has not countersued Carrera. David Campos, a city attorney who works in the SFUSD's superintendent's office, says the district is exploring its legal options but refuses to elaborate.
In correspondence with school officials, Carrera executives have complained about the district's lack of staffing and poor working conditions, charges supported by district employees. But those same employees also took issue with Carrera's performance.
"The consensus was that they were a bunch of amateurs," says one former district employee who worked on the PeopleSoft project. "They were all rookies, except for one or two experts. ... The district didn't really have the resources, but Carrera definitely didn't have the talent."
E-mail correspondence between district employees during the implementation seems to confirm that judgment. In one message, a budget office employee, Jim Villaluna, describes a consultant to some of his superiors: "She has not produced anything of consequence for Position Management [one of the implementation projects]. You are paying her a substantial amount, and yet she's using the district's time to learn more about being a consultant. ... In the interest of the district, you guys should do something about this."
"This," it seems, had happened elsewhere.
It's not clear exactly when W.L. Gore & Associates got fed up with PeopleSoft, but it's probably safe to bet that it happened around the time that consultants endorsed by the company were unable to stop the new Gore payroll system from cutting valid paychecks to Mickey Mouse and Donald Duck.
Several business publications found the cartoon-character paychecks funny enough to write about (Mickey and Donald were entered into the system as a demonstration, and then couldn't be deleted for more than a month), but Gore -- the billion-dollar Delaware company that manufactures the high-tech fabric Gore-Tex -- was not amused. It sued PeopleSoft and its implementation partner, the consulting arm of Deloitte & Touche LLP, alleging fraud, breach of contract, and violation of the federal Racketeer Influenced and Corrupt Organizations act.
The events chronicled in the lawsuit, which was settled out of court in October, parallel the events at the SFUSD. According to the complaint, Gore:
The Gore lawsuit also charged Deloitte with using Gore's implementation as a "training ground," in which consultants routinely needed to consult PeopleSoft's telephone help line, a charge that echoes district complaints about Carrera's "rookies."
The lawsuit claimed that "PeopleSoft benefitted ... by retaining partnership fees from Deloitte [worth tens of thousands of dollars annually] for listing Deloitte as a competent partner."
A PeopleSoft spokeswoman, Michelle Loesch, says Alliance Partners do pay fees to PeopleSoft, but eligibility to be a certified PeopleSoft consultant hinges first and foremost on a positive track record with PeopleSoft implementations. She adds that the company announced more rigid certification requirements -- including tests for individual consultants who work on implementations -- at a recent conference of its users.
As Gore learned, PeopleSoft software often comes with no guarantees. Sometimes this is actually spelled out in the contract, such as in the $10.7 million agreement signed by the Houston Independent School District in 1998 that read: "The delivered functionality in PeopleSoft may not handle K-12 through modification."
And if the HISD, using PeopleSoft's own consultants and not an Alliance Partner, was able, eventually, to successfully install the software, other school districts have been less fortunate. The Elgin (Illinois) school district used a PeopleSoft-endorsed consulting firm to install software, and when the District's Oct. 31 payroll included 231 paycheck errors, school officials heralded the news as progress. The previous paycheck cycle had included 352 erroneous checks, and direct deposit camouflaged many of the errors, causing teachers to bounce checks and miss mortgage payments.
Several hundred employees even staged a full-fledged sign-wielding, line-toeing "informational picket" over the computer problems.
Larry Ascough, spokesman for the 40,000-student district, says that consultant costs on the $750,000 software have already surpassed $1 million, with a new batch of consultants collecting $5,000 a day to try to get the system functioning. The system also fouled up the district's retirement benefits enough to draw a $14,000 fine from the state. Plus, the district was recently forced to spend $36,000 on temporary accounting help to deal with the mess. The year-old project has already exceeded its $1.7 million budget, with no end in sight.
Although PeopleSoft does not make software specifically for public school districts, it does sell software designed with college campuses in mind. These programs are supposed to manage everything from financial aid to admissions to registration, which means that, when bugs arise, they can damage a university's operations in countless ways.
The Jan. 7, 2000, edition of The Chronicle of Higher Education, an influential trade publication, featured a story about a letter sent by the provosts or vice presidents of seven Big Ten universities to David Duffield, PeopleSoft's chief executive at the time. In the letter, the universities complained that the "performance of the software, in terms of responsiveness, is simply unacceptable." (For instance, at the University of Wisconsin at Oshkosh, system quirks helped failing students escape mandatory expulsions.)
The complaints were not limited to the Big Ten. Malfunctions at Cleveland State University, for instance, led to the delaying of thousands of financial aid checks as installation costs ballooned from an estimate of $4.2 million to $11 million. At Boise State University, the problem was delayed transcripts. The cost: $16 million, for a project that was budgeted at just more than $5 million. University dissatisfaction was so widespread at one point that The Chronicle ran a story about events that seemed unusual enough to be news: Some schools were actually able to install the software smoothly.
"We knew going in that this was a high-risk project in general," says Steve Cawley of the University of Minnesota, which has PeopleSoft running well, but only after spending three years and $60 million implementing it (on a $42 million budget). "Going in, our research showed us that ERP systems typically come in 170 percent over budget with about 60 percent functionality.
"Now we're happy PeopleSoft customers."
Nearly all of the schools that struggled implementing PeopleSoft software eventually got it running. Even UW-Oshkosh eventually figured out whom to expel, and it now proclaims itself satisfied with the software's performance.
As does W.L. Gore.
But the San Francisco Unified School District, after spending more than $5 million, five years removed from its initial PeopleSoft purchase, having invested in more than three years of nonstop consulting, is not even close.
"We've had something like 900 successful implementations, so I don't think you can say it's the product," says Chris Feeley, PeopleSoft's vice president for education and government. "Obviously some of the things you need for a successful implementation weren't in place."
Feeley's assessment may contain an element of self-service; in educational quarters, PeopleSoft has become nearly legendary for late, overbudget software installations. Still, the San Francisco school district seems to have helped make its PeopleSoft project an undeniable disaster.
Part of the problem in San Francisco was the group that proposed and then led the project. Many of those at the highest levels of the project have quit or been fired or suspended, which makes it difficult to gauge exactly what happened on the district's end of the implementation. What is clear, however, is that the board approved the original $283,000 1995 contract for PeopleSoft human resources and payroll software -- as well as a subsequent $700,000 contract for PeopleSoft financial software -- at the urging of Coleman and Rojas. This is software that, Rojas and Coleman should have known, was designed with business -- not school districts -- in mind, and would have to be extensively tailored by consultants to become functional.
It is unclear how much Rojas and Coleman initially knew about PeopleSoft's deficiencies, but it is apparent the school district's computer system does almost nothing as well as advertised, and is especially deficient at the functions most specific to school districts. The system cannot, for instance, properly pay an English teacher who coaches basketball on the side (and thus gets paid from two different funds). It cannot communicate with the district's student information system -- a far less expensive, homemade system that also works poorly -- so changes in enrollment that should affect levels of state funding are not registered automatically. And its inability to store credentialing information has the district keeping some of its most important records in paper files.
Which is to say that the PeopleSoft system does little the district's old homemade-in-the-'70s system couldn't do. Only the old system was more reliable.
"What happens in the public sector sometimes is that administrators become cheerleaders for an organization, which lets the vendors, wittingly or unwittingly, put themselves into contractual situations that no rational person could believe," says Joe Moriarty of Claremont's Kerry Consulting Group, which investigates technology purchases for public-sector organizations.
And then there are the district employees working directly under Rojas and Coleman, who were aware of festering problems with Carrera's software installation plan from the moment it was drafted yet did little -- if anything -- to intervene, who routinely gave conflicting directions, who took vacations at key junctures of the project, and -- in the case of a district technology officer -- who used to work for Carrera Consulting itself.
There was more than adequate warning, early on, that the school district's PeopleSoft project was in trouble.
Staff correspondence shows that high-ranking school district officials knew Carrera Consulting's plan to install PeopleSoft software was flawed. It also shows that communication between the people who should have been in charge of the implementation was muddled at best. Documents and numerous interviews suggest it was never quite clear who, exactly, had overall authority over the implementation plan.
And then, as the PeopleSoft project was going nowhere, Rojas took the job as superintendent of Dallas schools, and Coleman and the chief technology officer from the San Francisco district soon followed, leaving a new team to deal with the Carrera/PeopleSoft debacle.
On March 11, 1998, Naomi Dorsch, an independent software consultant hired by the district to assist with the PeopleSoft installation, told the district's chief technology officer that Carrera's plan to install the software was far less detailed than standard, professional implementation plans.
In fact, it would not be approved by any consultants other than Carrera, she wrote, but "they are being paid to project manage. As long as they do their job properly and the district is not left lurching from one crisis to another, I say approve their plan with the conditions stated in the memo that I presented." Dorsch, who now works for PeopleSoft, also passed on her very conditional endorsement to Glenston Thompson, then the district's director of administrative and support services, who was responsible for implementation of the financial services portion of the software.
The district's budget director for the bulk of the implementation was Enrique Navas (who was demoted by new Superintendent Arlene Ackerman in August). According to Carrera's review of the SFUSD installation, difficulties with the budget director's office were the "most significant" reason for the district's current problems. These problems included repeated missed deadlines, vague directions, and a lack of technical support. The report also said Navas took a one-month vacation during a crucial testing period in the summer of 1998 without appointing someone to take his place.
Ruben Bohuchot was one of the first Carrera consultants on site at the district, personally invoicing more than $40,000 during the first few months of the project. In October 1997, he wrote a letter asking Coleman to authorize additional consulting help for the district. Less than a month later, he was hired as the district's chief technology officer.
Reached in Dallas, where he is the chief technology officer for the Dallas school district, Bohuchot says that although Coleman hired him away from Carrera because of his project management skills, he had no control over the PeopleSoft project at the SFUSD, which Thompson clumsily dominated. "By the time I understood the magnitude of the problems," he says, "it was fundamentally too late."
When asked about the e-mail he received from Dorsch expressing concerns about the original project plan before it was set in motion, the former technology director says he frequently brought concerns to Thompson, an accountant, but found himself ignored. Thompson, whose contract was recently terminated by Ackerman, could not be located for comment. The district's benefits office said he did not leave a forwarding address.
But public records suggest Thompson felt he had less control than Bohuchot credits him with.
"I have a very disquieting feeling over the prospect of being held responsible for something that I had no or very little control over," Thompson wrote in an e-mail to Coleman. "I am requesting that you remove me from the slot of project director. This will remove all ambiguities and eliminate the possibility of sending mixed signals concerning the leadership on the project."
In her aforementioned e-mail to Bohuchot, Dorsch described that ambiguity more succinctly: "There will be many times in the next few months when I will be getting conflicting direction from you and Glenston."
During July 1999, Bohuchot -- who ran the district's technology department on a year-to-year, consultant basis -- was informed by Acting Superintendent Linda Davis that his contract would not be renewed when it expired in November. Before the contract expired, Bohuchot was lured to Dallas, where he would work with Rojas and Coleman again, at a $140,000 salary.
"Bill Rojas asked me to come, and he said Bill Coleman was there as well, so I was in hog heaven," Bohuchot says. "They gave me the ability to run the show. They said, "Let's outline some basic goals you want to accomplish, and let you do the rest.'"
High on that list of priorities, apparently, was Bohuchot's first task in Dallas: seeking bids for a new ERP software package for a school district that installed a $6 million payroll system in 1995.
Dallas school board members had less patience with Rojas' management style than the San Francisco board. Rojas was fired from his $260,000-a-year Dallas post within a year. Coleman has since resigned.
At a public hearing on the school district's PeopleSoft project earlier this month, the technology consultant who coined the phrase "drop and run" in August is using other colorful terms (among them is "unpeeling the onion") to describe the effort to salvage the PeopleSoft system festering inside the San Francisco school district's computers.
Sandy Rosen, a slightly built gentleman who slicks back what gray hair he has left over his tanned scalp, tries to make his voice echo through the miserable acoustics of the scantly populated auditorium at Everett Middle School. What he says, basically, is: This is a long way from over.
He describes how a team of consultants, some hired directly by the district, others from Arthur Andersen Consulting, the city government, and PeopleSoft itself, are working -- within a $450,000 cap -- to determine if the software and the data entered into the system are salvageable. By the end of the year, Rosen says, the consultants might know whether the human resources and payroll systems are functioning. But there are other, larger, more complicated pieces of the software that will take more time to assess, and even longer to repair. If repair is possible.
"This is like a never-ending story," board member Frank Chong says. "We want some closure."
Seated next to Rosen is Cathi Vogel, the district's new chief financial officer, who has been periodically chiming in; Vogel notes that there are alternatives to staying with PeopleSoft. "We could farm it out," she says, referring to an embarrassing scenario in which a school district located at the heart of the Internet revolution sends its data to a district with a working system, perhaps in conjunction with a number of other districts.
Discussing this possibility, she sounds -- increasingly -- like someone preparing to cut her losses.
"We are painfully aware," she says to the board, "that some very hard decisions will have to be made soon."
While the district mulls those decisions, PeopleSoft counts its money. Public records show that the company has collected almost $2 million from the SFUSD in licensing, maintenance, and upgrade payments since 1995, and at least another $400,000 in consulting fees related to fixing work done by a firm it recommended as an Alliance Partner. If the district wants PeopleSoft to salvage the software, it will have to pay the company approximately $1 million more.
According to its contracts with the SFUSD, PeopleSoft has gone above and beyond its legal obligations. And PeopleSoft has read the contract.
"Look, we sent six consultants there for three weeks at no cost to the district," PeopleSoft VP Chris Feeley says. "I think we've done enough."