Building and Managing Systems

The Pennsylvania Department of Labor and Industry (DLI) is responsible for the admin- istration and operation of the state’s unem-

ployment compensation program, which provides temporary income to replace lost wages for quali- fied workers. DLI employs over 5,000 people and has approximately 200 offices statewide to serve Pennsylvania’s 6.4 million workers and nearly 300,000 employers. Unemployment compensation (UC) claims are usually filed online or by telephone or mailed to a UC service center.

DLI had a legacy mainframe system for processing unemployment benefits that was over 40 years old. However, it became increasingly expensive to main- tain and difficult to modify, with limited functional- ity for case management and integrating newer tools and technologies to enhance productivity.

In June 2006, DLI awarded IBM a fixed price con- tract totaling $109.9 million for the Unemployment Compensation Modernization System (UCMS), which would replace the antiquated mainframe system. The initial contract with IBM called for more modern and efficient technology and business processes for (1) maintaining wage records, (2) processing em- ployer taxes, and (3) claims processing, payment, and appeals, to be completed by February 2010. IBM won the UCMS contract after a three-year bidding process, claiming to be the only vendor with the type of proprietary databases capable of supporting a to- tally integrated computer system.

However, this project experienced significant delays and cost overruns, ultimately costing nearly $180 million, with much of the system never com- pleted when the contract expired in September 2013. By that time, the project was 45 months behind schedule and $60 million over budget. Pennsylvania taxpayers had paid IBM nearly $170 million for what was supposed to be a comprehensive, integrated, and modern system that it never got. IBM’s contract was not renewed. In March 2017, Pennsylvania sued IBM for breach of contract, fraudulent and negligent mis- representation, and charging taxpayers for services it did not provide. IBM said Pennsylvania’s claims had no merit and that it would fight the lawsuit. A spokesman for the company laid some of the blame for the project’s problems on the state, saying that there was responsibility on both sides for system

Pennsylvania’s Unemployment Compensation Modernization System: Unfinished Business

performance and service delivery. How did all of this happen?

Phase 1 of UCMS (wage records) was implemented in May 2008. Phase 2, which included the employer tax portion of the system, went live in March 2011 but required additional work, which took years to fix. Phase 3 for benefit claims processing, payment, and appeals continued to lag behind with problems and ultimately never went live.

In 2012, DLI enlisted the Carnegie Mellon Software Engineering Insitute to conduct an inde- pendent assessment of the UCMS. The study was completed in July 2013, and recommended continu- ing work on remaining Phase 2 problems, but stop- ping work on Phase 3. Many of the problems it iden- tified for Phase 3 could not be solved.

The Carnegie Mellon study found many flaws in the systems development process. IBM had extensive systems experience and technology knowledge but its proposal underestimated the project’s scope and complexity. DLI lacked sufficient staffing and experi- ence for effective oversight and management of the contract and project. There was no formal delegation of roles and responsibilities for managing the project. No one at DLI was held accountable. DLI essentially relied on the contractor to self-manage.

UCMS was considered a large-scale software proj- ect due to its complexity, large number of informa- tion requirements and business rules, and its cost. DLI’s solicitation for vendor proposals for UCMS exhibited ambiguity in communicating all of these requirements, and also neglected to define and de- scribe quantitative and qualitative performance mea- sures and metrics for the proposed system.

A large-scale software-intensive system such as UCMS requires a rigorous and disciplined testing strategy, but this was not implemented. IBM decided to use DLI users to help develop test scripts. They provided the business expertise, but IBM did not use IT test experts on its end. User acceptance test- ing was initiated before completing system tests for Phase 2 and Phase 3. Rigorous testing came too late in the project. DLI did not specify a minimum of metrics for UCMS system performance so that there were no identifiable criteria and evidence for deter- mining that Phase 2 and Phase 3 application releases were stable.

DLI staff had approved IBM’s representation of busi- ness system requirements without fully understanding what they were approving. IBM’s software development and testing program for this project lacked rigor. This resulted in a higher number of software defects than industry norms, software code that was excessively complex (which makes testing too difficult), and late discovery of missing business requirements.

The vast majority of the software defects were seri- ous, and 50 percent were not discovered until the User Acceptance test, very late in the system development cycle. Without thorough and complete testing through- out the development process, there is no way to know how many of the total defects residing in software will be discovered as a system is being used. Carnegie Mellon also found that IBM had not performed a stress test to determine the performance limits of the UCMS system.

IBM’s software development plan was supposed to use industry and company standards and practices, but there was no ongoing discipline to execute these stan- dards and practices during the project. DLI accepted Phase 2 prematurely for production in March 2011 with known defects impacting system performance, includ- ing software defects, unresolved data conversion issues, and problems with batch processing operations.

A project of this complexity and magnitude requires a high degree of continuity in knowledge throughout the system development cycle, but this was never achieved. During requirements determination, DLI didn’t have enough user subject matter experts to participate in joint application design (JAD; see Chapter 13) sessions with technical members of the project team. Thirty-six JAD subcontractors were prematurely removed from the proj- ect, leaving IBM with incomplete understanding of un- employment claims processing business requirements. System design and testing staff were not included in the JAD process, running counter to sound business prac- tice. Including them was essential to ensure UC business requirements were defined in sufficient detail to be test- able. DLI staff often approved JAD requirements docu- ments and Detailed System Design documents under pressure to meet short deadlines for approval.

Ineffective project management and constant changes in the contractor’s workforce prevented trans- fer of essential knowledge for the entire project, a loss of “project memory.” Since the UCMS project’s start, 638 different contractors and staff members worked on the project. The majority of the project workforce spent less than one year on the project and 75 percent spent less than 2 years. All of these discontinuities and workforce churn most likely contributed to IBM’s schedule delays and inability to provide an accu- rate picture of the state of the project.

Work on Pennsylvania’s UC system contin- ued without IBM. In 2013, the Pennsylvania Legislature passed Act 34, which created a Services Infrastructure Improvement Fund (SIIF) as a temporary supplemental funding source to improve UC services and systems. A total of $178.4 million was authorized and spent during calendar years 2013 through 2016. Even then the project stumbled. Pennsylvania Auditor General Eugene A. DePasquale initiated an audit in January 2017 to determine how the $178 million in SIIF funds were spent. The auditors found that DLI did not use proper accounting methods to record specific SIIF expenditures. DLI comingled unemployment compensation administrative funds from all sources, including federal funds for unemployment compensation administration and interest on unemployment compensation tax money as well as outlays from SIIF.

On a more positive note, there were noticeable improvements and efficiencies from 2013 through 2016 in services provided to UC claimants and in UC system infrastructure. For example, the per- centage of first payments paid promptly increased from 81.6 percent to 93.4 percent. However, DLI was unable to show how exactly the SIIF expendi- tures contributed to these outcomes.

When SIIF funding was not reauthorized and supplemental funding ended in December 2016, DLI was forced to cut $57.5 million from its UC ad- ministrative budget for 2017, causing the immediate closure of three of the state’s eight UC service cen- ters in December 2016 and the elimination of 521 positions. Customer service declined significantly with claimants not being able to get through on the phone lines and delays in processing claims.

Despite earlier setbacks, DLI is determined to complete the modernization of its unemploy- ment compensation benefits delivery system. In June 2017, DLI signed a $35 million contract with Florida-based Geographic Solutions to create a system that enhances customer service, improves quality, is more operationally efficient, and is sus- tainable into the future. Geographic Solutions spe- cializes in designing, developing, and maintaining web-based systems for the workforce development and unemployment insurance industries and has developed over 80 workforce systems for state and

local agencies across the United States. Geographic Solutions was scheduled to begin work on the system on August 1, 2017 with a projected 18 to 24 months for completion.

In 2015, DLI had hired Chicago-based CSG Government Solutions for $6.1 million to assist with planning for and monitoring this project. CSG spe- cializes in planning, managing, and supporting com- plex projects that modernize the information tech- nology and business processes of large government programs. CSG analyzed existing systems and work- flows, developed the project strategy and technology roadmap, and gathered business and technical re- quirements to develop an RFP. CSG also established a full-service Project Management Office to monitor project progress, and is providing technical oversight, UC subject matter expertise, requirements manage- ment, and testing support throughout the system modernization. Once the new system has been fully implemented, cost savings from benefit moderniza- tion are estimated to range from 5 to 10 percent of total UC administrative costs.

DLI’s 2017 contract with Geographic Solutions specified that the new benefits delivery system was supposed to go live by April 30, 2018. As of the spring of 2020, the old system was still in place and unable to handle the millions of Pennsylvania work- ers who applied for benefits during the coronavirus pandemic shutdown. Geographic Solutions President Paul Toomey said the contract with DLI was an ini- tial estimate based on similar systems in other states, but after analyzing the system they found over 1,000

programming changes required. Toomey added there was no additional cost increase and the project re- mained on budget and schedule for completion in October 2020.

CASE STUDY QUESTIONS

1. Assess the importance of the Unemployment

Compensation Modernization System project for the state of Pennsylvania.

2.Why was unemployment compensation mod- ernization a risky project in Pennsylvania? Identify the key risk factors.

3.Classify and describe the problems encoun- tered by the UCMS projects. What manage- ment, organization, and technology factors were responsible for these problems?

4. What could have been done to mitigate the risks of these projects?