A-14-03-23069 - Alternative Format

MEMORANDUM

Date: July 10, 2003

To: Martin H. Gerry
Deputy Commissioner for Disability and Income Security Programs

From: Assistant Inspector General Audit

Subject: Evaluation of the Accelerated eDib System - Fourth Assessment (A-14-03-23069)

The Social Security Administration's (SSA) Office of the Inspector General (OIG) has completed its fourth assessment in our ongoing evaluation of the Accelerated eDib (AeDib) system (formally the Electronic Disability or eDib system). We conducted this fourth assessment from January 2003 through June 2003 at SSA Headquarters in Baltimore, Maryland. While we did not conduct an audit of the AeDib process, our assessment addressed those issues that arose at the AeDib Steering Committee. We were unable to determine the full scope of some of the AeDib problems addressed in this memorandum due to access limitations to Office of Systems (OS) staff and documentation. Those limitations are described throughout the memorandum.

As part of our fourth assessment, we considered the following issues:

The Office of Disability and Income Security Programs' (ODISP) participation in the AeDib project.
The AeDib's 5.0 software validation and release.
Data security at the disability determination services (DDS).
Reduction of the number of scanned pages used in each disability case.
Most AeDib project deadlines met by deferring functionality.

ODISP PARTICIPATION IN THE AeDib PROJECT

SSA developed the Web-based Project Resource Guide (PRIDE) to guide project managers, team leaders and team members when planning and executing system development projects. Based on this guide, ODISP is the Sponsoring Component, and as such, is the principal user and owner of the AeDib system and is primarily responsible for the finances and strategic direction. OS is the Developer and should design the system to ODISP specifications.

PRIDE calls for the sponsors of projects, such as ODISP, to be extensively involved in the development of a new system, such as AeDib. The ODISP project team needs to aggressively enforce PRIDE requirements in the following phases.

PRIDE REQUIREMENTS
AeDib STATUS

Conduct a Security Risk Assessment

Lead - Sponsor & Developer

The Systems and Business Project Managers must ensure all appropriate security parties are involved in project planning.

The designated Developer and primary sponsor only attended one AeDib Security meeting as of 05/16/03.
Conduct Appropriate Database Activities

Lead - Sponsor & Developer

Identify resources & requirements early
ODISP and OS have not yet completed assessing the fields the electronic folder needs
Plan for Validation and Testing

Lead - Sponsor and Developer

Although not required by PRIDE, the Validation Strategy Plan example in PRIDE shows both Sponsor & Developer sign-offs. No formal ODISP concurrence or sign off of the Validation Plan

The OS did not provide any evidence of its validation efforts, except for its Validation Plan.
Conduct Validation and Verification

Lead - Sponsor and Developer

Validation Analysis Report is completed, and the Systems Release Certification form is completed and signed. OS did not provide us with the Validation Analysis Report and System Release Certification form.

ODISP doesn't verify that system releases meet the needs set forth in its functional requirements.
Release (Move) to Production

Lead - Deputy Commissioner for Systems (DCS)

Sponsor determines when the software is moved to the Production environment. OS periodically contacts ODISP via telephone to inform them of the status of the validation.

No formal release and certification by ODISP.

THE AeDib 5.0 SOFTWARE VALIDATION AND RELEASE

OS issued software release 5.0 for the AeDib Systems on March 3, 2003. Release 5.0 adds the capability for field offices to take Title XVI disabled child claims. The following flow chart documents the Agency's required process for conducting validations during systems development.

The OIG found about 30 production problems shortly after the release of AeDib 5.0 software. As a result, the software may need to be fixed in later, more costly, stages of the software development process. We also identified the following AeDib software production problems:

There were 5,000 duplicate cases.

There were claims taken for Title XVI disabled individuals in the Modernized Supplemental Security Income Claims System that were recorded in the Electronic Disability Collect System (EDCS) as disabled children.

There were numerous incorrect EDCS edits based on the Internet 3368s that claims representatives needed to resolve. One claims representative reported taking 3 hours to "clean up" an Internet 3368 imported into EDCS.

The OIG attempted to determine the causes of the production errors through examination of the AeDib 5.0 validation documents and interviews with the validation team members. However, we were only provided with one of the validation and testing documents and were not provided the necessary access to individuals involved in the validation process. The only validation document received was the AeDib 5.0 Validation Plan. However, because this Validation Plan was dated 50 days after AeDib 5.0 software was released into production (March 3, 2002), we could not determine the adequacy and usefulness of the Validation Plan at the time of testing.

AeDib release 5.0 was validated in only 22 days with the number of validation runs reduced from 3 to a single run. Normally, the Division of Configuration Management and Validation Technology validates a system in an average of 86 days. AeDib release 4.2.3, which was not a major release, was validated over a 60-day period. Due to the limited time spent in validation there were a number of significant problems associated with this release.

The premature release of software is both time consuming for individuals processing claims and expensive to the Agency. The Baziuk Study, detailed on page 5, is just one of many studies that show the substantial cost of repairing software defects found after the software is operational. This study indicated that the cost to repair an error during systems testing is 90 times more expensive than when the error is found during the requirements phase. Furthermore, once the system is operational, the cost increases to between 470 to 880 times than when the error is found in the requirements phase. Therefore, SSA's systems development efforts would benefit, in the long term, by identifying and correcting software errors at the earliest possible stage in its systems development.

SSA plans to begin a nationwide rollout of AeDib beginning in January 2004. However, before the AeDib software is expanded nationwide, it is critical that the thorough validation required by PRIDE is completed and the system is certified as ready for release into production. This certification should be completed by the project's sponsor who is familiar with the systems requirements, yet independent from the AeDib software design and development teams.

DATA SECURITY AT DDSs NEEDS IMPROVEMENT

Recent OIG and PricewaterhouseCoopers (PwC) audits found that data security at State DDSs needs improvement. DDS security will be even more important in 2004, when SSA begins granting the DDSs direct access to the Agency's electronic folder. In 2004, the Agency's electronic folder is expected to house sensitive data from SSA, DDS and the Office of Hearings and Appeals (OHA). Therefore, someone gaining improper access through a DDS could obtain access to the Agency's data. To increase security requirements for the State DDSs, the Agency has already agreed to, and is in the process of implementing, many of the reported OIG and PwC recommendations. Once the DDSs implement these new requirements, the security of SSA's data should be greatly enhanced.

REDUCTION OF THE NUMBER OF SCANNED PAGES USED IN EACH DISABILITY CASE

The Agency will spend $375 million over a 10-year period to scan paper documents into its electronic folder according to Booz Allen Hamilton estimates. These paper documents, which will be part of the AeDib process, are now maintained in the Agency's disability case folders, and average 250 pages per case file when taken through the full disability process to the OHA.

SSA should only scan documents that are essential to support disability determinations. In addition, the Agency should evaluate whether it can further reduce the scanning of the 95 additional paper documents used through the continuing disability review phase. We formally asked for this evaluation at the March 11, 2003, AeDib Steering Committee meeting. Please provide us with an update on any plans the Agency has to reduce the number of pages it plans to receive and scan into the electronic folder.

According to the National Council of Disability Determination Directors' memorandum to the Commissioner, SSA should regard the conversion of paper to digital images as an interim solution. We concur. SSA's long-term objective should be the direct electronic transfer of medical records from providers to DDS systems. President Bush recently stated that the "implementation of E-Government is important in making Government more responsive and cost-effective."

For example, the Internal Revenue Service's (IRS) E-Government Program Office has informed us that the IRS experienced a 50 percent reduction in processing costs by accepting tax returns electronically. IRS costs vary, but generally, it saves about $1.00 for each e-filed return. However, the true savings for the IRS is in its downstream operations. For example, the difference in the error rate between paper and e-filed returns is 22 percent versus 1 percent. Also, because the IRS receives the data electronically, everything (100 percent) is captured, rather than keying just a subset of the data. This additional electronic information has also enabled the IRS to achieve benefits resulting in fewer taxpayer notices, more effective compliance research, fewer taxpayer contacts (phone or in person), fewer processing centers, and higher accuracy.

MOST AeDIB PROJECT DEADLINES ARE MET BY DEFERRING FUNCTIONALITY

As the chart below details, the Agency is on schedule with respect to most of the AeDib's actual release dates; however, much of the functionality is being delayed to future releases. We understand that the AeDib's project delays may be necessary to ensure the release of quality software. OIG informed the AeDib Steering Committee leaders that any slippage of functionality should be fully discussed with the AeDib Steering Committee. The Steering Committee leaders did that and discussed the delay in release 6.0 with the Committee.

Reduction in the Functionality of AeDib
Version
Original
Dates Revised
Dates Original Functionality
See Attachment A Revised Functionality
See Attachment B
4.2.3 10/02 10/02 1. Record of Change
2. Subsequent Filings
3. Medical Source File updates
4. Interface to the Internet 3368 1. Record of Change
2. Subsequent Filings
3. Medical Source File updates
4. Interface to the Internet 3368
5.0 02/03 02/03 1. Children at the Initial
Adjudicative Level
2. Reconsiderations 1. Children at the Initial
Adjudicative Level
5.1

05/03 05/03 1. Continuing Disability Reviews (CDR)
2. CDR Reconsiderations 3. Hearing Cases 4. All Other Related Forms 1. Reconsiderations 2. Initial Appeals 3. Hearing Cases
6.0 07/03 07/03 1. EDCS Interface to Electronic Folder 1. 1. CDRs 2. CDR Reconsiderations 3. All Other Related Forms
6.1 12/03 12/03 2. 1. EDCS Interface to Electronic Folder

There is no expectation for the Agency to formally respond to this document. If you have any questions or comments, please call me or have your staff contact Kitt Winter, Director, Data Analysis and Technology Audit Division at (410) 965 9702, or Al Darago at (410) 965 9710.

Steven L. Schaeffer

Accelerated eDib System Timeline
05/03/02
Internet Disability

The Internet disability applications collect information currently gathered from the Agency's paper disability form. The initial release, Internet 3368, will collect medical and work history from disability claimants. Additional applications will be developed to support the disability process. These applications will collect supplemental disability and more detailed work information, information about childhood disabilities, and information required for subsequent appeal processes. Internet disability applications will improve service to the public, compensate for resource losses and workload increases, improve the disability report collection process, and contribute to meeting the Government Paper Elimination Act requirements.

Date Milestone
8/02 Production Ready for Initial Functionality of I3368
1/03 Production Ready for Internet I827
4/03 Production Ready for Fully Functional I3368
7/03 Production Ready for Internet I3820
11/03 Production Ready for Internet I3369
12/03 Production Ready for Internet I3441
12/03 Production Ready for Internet I454, I4486, I4631

Electronic Disability Collect System ver 4.2.2
Electronic Disability Collect System (EDCS) provides the means for our employees to collect information about a claimant's disability. EDCS 4.2.2 is a technical release to convert the EDCS from a client/server application to an intranet application. This release is limited to adult disability cases at the initial adjudicative level.
Date Milestone
7/02 Production Ready (Delaware, Texas, & California)
Electronic Disability Collect System ver 4.2.3
Adds the following functionality:
1. Record of Change
2. Subsequent Filings
3. Alternative Methods to Populate the Medical Source Reference File
4. Interface to the Internet 3368
Date Milestone
10/02 Production Ready for Delaware, Texas, & California
10/02 Production ready for National Rollout

Electronic Disability Collect System ver 5.0
Adds the following types of disability cases
1. Child cases at the initial adjudicative level
2. Reconsiderations
Date Milestone
2/03 Production Ready
Electronic Disability Collect System ver 5.1
Adds the following:
1. Continuing Disability Reviews (CDR)
2. CDR Reconsiderations
3. Hearing Cases
4. All other related forms
Date Milestone
5/03 Production Ready
Electronic Disability Collect System ver 6.0
EDCS interface to Electronic Folder using MQSeries as the transport mechanism.
Date Milestone
7/03 Production Ready
Electronic Disability Collect System ver 6.1
The Disability Determination Services and the Social Security Administration's Legacy Applications interface to Electronic Folder using MQSeries as the transport mechanism. Includes the storage and retrieval of data to a data repository as well as images and other objects to the Enterprise Document Imaging Architecture (EDIMA).
Date Milestone
12/03 Production Ready

Enterprise Document Imaging Architecture
This project will identify and implement the document imaging architecture and infrastructure required to support the Accelerated eDIB System's business process.
Date Milestone
10/02 Architecture and Infrastructure Recommendations Documented
10/03 Complete Procurements for EDIMA Infrastructure
1/04 Complete EDIMA Infrastructure Installation in Required Sites

AS400/Legacy Software
This project includes the migration of Wang/Levy states to IBM AS/400 platform; migration of Levy code incorporating readiness for EFI; upgrade/replacement of existing AS/400s in order to accommodate EFI; readiness of Versa, Midas, and independent software systems for EFI.

The Office of Hearings and Appeals Case Processing and Management System
The Office of Hearings and Appeals (OHA) Case Processing and Management System will provide automation to the Hearing Offices activities.
Date Milestone
10/02 Determine Systems Design
12/03 Pre-Production Implementation
1/04 Production Ready

Complete Business Process Description
Date Project
6/02 OHA, Operations, Office of Quality Assurance and Office of Disability

Accelerated eDib System Timeline
03/28/03
Internet Disability
The Internet disability applications collect information currently gathered from the Agency's paper disability form. The initial release, i3368, will collect medical and work history from disability claimants. Additional applications will be developed to support the disability process. These applications will collect supplemental disability and more detailed work information, information about childhood disabilities, and information required for subsequent appeal processes. Internet disability applications will improve service to the public, compensate for resource losses and workload increases, improve the disability report collection process, and contribute to meeting the Government Paper Elimination Act requirements.
Date Milestone
08/02 Production Ready for Initial Functionality of i3368
04/03 Production Ready for Fully Functional i3368
05/03 Production Ready for Internet i454
07/03 Production Ready for Internet i3820
09/03 Production Ready for Internet i3369
12/03 Production Ready for Internet i3441and i827

Electronic Disability Collect System ver. 4.2.2
Electronic Disability Collect System (EDCS) provides the means for our employees to collect information about a claimant's disability. EDCS 4.2.2 is a technical release to convert the EDCS from a client/server application to an intranet application. This release is limited to adult disability cases at the initial adjudicative level.
Date Milestone
07/02 Production Ready for Delaware, Texas, & California
Electronic Disability Collect System ver 4.2.3
Adds the following functionality:
5. Record of Change
6. Subsequent Filings
7. Alternative Methods to Populate the Medical Source Reference File
8. Interface to the Internet 3368
Date Milestone
10/02 Production Ready for Delaware, Texas, & California
10/02 Production ready for National Rollout
Electronic Disability Collect System ver 5.0
Adds Child cases at the Initial adjudicative level
Date Milestone
03/03 Production Ready

Electronic Disability Collect System ver 5.1
Adds Initial Appeals
Date Milestone
05/03 Production Ready
Electronic Disability Collect System ver 6.0
EDCS interface to Electronic Folder using MQSeries as the transport mechanism and the following
1. Alerts of field office Update -- Post Transfer
2. Viewing of Document Management Architecture (DMA) Contents (for pilot sites)
3. Continuing Disability Reviews (CDR)
4. CDR Reconsiderations
5. All Other Related Forms
Date Milestone
07/03 Production Ready
Electronic Disability Collect System ver 6.1
The Disability Determination Services and the Social Security Administration's Legacy Applications interface to Electronic Folder using MQSeries as the transport mechanism. Includes the storage and retrieval of data to a data repository as well as viewing of images and other objects within the DMA. Jurisdictional and workflow controls are also included.
Date Milestone
12/03 Production Ready

Document Management Architecture

This project will identify and implement the document imaging architecture and infrastructure required to support the Accelerated eDib System's business process.
Date Milestone
10/02 Architecture and Infrastructure Recommendations Documented
07/03-12/03 DMA Pilots
10/03 Complete Procurements for DMA Infrastructure
01/04 Complete DMA Infrastructure Installation in Required Sites

The Office of Hearings and Appeals Case Processing and Management System
The Office of Hearings and Appeals (OHA) Case Processing and Management System will provide automation to the Hearing Offices activities.
Date Milestone
10/02 Determine Systems Design
12/3/03 Pre-Production Implementation
1/26/04 Production Ready

AS400/Legacy Software

This project includes the migration of Wang/Levy states to IBM AS/400 platform; migration of Levy code incorporating readiness for EFI; upgrade/replacement of existing AS/400s in order to accommodate EFI; readiness of Versa, Midas, and independent software systems for EFI.

Group 1 States = VA, WV, MD, WI, IN, GA, AR, OH, OK, IA, NC, FL, NM, RI, SD, Federal Disability Determination Service (FDDS)
Group 2 States = KS, MA, WA, DC, KY, MT, CT, MI, CO, AZ, LA, VT, PR
Date Milestone
06/02 Installation of AS/400 Complete for RI, SD, KS, MA, FDDS
09/02 Installation of AS/400 for DC, KY, MT, CT, MI, CO, AZ, LA, VT, PR
10/02 Production Ready Version 4.2.3 in MIDAS pilot state
12/02 Production Ready ALL Group 1 States with Levy AS/400 Application Code in Preparation for EFI
02/03 Production Ready Version 4.2.3 in Versa and Levy pilot states
03/03-04/03 Production Ready Version 5.0 in Versa (TX, MN, WY) and Levy pilot states (WV, OK, IA) and MIDAS (DE, CA)
07/03 Production Ready Version 5.1 rollout to remaining states
07/03-12/03 DMA pilot in MIDAS, Versa and Levy pilot states
08/03 Production Ready Version 6.0 in MIDAS, Versa and Levy pilot states
10/03 Production Ready ALL Group 1 and Group 2 States with Levy AS/400 Application Code with EFI
12/03 Production Ready Version 6.1 in MIDAS, Versa and Levy pilot states

Quality Assurance Systems

This project will integrate the existing disability Quality Assurance systems (DICARS, OHAPER and DHQRS) with AeDib. It will support migration to a paperless business process and provide the Office of Quality Assurance (OQA) with access to the electronic folder.
Date Milestone
6/03 Determine Systems Design
8/03 Initial Pilot
1/04 Production Ready

Complete Business Process Description
Date Project
6/02 OHA, Operations, OQA and Office of Disability