Date: September 20, 2004

To: The Commissioner

From: Acting Inspector General

Subject: Performance Indicator Audit: Employment for Disabled Beneficiaries (A-02-04-14068)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 16 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. The attached final report presents the results of one of the performance indicators PwC reviewed. For the performance indicator included in this audit, PwC's objectives were to:

Test critical controls over the data generation and calculation processes for the specific performance indicator,

Assess the overall adequacy, accuracy, reasonableness, completeness, and consistency of the performance indicator and supporting data, and

Determine if each performance indicator provides meaningful measurement of the program and the achievement of its stated objectives.

This report contains the results of the audit for the following indicator:

Percent increase in the number of Supplemental Security Income disabled beneficiaries earning at least $100 per month.

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at
(410) 965-9700.

Patrick P. O'Carroll, Jr.

OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
EMPLOYMENT FOR
DISABLED BENEFICIARIES

September 2004

A-02-04-14068

AUDIT REPORT

Mission

We improve SSA programs and operations and protect them against fraud, waste, and abuse by conducting independent and objective audits, evaluations, and investigations. We provide timely, useful, and reliable information and advice to Administration officials, the Congress, and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

By conducting independent and objective audits, investigations, and evaluations, we are agents of positive change striving for continuous improvement in the Social Security Administration's programs, operations, and management and in our own office.

MEMORANDUM

Date: September 9, 2004

To: Acting Inspector General

From: PricewaterhouseCoopers LLP

Subject: Performance Indicator Audit: Employment for Disabled Beneficiaries (A-02-04-14068)

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

OBJECTIVE

For each performance indicator included in this audit, our objectives were to:

1. Test critical controls over the data generation and calculation processes for the specific performance indicator.
2. Assess the overall adequacy, accuracy, reasonableness, completeness, and consistency of the performance indicator and supporting data.
3. Determine if each performance indicator provides meaningful measurement of the program and the achievement of its stated objectives.

We audited the following performance indicator as stated in the SSA Fiscal Year (FY) 2003 Performance and Accountability Report (PAR):

Performance Indicator FY 2003 Goal FY 2003 Reported Results
Percent increase in the number of Supplemental Security Income (SSI) disabled beneficiaries earning at least $100 per month. 8% (269,109)
N/A 232,654 through July 2003.

BACKGROUND

SSA has several work incentive programs that assist SSI disabled beneficiaries to return to work. These work incentive programs include the following:

Ticket to Work Program (TTW) - The program is designed to assist disabled beneficiaries to obtain employment. Enrollees may use the ticket to obtain the vocational rehabilitation services, employment services and other support services needed to return to work, or go to work for the first time. As long as a beneficiary is using a ticket (as determined under SSA criteria), SSA will not initiate a Continuing Disability Review.

Plan to Achieve Self-Support (PASS) - The program allows SSI beneficiaries to set employment goals and set aside money for these goals so that these funds will not affect their SSI eligibility or payment amount.

1619A and 1619B Programs - These programs allow SSI beneficiaries to return to work over time without losing SSI and Medicaid eligibility.

Impairment-Related Work Expense (IRWE) - The program allows SSI beneficiaries to obtain items, such as a cane or wheelchair, without the income used for such items affecting their SSI eligibility or benefit amount.

Blind Work Expense (BWE) - The Blind Work Expenses program does not count any earned income that a SSI beneficiary uses to meet expenses that are required for the beneficiary to work. Candidates must be receiving SSI payments due to blindness. BWE items do not have to be related to their blindness.

The SSI Program, authorized by Title XVI of the Social Security Act, was designed as a needs-based program to provide or supplement the income of aged, blind, and/or disabled individuals with limited income and resources. SSI beneficiaries are required to report their earnings to SSA. Title XVI of the Act provides for the reduction or stoppage of SSI benefits based on the earnings of the beneficiaries. The main methods to report earnings include the following:

SSI beneficiaries report their earnings to SSA field offices, SSA's 1-800 number, or by sending a letter to SSA. After receipt of the beneficiaries' earnings evidence (such as pay stubs for wages or a tax return for self-employment), SSA inputs earnings information into the Supplemental Security Record (SSR) via the Modernized Supplemental Security Income Claims System (MSSICS).

If SSI beneficiaries do not report their earnings to SSA, SSA will use information contained within the Master Earnings File (MEF) to determine if there were new earnings reported against the beneficiary's record. Most of the information included in the MEF is provided by the Internal Revenue Service (IRS). The MEF interfaces with the SSR and will create alerts/diaries that indicate that the SSI beneficiaries' earnings may have changed. This interface between the MEF and SSR occurs several times per year.

SSA also receives quarterly State wage data from the Office of Child Support Enforcement, and alerts are generated when earned income data on the SSR does not match, within a tolerance.

On a quarterly basis, the Office of Research, Evaluation, and Statistics (ORES) receives the Work Incentive File from the Office of Disability and Supplemental Security Income Systems (ODISSIS). This file contains terminated and active data records that include beneficiaries' Social Security Numbers (SSN) and earnings information. The type of earnings for each beneficiary is recorded in the file as one of the following categories: S (self-employment), W (wages), C (blind work expense), D (income excluded under approved plan), T (impairment related work expenses), N (net loss), and B (student earned income exclusion). These data elements are extracted from the Work Incentive File and formatted into an Excel file using Statistical Analysis Software (SAS). On a quarterly basis, this Excel spreadsheet is sent to the performance indicator owner in the Office of Disability Income Security Programs (ODISP), Office of Employment Support Programs (OESP). The performance indicator owner manually calculates the results of the SSI beneficiaries earning at least $100 per month from the Excel spreadsheet.

The indicator was calculated as follows:
Current year number of SSI disabled beneficiaries earning at least $100 for the last date of the particular quarter

Sum of the SSI disabled beneficiaries earning at least $100 for the last date of the particular quarter.

Percent increase (decrease) in SSI beneficiaries earning at least $100 per month

Current year number of SSI disabled beneficiaries earning at least $100 for a particular quarter minus previous year number of SSI disabled beneficiaries earning at least $100 for the last date of the particular quarter.

Previous year number of SSI disabled beneficiaries earning at least $100 for the last date of the particular quarter.

This performance indicator represents the percent increase in the number of SSI disabled beneficiaries earning at least $100 per month for the last date of that particular quarter.

On a quarterly basis, the performance indicator is sent to the Office of Strategic Management (OSM) to be incorporated as part of the centralized performance indicator tracking report.

RESULTS OF REVIEW

Percent increase in the number of Supplemental Security Income disabled beneficiaries earning at least $100 per month.

FY 2003 Goal: 8 Percent (269,109)
Actual FY 2003 Performance: Not Available. SSA did not have the data to determine if it met its goal.

Findings

While we found this performance indicator to be reasonable and meaningful, SSA can strengthen the security over the measurement process and take steps to better report performance in this area. We believe that SSA should strengthen the logical access privileges given to employees to help ensure that the data supporting this performance indicator is not inadvertently deleted or changed without appropriate approval. Due to the volatility of the SSI program, we believe that SSA should consider averaging this information over a relevant period, rather than reporting it at a single point in time. Finally, we believe that the Data Definition associated with this performance indicator should be clarified to reflect all of the various programs and activities utilized by SSA to positively impact the earnings potential of SSI Disabled Beneficiaries.

We tested the "Work Incentive File" mainframe data used to calculate the indicator and found that a total of nine SSA employees and contractors had the "All" access designation within the Top Secret security software to these datasets. The "All" level of access allows users to create, delete and update any of the data (or datasets) contained within the datasets we reviewed without appropriate review or approval of the changes. The "All" level of access prevents SSA from ensuring the integrity of this production data. Additionally, by allowing employees and contractors to have the "All" access designation, SSA is not conforming to the principles of "least privileged access" or segregation of duties.

SSA measures this indicator on a periodic basis throughout the year. However, the indicator reported in the PAR represents SSI beneficiaries recorded on the SSR as earning over $100 for the month of June. Such single point in time measurements are susceptible to seasonal fluctuations or other anomalous changes in the underlying data, which have the potential to impact their reliability as indicators of sustained progress against objectives. Because SSA has interim data available regarding this indicator, we believe that averaging this information over a relevant period, rather than reporting it at a single point in time, would help ensure that reliability is not adversely impacted by seasonal or other anomalous changes in the underlying data. For example, the performance indicator could be calculated using the average result of the 4 quarters for the current year and comparing it to the average result of the 4 quarters for the prior year.

We found that the results in the FY 2003 PAR did not discuss all of the SSA programs used to assist SSI disabled beneficiaries to return to work such as the PASS, 1619A and 1619B, IRWE and BWE programs. In addition, the PAR does not discuss any other means of increasing SSI beneficiaries' earnings.

SSA had not documented policies and procedures related to the process to collect, review and provide the performance indicator data to OSM.

The FY 2003 PAR did not report the correct month for the results of the performance indicator. The FY 2003 PAR reported the time period as through July 2003, although the data was only produced through June 2003.

RECOMMENDATIONS

We recommend SSA:

1. Ensure that SSA personnel do not have the ability, through inappropriate access, to directly modify, create or delete the datasets used to calculate the results of this indicator.
2. Consider revising the calculation of the performance indicator by averaging the interim data available for the indicator over a relevant period, rather than reporting it at a single point in time.
3. Disclose and discuss all of the SSA programs that assist the SSI disabled beneficiaries to increase their monthly earnings.
4. Document the policies and procedures used to prepare and disclose the results of the performance indicator.
5. Ensure that the performance indicator results are properly reported in conformity with the data definition.

AGENCY COMMENTS

The Agency agreed with our recommendations. The full text of the Agency's comments is in Appendix D.

Patrick P. O'Carroll, Jr.

Appendices
APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Agency Comments

Appendix A
Acronyms
BWE Blind Work Expenses
ERMS Earnings Record Maintenance System
FY Fiscal Year
GAO Government Accountability Office
GPRA Government Performance and Results Act
IRS Internal Revenue Service
IRWE Impairment Related Work Expense
MEF Master Earnings File
MSSICS Modernized Supplemental Security Income Claims Systems
ODISP Office of Disability Income Security Programs
ODISSIS Office of Disability and Supplemental Security Income Systems
OESP Office of Employment Support Programs
ORES Office of Research, Evaluation, and Statistics
OSM Office of Strategic Management
PAR Performance and Accountability Report
PASS Plan for Achieving Self-Support
SAS Statistical Analysis Software
SGA Substantial Gainful Activity
SSA Social Security Administration
SSI Supplemental Security Income
SSIRMS Supplemental Security Income Records Maintenance System
SSN Social Security Number
SSR Supplemental Security Record
TTW Ticket to Work Program

Appendix B
Scope and Methodology
We updated our current understanding of the Social Security Administration's (SSA's) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing including testing of source documentation, we performed the following as applicable:

Reviewed prior SSA, Government Accountability Office, and other reports related to SSA GPRA performance and related information systems.
Met with the appropriate SSA personnel to confirm our understanding of each individual performance indicator.
Flowcharted the processes (see Appendix C).
Where applicable, we tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).
Conducted and evaluated tests of the automated and manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.
For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.
Identified and extracted data elements from relevant systems and obtained source documents for detailed testing selections and analysis.
Identified attributes, rules, and assumptions for each defined data element or source document.
Tested the adequacy, accuracy, reasonableness, consistency, and completeness of the selection.
Recalculated the metric or algorithm of key performance indicators to ensure mathematical accuracy.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators being used appear to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes. We followed all performance audit standards. In addition to the steps above, we specifically performed the following to test the indicator included in this report:

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following areas:

o Ensured that monthly earnings information provided by the claimant were accurately posted to the Supplemental Security Record by reviewing 120 redetermination cases and observing 9 redeterminations in the field. Redeterminations are a review of the beneficiaries' non-medical eligibility factors (i.e., income, resources and living arrangements) to ensure that they are still eligible for and are receiving the correct Supplemental Security Income (SSI) payment. Documents, such as pay stubs, are reviewed and used to determine the beneficiaries' eligibility when applicable.

o Completed application control reviews over Modernized Supplemental Security Income Claims Systems (MSSICS), Supplemental Security Income Records Maintenance System (SSIRMS), and Earnings Record Maintenance System (ERMS).

o Completed a general computer control review as it relates to MSSICS, SSIRMS and ERMS.

Used a programming specialist to determine the adequacy of the programming logic used by SSA to extract the data elements from the Work Incentive File.

Appendix C
Flowchart - Percent Increase in the Number of SSI Disabled Beneficiaries Earning at Least $100 per Month

Percent increase in the number of SSI disabled beneficiaries earning at least $100 per month

Supplemental Security Income (SSI) beneficiaries provided Social Security Administration (SSA) with earnings information. Earnings information was inputted into Modernized Supplemental Security Income Claims Systems (MSSICS).

Supplemental Security Income Records Maintenance System (SSIRMS) processed earnings information to the beneficiaries' Supplemental Security Record (SSR).

Earnings Record Maintenance System interfaced with SSIRMS to provide SSI beneficiary earnings information in the form of Master Earnings File alerts/diaries.

SSIRMS processed earnings information to the beneficiaries' SSR.

The Work Incentive File, a data extract from the SSR, was created.

The Work Incentive File was reformatted to obtain the performance indicator data.

The performance indicator data was transferred to an Excel spreadsheet.

The Excel spreadsheet was used to calculate the performance indicator results and the SSI Disabled Recipients Who Work Report, Table 4.

Appendix D
Agency Comments

SOCIAL SECURITY

MEMORANDUM 33296-24-1156

August 30, 2004

To: Patrick P. O'Carroll, Jr.
Acting Inspector General

From: Larry W. Dye
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report "Performance Indicator Audit: Employment for Disabled Beneficiaries" (A-02-04-14068)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Please let me know if you have any questions. Staff inquiries may be directed to Candace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL (OIG) DRAFT REPORT "PERFORMANCE INDICATOR AUDIT: EMPLOYMENT FOR DISABLED BENEFICIARIES" (A-02-04-14068)

We appreciate the opportunity to review and comment on the draft report. We agree that we can strengthen the security over the measurement process and take steps to better report performance in this area. Our responses to the specific recommendations are provided below. We have also included some technical comments for your consideration.

Recommendation 1

SSA should ensure that personnel do not have the ability, through inappropriate access, to directly modify, create or delete the datasets used to calculate the results of this indicator.

Response

We agree. We will evaluate the need to limit security access to the datasets to ensure the integrity of the production data and to conform to the principles of "least privileged access" or segregation of duties. We are pleased that the report contains no evidence that the data upon which this performance indicator is calculated had been inappropriately accessed or compromised.

Recommendation 2

SSA should consider revising the calculation of the performance indicator by averaging the interim data available for the indicator over a relevant period, rather than reporting it at a single point in time.

Response

We agree. The proposed change is more apt to reflect our efforts to promote several of the work incentive programs that assist Supplemental Security Income (SSI) disability recipients to return to work. We will consider revising the calculation method to determine fiscal year (FY) 2005 performance.

Recommendation 3

SSA should disclose and discuss all of the SSA programs that assist the SSI disabled beneficiaries to increase their monthly earnings.

Response

We agree. We will include a discussion of the appropriate programs in the FY 2005 PAR. Currently, we do disclose and discuss these programs to a diverse audience (i.e., advocacy groups, beneficiaries, general public, vocational rehabilitation, employment networks) through other channels (e.g., forums, town-hall meetings) and including this information in the PAR would further demonstrate our commitment to helping SSI disability recipients return to work.

Recommendation 4

SSA should document the policies and procedures used to prepare and disclose the results of the performance indicator.

Response

We agree and will provide documentation to our auditors (PwC), as it is related to Key Performance Indicators (KPIs) in the FY 2004 PAR. We have already started working with PwC to ensure that PwC has all the required documentation associated with the KPIs for the FY 2004 PAR.

Recommendation 5

SSA should ensure that the performance indicator results are properly reported in conformity with the data definition.

Response

We agree. Currently, information on an individual's assignment of their Ticket is maintained within the Disability Control File rather than on the Supplemental Security Record (SSR) or Master Beneficiary Record. We will consider enhancing SSR data elements to capture this information for inclusion in the Work Incentive File.

[SSA also provided technical comments which have been addressed in this report, as needed.]

Overview of the Office of the Inspector General

The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Executive Operations (OEO). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit

OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations

OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General

OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Executive Operations

OEO supports OIG by providing information resource management and systems security. OEO also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, OEO is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.