SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
PRODUCTIVITY

November 2004

A-15-04-14073

AUDIT REPORT

Mission

We improve SSA programs and operations and protect them against fraud, waste, and abuse by conducting independent and objective audits, evaluations, and investigations. We provide timely, useful, and reliable information and advice to Administration officials, the Congress, and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

By conducting independent and objective audits, investigations, and evaluations, we are agents of positive change striving for continuous improvement in the Social Security Administration's programs, operations, and management and in our own office.

MEMORANDUM

Date: November 17, 2004 Refer To:

To: The Commissioner

From: Acting Inspector General

Subject: Performance Indicator Audit: Productivity (A-15-04-14073)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 16 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. Attached is the final report that presents the results of three of the performance indicators PwC reviewed. For each performance indicator included in this audit, PwC's objectives were to:
Test critical controls over the data generation and calculation processes for the specific performance indicator;

Assess the overall adequacy, accuracy, reasonableness, completeness, and consistency of the performance indicator and supporting data; and

Determine if each performance indicator provides meaningful measurement of the program and the achievement of its stated objectives.

This report contains the results of the audit for the following indicators:

Percent Improvement in Agency Productivity,

Supplemental Security Income Aged Claims Processed per Workyear, and

Disability Determination Service Cases Processed per Workyear.

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at
(410) 965-9700.

Patrick P. O'Carroll, Jr.

Attachment

MEMORANDUM

Date: November 9, 2004

To: Acting Inspector General

From: PricewaterhouseCoopers LLP

Subject: Performance Indicator Audit: Productivity (A 15-04-14073)

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

To enhance the practical use of performance information, the Office of Management and Budget (OMB), in collaboration with other Federal agencies, developed the Program Assessment Rating Tool (PART), which is comprised of assessment criteria on program performance and management. The PART establishes a high, "good government" standard of performance and is used to rate programs in an open, public fashion. Two of the indicators included in this audit report, Supplemental Security Income (SSI) Aged Claims Processed per Workyear (PPWY) and Disability Determination Service (DDS) Cases PPWY, are PART measures.

OBJECTIVE

For each performance indicator included in this audit, our objectives were to:

1. Test critical controls over the data generation and calculation processes for the specific performance indicator.

2. Assess the overall adequacy, accuracy, reasonableness, completeness, and consistency of the performance indicator and supporting data.

3. Determine if each performance indicator provides meaningful measurement of the program and the achievement of its stated objectives.

We audited the following performance indicators as stated in the SSA
Fiscal Year (FY) 2003 Performance and Accountability Report (PAR):

Performance Indicator FY 2003 Goal FY 2003 Reported Results
Percent Improvement in Agency Productivity 2% 2.1%
SSI Aged Claims PPWY 497 556
DDS Cases PPWY 264 cases per workyear 270.4 cases per workyear

BACKGROUND

SSA's performance indicators of Percent Improvement in Agency Productivity, SSI Aged Claims PPWY, and DDS Cases PPWY aim to measure agency productivity. Productivity is the measure of the efficiency with which available resources are used to produce necessary products and services. It is the amount of work that is processed by an organization or an organizational entity using a specified amount of workpower. Workpower is a measure of labor time provided by all employees to support the work of the Agency. Productivity is normally expressed as an index comparing the value of output (e.g. claims, hearings, appeals, change of addresses) over the value of input (usually expressed in labor units, costs, or workyear) in 1 period (the base period) to the comparable ratio for a second, usually later period.

SSA's input for all of the indicators above is expressed as workyears. SSA defines a workyear as the measure of workpower equivalent to 1 FY of paid labor time provided by an SSA employee to support the work of the Agency and the programs that it administers (including paid non-duty time such as leave). For simplification and consistency among fiscal periods, an SSA workyear is generally defined as 2,080 straight-time hours or their equivalent.

The percent improvement in Agency productivity aims to measure entity--or agency-wide productivity. The SSI aged claims PPWY aims to measure the adequacy of the agency resource allocation for this vulnerable population. DDS Cases PPWY directly evaluates the DDS productivity in the disability process.

RESULTS OF REVIEW

Percent Improvement in Agency Productivity

FY 2003 Goal: 2%

Actual FY 2003 Performance: 2.1%

SSA met its goal.

Indicator Background

As one of the largest, most workload-intensive agencies in government, SSA has been reporting the productivity of its employees for over 30 years. SSA measures productivity solely in terms of its program mission workloads, of which SSI aged claims processed are a part.

The Percent Improvement in Agency Productivity indicator is developed by the Division of Cost Analysis (DCA) under the Deputy Commissioner for Finance, Assessment and Management. DCA is responsible for managing SSA's Cost Analysis System (CAS). CAS provides a wide range of management information and analysis on the Agency's administrative costs, workloads, labor utilization and productivity.

The percent change in productivity is measured by comparing the total number of SSA and DDS workyears that would have been expended to process current year workloads at the prior year's rates of production to the actual SSA and DDS workyears expended.

The workyear number is composed of paid workyear information obtained for all SSA's components from the Payroll Operational Data Storage (PAYODS) system through pre scripted queries and from the data files sent by the DDSs. The component workload information is obtained from various SSA sub-systems, including the Processing Center Action Control System (PCACS), Earnings and Recording Maintenance System, Disability Operational Data Store (DIODS) system, Hearing Office Tracking System (HOTS), and Supplemental Security Income Records Maintenance System (SSIRMS).

The workload and workyear components are compiled by analysts in DCA and uploaded into the Pre-Input Cost Analysis (PICA) system before input into CAS. Once in PICA they are reviewed for reasonableness. After the information is input into CAS, the SC3 Sum report is generated from CAS, which includes the consolidated information for workload and workyears for the period. The information on the SC3 Sum report is then manually input into the Excel productivity worksheet for comparison to the base year. (Refer to Appendix C for an overview of the process.)

Findings

We were unable to determine the reliability of the information reported within the PAR because SSA could not produce documentation providing a full audit trail of the information, processes and programs used to perform the productivity calculation for the performance indicator.

The SSA productivity performance indicator calculation uses data from the 55 established SSA-level workloads. The initial component data is obtained from numerous system files and reports with some limited manual processes. Component level workyears and counts for end-products are compiled to generate the Agency level workloads, which are used in the calculation of this indicator.

SSA management provided limited documentation including: reports used at various stages of the process to create the indicator; some of the processes performed in the CAS system; and information pertaining to the source of some of the data used in the calculation. However, the following documentation was not available for all 55 workloads and therefore SSA was unable to provide a complete audit trail for the recalculation and verification of the indicator:

A complete description and identification of the exact processes and computer programs used to obtain data from the source system files.

Clear identification of the exact system files and fields used to obtain initial source data.

A complete description and identification of the computer programs and interim calculations used to selectively combine, delete or alter original data, extracted from source system files, during each stage of the data refinement process.

A copy of the original source data extractions used in the calculation process to provide an audit trail for recalculation of the performance indicator.

Our audit also found that the calculation of the performance indicator does not include time spent by contractors, which impacts the overall meaningfulness of this indicator. The use of contractors varies from process to process, and contractor workyears may represent a significant input into SSA and DDS processes in some States.

SSA does not have an annual review process in place to ensure that relevant workloads are used in the calculation. This would ensure that the workloads included remain relevant and fully representative of SSA's priorities and actions and that the calculation reflects SSA's priorities and actions.

SSI Aged Claims Processed Per Workyear

FY 2003 Goal: 497

Actual FY 2003 Performance: 556

SSA met its goal.

Indicator Background

The main objective of the SSI program is to provide the basic cash support of needy aged, blind, and/or disabled individuals. The SSI aged program is a means based program intended to supplement the income of aged individuals. The SSI aged claims PPWY aims to measure the adequacy of the Agency resource allocation for this vulnerable population.

DCA is also responsible for calculating the SSI aged claims PPWY performance indicator. Similar to the percent improvement in agency productivity indicator, DCA utilizes CAS to calculate the SSI cases processed per workyear indicator.

The SSI aged claims processed per workyear is calculated as follows:

SSI Aged Claims Processed = SSI aged cases completed by field offices Workyears expended by Field Offices on this workload = Direct and Indirect time, including overhead (time spent on training, travel, leave, holidays, etc.)

The workyear number is composed of payroll information obtained for all SSA's components from the PAYODS system through pre-scripted queries. The number of SSI aged claims processed, also known as the workload, is obtained from various sources, including SSIRMS.

The workload and workyear components are compiled by analysts in DCA and uploaded into the PICA system before input into CAS. Once in PICA they are reviewed for reasonableness. After the information is input into CAS, the C2-15A Report is produced from CAS. The C2-15A Report includes the SSI aged claims PPWY figure.

Findings

We found the indicator to be meaningful. However, we were unable to determine the reliability of the information reported within the PAR because SSA could not readily produce documentation providing a full audit trail of the information, processes and programs used to perform the calculation of the SSI aged claims processed indicator.

SSA management provided limited documentation including reports used at various stages of the process to create the indicator and information pertaining to the source of some of the data used in the calculation. However, the following documentation was not available to provide a complete audit trail allowing full recalculation and verification of the indicator:

A complete description and identification of the exact processes and computer programs used to obtain data from the source system files.

Clear identification of the exact system files and fields used to obtain initial source data.

A complete description and identification of the computer programs and interim calculations used to selectively combine, delete or alter original data, extracted from source system files, during each stage of the data refinement process.

A copy of the original source data extractions used in the calculation process to provide an audit trail for recalculation of the performance indicator.

Disability Determination Service Cases Processed Per Workyear

FY 2003 Goal: 264 cases per workyear

Actual FY 2003 Performance: 270.4 cases per workyear

SSA met its goal.

Indicator Background

SSA's Office of Field Disability Operations (OFDO) is responsible for the budget formulation and execution for the DDSs. OFDO monitors the DDS' performance, productivity, accuracy and processing time.

OFDO calculates the DDS Cases PPWY indicator by generating the DDSs Staffing and Workloads Analysis Report and FD-15 report by using the DIODS system. DIODS is a relational database located at SSA Central office that captures and stores detailed data about DDS workloads.

FD-15 Report

The FD-15 report is a fiscal Year-to-Date (YTD) cumulative report. It shows the counts for projected initial case receipts YTD supplied by the Office of Disability, as well as actual DDS initial case receipts, cases disposed of, and pending cases. It also shows actual workyears (WY) used YTD and an annualized rate of WYs used. It displays full time, part-time, overtime and total full-time equivalent positions for each week, as well as cumulative full-time and examiner additions, losses, and attrition rate. Finally it displays productivity per workyear for current week, rolling 4 weeks, quarter, cumulative, and adjusted cumulative.

DDS cases processed per workyear is calculated in the FD-15 reports as follows:

DDS Cases Processed = Cases disposed of by DDSs
Workyears expended by DDSs = Direct and Indirect time, including overhead (time spent on training, travel, leave, holidays, etc.)

Information regarding cumulative dispositions by the DDSs is obtained via an interface between the National Disability Determinations Service System (NDDSS) and the DIODS system. The NDDSS is used by the DDSs to input case status information and transmit the information to SSA.

Information regarding workyears is obtained directly from the DDSs. DDSs are required to input on a weekly basis personnel information related to both direct and indirect time expended for all work in DIODS. (Refer to Appendix C for an overview of the process.) To ensure the accuracy of hours charged to SSA by DDSs is accurate, DDS directors must certify on a quarterly basis hours expended by DDS employees and contractors.

Findings

Improvements could be made regarding documentation, calculation and controls surrounding the underlying data of the performance indicator. We found that SSA had not documented policies and procedures related to the formal process used to collect, review and provide the performance indicator data.

We also found that there are no formal procedures in place to ensure that reviews of the FD-15 reports were performed in a timely manner. We understand that management was involved in the review only if anomalies were identified.

We reviewed a sample of 45 DDS quarterly certifications and found that the controls in place to ensure the accuracy of the DDS hours expended were not working as intended. We found that eight of the certification reports did not include a valid/authorizing signature. SSA Central Office and the Regional Office require a signature from the DDS Administrator or the DDS Fiscal Auditor for the certification.

A formal log of errors and corrective actions taken was not maintained for: 1) output product errors relating to the generation of the FD-15 report; and 2) errors found as part of the review by SSA of the quarterly certification of hours expended by the DDSs. The lack of a formal log diminishes SSA's ability to monitor and minimize errors in calculating the performance indicator.

Our review also revealed that the calculation of this indicator did not include time spent by contractors. An adjusted PPWY, which includes time spent by contractors, is calculated and reported internally, however; the results are not externally reported. The use of contractors varies from State to State. The contractors' workyears may be a significant input into the DDS process in some States. Depending on the extent of the use of contractors, the meaningfulness of this indicator may be significantly impacted by excluding contractors' time in the calculation.

The PAR narrative states that there has been a "steady increase in the cases processed per workyear" but the goal for FY 2003 represented a decline relative to the FY 2002 performance. The narrative additionally explains that the workyear calculation "is inclusive of everyone on the DDS payroll plus doctors under contract to the DDS." However, we found that not all contractor hours are included in the calculation. This is misleading.

RECOMMENDATIONS

We recommend SSA:

1. Improve documentation by maintaining documents that describe how the performance indicator goals were established, documenting the policies and procedures used to prepare and disclose the results of the performance indicators, and maintaining a complete audit trail for the results of the performance indicators.

2. Ensure that the performance indicator titles, definitions, and goals are explicit, complete, and consistent.

Specific to the performance indicators, "Percent Improvement in Agency Productivity" and "SSI Aged Claims Processed Per Workyear," we recommend SSA:

3. Establish formal written documentation for the performance indicator that includes a complete description and identification of the exact processes and computer programs used to obtain data from the source system files for all 55 workloads.

4. Establish formal written documentation for the performance indicator that includes a clear identification of the exact system files and fields used to obtain the initial source data for each workload.

5. Establish formal written documentation for the performance indicator that includes a complete description and identification of the computer programs and interim calculations used to selectively combine, delete or alter original data extracted from source system files during each stage of the data refinement process.

6. Establish formal written documentation for the performance indicator that includes a requirement to maintain an exact copy of the original source data extracted for each of the 55 workloads.

Specific to the performance indicator, "Percent Improvement in Agency Productivity," we recommend SSA:

7. Review on a yearly basis workloads included in the calculation to ensure they reflect SSA's priorities and actions.

Specific to the performance indicator, "Disability Determination Services Cases Processed per Workyear" we recommend SSA:

8. Implement formal procedures to ensure that reviews of the FD-15 reports are performed in a timely manner.

9. Ensure that the Forms 4514 are certified by DDS personnel with the appropriate level of authority.

10. Ensure that a formal log of errors and corrective actions taken is maintained for output product errors relating to the generation of the FD-15 report, and errors found as part of the review by SSA of the quarterly certification of hours expended by the DDS. The log should contain the date, error, and corrective action taken in a detailed form.

AGENCY COMMENTS

SSA agreed with the recommendations in our report. However, for recommendations 3, 4, and 5, the Agency stated it does not have the systems resources to implement them at this time.

Appendices

APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Agency Comments

Appendix A
Acronyms

CAS Cost Analysis System
DCA Division of Cost Analysis
DDS Disability Determination Services
DIODS Disability Operational Data Store
FDDS Federal Disability Determination Services
FY Fiscal Year
GPRA Government Performance and Results Act
HOTS Hearing Office Tracking System
NDDSS National Disability Determination Services System
ODIO Office of Disability and International Operations
OEO Office of Executive Operations
OFDO Office of Field Disability Operations
OHA Office of Hearings and Appeals
OMB Office of Management and Budget
PAR Performance and Accountability Report
PART Performance Assessment Rating Tool
PAYODS Payroll Operational Data Storage
PCACS Processing Center Action Control System
PI Performance Indicator
PICA Pre-Input Cost Analysis
PPWY Processed per Work Year
P/R ODS Payroll Operational Data Store
PSC Program Service Center
Pub. L. No. Public Law Number
PwC PricewaterhouseCoopers LLP
SSA Social Security Administration
SSAMIS Social Security Administration Management Information System
SSI Supplemental Security Income
SSIRMS Supplemental Security Income Records Maintenance System
U.S.C. United States Code
WY Workyear
YTD Year-to-Date

Appendix B
Scope and Methodology
We updated our understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing including testing of source documentation, we performed the following as applicable:

Reviewed prior SSA, Government Accountability Office, and other reports related to SSA's GPRA performance and related information systems.

Met with the appropriate SSA personnel to confirm our understanding of each individual performance indicator.
Flowcharted the processes (see Appendix C).

Where applicable, we tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).

Conducted and evaluated tests of the automated and manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.

Identified and extracted data elements from relevant systems and obtained source documents for detailed testing selections and analysis.

Identified attributes, rules, and assumptions for each defined data element or source document.

Tested the adequacy, accuracy, reasonableness, consistency, and completeness of the selection.

Recalculated the metric or algorithm of key performance indicators to ensure mathematical accuracy.

For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators being used appear to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes. We followed all performance audit standards.

In addition to the steps above, we specifically performed the following to test the indicators included in this report:

PERCENT IMPROVEMENT IN AGENCY PRODUCTIVITY

Tested the effectiveness of SSA's payroll and time attendance controls by selecting a sample of 45 payroll employee records to test the accuracy of the workyear information.

Reviewed the controls over the reporting of hours worked by the Disability Determination Services (DDS).

Tested the controls in place to ensure the appropriateness of the monthly SC3 Sum reports.

SSI AGED CLAIMS PROCESSED PER WORKYEAR (PPWY)

Reviewed the controls surrounding the generation of the C2-15A report.

DISABILITY DETERMINATION SERVICE CASES PPWY

Reviewed the controls surrounding the DDS quarterly payroll certification process.

Tested the accuracy of the workload information in Disability Operational Data Store (DIODS) by selecting 45 Disability Quality Assurance cases and ensuring that the disposition information in the files agreed to the information in DIODS.

Tested the controls and accuracy of the FD-15 reports.

Appendix C
Flowchart of Percent Improvement in Agency Productivity

Percent Improvement in Agency Productivity

Control workyears (WYs)
DCA Analyst extract P/R ODS data
Calculate Agency control WYs by component
Is data reasonable and accurate?
Yes - Approve and distribute to DCA analyst
No - DCA analyst re-extracts P/R ODS data
DCA Analyst perform split by category and workload
Sent to Feeder CAS
Workload counts
Include workload counts from DDS, Federal Disability Determination Services(FDDS), DDS, Office of Hearings and Appeals(OHA), Field Offices, Social Security Administration Management Information System(SSAMIS), Office of Executive Operations(OEO), Program Service Center(PSC), and Office of Disability and International Operations(ODIO)
DCA Analyst extract workload data
Is data reasonable and accurate?
Yes - Sent to Feeder CAS
No - DCA analyst investigates and resolves the issue and re-extracts workload data
Feeder CAS processing
Is data reasonable and accurate?
Yes - Create PICA
No - Sent to DCA Analyst for resolution
Sent to Feeder CAS for reprocessing after resolution
Create PICA
Is data reasonable and accurate?
Yes - Upload to CAS
No - Sent to DCA Analyst for resolution
Create new PICA after resolution
Run CAS
Generate reports and distribute to DCA Analysts
Is data reasonable and accurate?
Yes - Data extracted from CAS and input to Excel file
No - Sent to DCA Analyst for resolution
After resolution, uploaded to CAS before it is run again
Is data reasonable and accurate?
Yes - Distribute to Division Director for Approval
No - Sent to DCA Analyst to investigate and resolve
Is data reasonable and accurate?
Yes - Distribute to Office Director for Approval
No - Sent to DCA Analyst to investigate and resolve
Is data reasonable and accurate?
Yes - Distribute reports
No - Sent to DCA Analyst to investigate and resolve
Flowchart of SSI Aged Claims PPWY

Supplemental Security Income (SSI) Aged Claims PPWY

Control workyears
DCA Analyst extract P/R ODS data
Calculate Agency control WYs by component
Is data reasonable and accurate?
Yes - Approve and distribute to DCA analyst
No - DCA analyst re-extracts P/R ODS data
DCA Analyst perform split by category and workload
Sent to Feeder CAS
Workload counts
Include workload counts from DDS, FDDS, OHA, Field Offices, SSAMIS, OEO, PSC, and ODIO
DCA Analyst extract workload data
Is data reasonable and accurate?
Yes - Sent to Feeder CAS
No - DCA analyst investigates and resolves the issue and re-extracts workload data
Feeder CAS processing
Is data reasonable and accurate?
Yes - Create PICA
No - Sent to DCA Analyst for resolution
Sent to Feeder CAS for reprocessing after resolution
Create PICA
Is data reasonable and accurate?
Yes - Upload to CAS
No - Sent to DCA Analyst for resolution
Create new PICA after resolution
Run CAS
Generate C2-15A report
Is data reasonable and accurate?
Yes - Distribute to users
No - Sent to DCA Analyst for resolution
After resolution, data is uploaded again to CAS and CAS is rerun
C2-15A data for SSI Aged claims provided for input to Performance and Accountability Report

Flowchart of DDS Cases PPWY

DDS Cases PPWY

DDS inputs workload data into National Disability Determination Service System (NDDSS)
NDDSS processes workload data
Data sent to Disability Operational Data Store (DIODS)
Personnel information inputted to DIODS
FD-15 Report produced weekly from DIODS
Staff reviews weekly FD-15 Reports to identify anomalies so that obvious errors are corrected
Year-end Cumulative FD-15 Repot is used to produce year end total for Performance Indicator (PI) #14 and Accountability Report

Appendix D
Agency Comments

SOCIAL SECURITY

MEMORANDUM 33296-24-1161

Date: September 27, 2004

To: Patrick P. O'Carroll, Jr.
Acting Inspector General

From: Larry W. Dye
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report "Performance Indicator Audit: Productivity" (A-15-04-14073)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Let me know if we can be of further assistance. Staff inquiries may be directed to Candace Skurnik, Director, Audit Management and Liaison Staff on extension 54636.

Attachment:
SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL (OIG) DRAFT REPORT "PERFORMANCE INDICATOR AUDIT: PRODUCTIVITY"
(A-15-04-14073)

Thank you for the opportunity to review and comment on the draft report.

Our responses to the specific recommendations are provided below.

Recommendation 1

Improve documentation by maintaining documents that describe how the performance indicator goals were established, documenting the policies and procedures used to prepare and disclose the results of the performance indicators, and maintaining a complete audit trail for the results of the performance indicators.

Response

We agree. We are currently determining the best approach for maintaining documentation regarding how performance indicators and related goals are established as part of the audit trail.

Recommendation 2

Ensure that the performance indicator titles, definitions, and goals are explicit, complete, and consistent.

Response

We agree. In preparation of the FY 2004 Performance and Accountability Report and the FY 2005/2006 Annual Performance Plan, the Office of the Chief Strategic Officer worked with components to ensure performance titles, definitions, and goals were explicit, complete, and consistent. We will continue to monitor this area.

Recommendation 3

Establish formal written documentation for the performance indicators that includes a complete description and identification of the exact processes and computer programs used to obtain data from the source system files for all 55 workloads.

Response

We agree. However, we do not have the budgeted systems resources to implement them at this time.

Recommendation 4

Establish formal written documentation for the performance indicators that includes clear identification of the exact system files and fields used to obtain source data for each workload.

Response

See response to Recommendation 3.

Recommendation 5

Establish formal written documentation for the performance indicators that includes a complete description and identification of the computer programs and interim calculations used to selectively combine, delete or alter original data extracted from source system files during each stage of the data refinement process.

Response

See response to Recommendation 3.

Recommendation 6

Establish formal written documentation for the performance indicators that includes a requirement to maintain an exact copy of the original source data extracted for each of the 55 workloads.

Response

We agree. However, we suggest for economy of time and efficiency of space to store information, that exact copies only be kept for data that cannot be easily replicated once it has been extracted. Further, we do not have the budgeted systems resources to implement at this time.

Recommendation 7

Review on a yearly basis workloads included in the calculation to ensure they reflect SSA's priorities and actions.

Response

We agree. We will work with the sponsoring component to ensure this review occurs on a regular basis.

Recommendation 8

Implement formal procedures that ensure that reviews of the FD-15 reports are performed in a timely manner.

Response

We agree in part. The FD-15 reports are always reviewed routinely and in a timely manner as soon as they are available each week. Nevertheless, we will formally document this process.

Recommendation 9

Ensure that the Forms 4514 are certified by DDS personnel with the appropriate level of authority.

Response

We agree in part. The POMS instructions already require a signature on the SSA-4514. We will ensure routine enforcement by monitoring 4514 for signatures and return unsigned forms for correction.

Recommendation 10

Ensure that a formal log of errors and corrective actions taken is maintained for output product errors relating to the generation of the FD-15 report, and errors found as part of the review by SSA of the quarterly certification of hours expended by the DDS. The log should contain the date, error, and corrective action taken in a detailed form.

Response

We agree. The Office of Disability and Income Security Programs will coordinate with the Office of Systems to maintain a log of errors and corrective actions on the Disability Operational Data Store (DIODS) web page, for errors relating to the generation of the FD-15 Report. The Office of Disability Determinations will maintain a separate file log of errors and corrective actions for errors detected as a part of the review by SSA of the quarterly certification of hours expended by each DDS. A log of errors and corrective actions would enhance monitoring and minimize errors in the calculation of this performance indicator. We will implement the maintenance of the FD-15 error log, and will immediately begin maintaining our log on hours certification reporting errors.

[SSA also provided technical comments which have been addressed in this report, where appropriate.]

Overview of the Office of the Inspector General

The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Executive Operations (OEO). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit

OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations

OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General

OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Executive Operations

OEO supports OIG by providing information resource management and systems security. OEO also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, OEO is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.