Date: January 27, 2006

To: The Commissioner

From: Inspector General

Subject: Performance Indicator Audit: Outstanding Debt (A-02-05-15116)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 16 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. Attached is the final report presenting the results of two of the performance indicators PwC reviewed. For the performance indicators included in this audit, PwC's objectives were to:

Assess the effectiveness of internal controls and test critical controls over the data generation, calculation, and reporting processes for the specific performance indicator.

Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.

Test the accuracy of results presented and disclosed in the Fiscal Year 2004 Performance and Accountability Report.

Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

This report contains the results of the audit for the following indicators:

" Percent of Outstanding Old Age, Survivors, and Disability Insurance Debt in a Collection Arrangement.
" Percent of Outstanding Supplemental Security Income Debt in a Collection Arrangement.

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at
(410) 965-9700.

Patrick P. O'Carroll, Jr.

OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
OUTSTANDING DEBT

January 2006

A-02-05-15116

AUDIT REPORT

Mission

We improve SSA programs and operations and protect them against fraud, waste, and abuse by conducting independent and objective audits, evaluations, and investigations. We provide timely, useful, and reliable information and advice to Administration officials, the Congress, and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

By conducting independent and objective audits, investigations, and evaluations, we are agents of positive change striving for continuous improvement in the Social Security Administration's programs, operations, and management and in our own office.

MEMORANDUM

Date: January 23, 2006

To: Inspector General

From: PricewaterhouseCoopers, LLP

Subject: Performance Indicator Audit: Outstanding Debt (A-02-05-15116)

OBJECTIVE

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

Our audit was conducted in accordance with generally accepted government auditing standards for performance audits. For the performance indicator included in this audit, our objectives were to:

1. Assess the effectiveness of internal controls and test critical controls over the data generation, calculation, and reporting processes for the specific performance indicator.

2. Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.

3. Test the accuracy of results presented and disclosed in the Fiscal Year (FY) 2004 Performance and Accountability Report (PAR).

4. Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

BACKGROUND

We audited the following performance indicators as stated in the SSA FY 2004 PAR:

Performance Indicator FY 2004 Goal FY 2004 Reported Results
Percent of Outstanding Old Age, Survivors, and Disability Insurance (OASDI) Debt in a Collection Arrangement 38% 42.2%
Percent of Outstanding Supplemental Security Income (SSI) Debt in a Collection Arrangement 55% 53.5% (May - September)

Under the OASDI and SSI programs, SSA issues benefit payments to millions of beneficiaries for various entitlement programs. At times, SSA pays beneficiaries more than they are entitled to receive. This results in an overpayment that SSA then recognizes as an outstanding debt owed by the beneficiary. During the audit period, SSA had $4 billion dollars of OASDI and SSI debt that had not been resolved (i.e. has not been collected, removed as not actually an overpayment, waived, or written off conditionally). New debt that is determined to be an outstanding overpayment eligible for collection primarily occurs as result of errors in: Wages/Self-Employment Income (Old Age, and Survivors Insurance (OASI) and SSI), Computations (OASI), Annual Earnings Test (OASI), Relationship/Dependency (OASI), Substantial Gainful Activity (DI), Unreported Death (DI), Workers' Compensation (DI) and Financial Accounts (SSI).

SSA classifies outstanding debt in four principal categories: 1) newly established debt (i.e. debt in a due process period and less than 60 days old), 2) debt under appeal (i.e. debt with an appeal pending and less than 90 days old), 3) debt in a collection arrangement (i.e. collection is anticipated or debt is being collected, primarily through timely installments or monthly benefit withholding), and 4) debt not in a collection arrangement (i.e. all remaining debt that is neither in collection nor anticipated to be collected, but requires further action). These performance indicators were developed to target an average annual percentage of debt in a collection arrangement compared to the overall status of outstanding debt.

When an overpayment is detected, it is initially placed in the "newly established debt" category and may remain in this category for up to 60 days, the maximum period permitted for a beneficiary to appeal or apply for a waiver. After 60 days, the debt moves to either "debt in a collection arrangement" or "debt not in a collection arrangement." At any time, if the overpayment has been determined to be "debt in a collection arrangement," the amount would be moved to this category in the monthly debt report, which is used to calculate the indicator results. Delinquent debt, i.e. "debt not in a collection arrangement" is typically referred to the Treasury Offset Program and credit bureaus (i.e. Equifax, TransUnion, Innovis and Experian).

At the end of each month, SSA uses software packages ("DEBTCATC" for OASDI and "SSIDEBT29" for SSI) to review the data generated from the relevant source systems (Recovery of Overpayment, Accounting and Reporting (ROAR) System for OASDI and the Supplemental Security Record (SSR) for SSI). The software packages determine the status of each unresolved debt based on the four principal categories and then sum the amounts for each principal category to determine the proportion of debt in a collection arrangement. The percentage calculated represents the proportion of debt in a collection arrangement for the most recent 12 months. The indicator results presented and disclosed in the PAR represent a 12-month rolling average.

Performance Indicators' Calculation

% of Debt in a Collection Arrangement

Total Dollar Amount in a Collection Arrangement / Total Dollar Amount of Outstanding Debt

(For additional detail on the calculation of these indicators, refer to the flowcharts in Appendix C.)

These performance indicators are linked to the SSA strategic objective to "Prevent fraudulent and erroneous payments and improve debt management," which is linked to the SSA strategic goal "To ensure superior Stewardship of Social Security programs and resources." This strategic goal is linked to one of the five Governmentwide goals on the President's Management Agenda, "Improved Financial Management," which directs agencies to set goals to reduce improper payments for each of their programs. To assess its progress in meeting this objective, SSA established these performance indicators to measure the percentage of OASDI and SSI debt in a collection arrangement.

RESULTS OF REVIEW

We found SSA did not have adequate internal controls over the processes used to generate, calculate, and report the performance data used in the performance indicators and did not retain support for the calculations. As a result, we were unable to examine sufficient support for the reported indicators to conclude on the accuracy of the calculation. Additionally, the indicators did not provide a clear linkage of: SSA's progress to align with existing Federal debt collection laws; relationships to complementary debt collection measures to enable a comprehensive assessment of overall debt collection efforts; and SSA's decision to maintain, not raise, annual performance targets. Lastly, SSA can further develop a more meaningful indicator to measure debt that is eligible to be collected.

Internal Controls

We found that SSA did not have sufficient documentation of the indicator's intended purpose, automated and manual controls in operation, accountability for indicator measurement, a comprehensive set of supporting data sources used in the calculation, and step-by-step procedures to reperform the indicator results from origination to publication in the PAR. Accordingly, SSA did not comply with standards defined by the Office of Management and Budget's (OMB) Circular A-123, Management Accountability and Control that require, "The documentation for transactions and management controls, and other significant events must be clear and readily available for examination."

The following finding is specific to the "Percent of Supplemental Security Income (SSI) Debt in a Collection Arrangement" performance indicator. During our FY 2004 Financial Statement Audit, we found that SSA policy was not adhered to for approvals of overpayment waivers over $2,000 in 1 out of 9 field offices examined. Further, we found inaccurate overpayment calculations existed in 1 out of 46 overpayments examined. In addition, during an audit of SSA's Controls over the Title XVI Overpayment Waiver Process, the SSA Office of the Inspector General found significant deficiencies in documentation support and policy decisions for waiver approvals. These weaknesses identified in internal controls over overpayment processing may impact the indicator results, as the overpayments waived would have affected the denominator for the calculation of this indicator, and thereby incorrectly increase the percentage reported.

Data Reliability and Accuracy of PAR Presentation and Disclosure

We found that SSA did not archive the data supporting the performance indicators' calculation. As a result, we were unable to conclude on the reliability of the computer processed data used to support the performance indicators' calculation. In addition, as a result of our inability to recalculate the performance results, we were unable to determine if the results reported in the PAR were accurately presented and disclosed. SSA is currently examining data storage possibilities to address this issue.

We found that SSA's programmers have ALL access to production datasets. This level of access would allow users to create, delete and update any of the data we reviewed without appropriate review or approval of the changes. This access level does not comply with generally accepted standards for limited access control defined by OMB Circular A-130, National Institute of Standards and Technology (NIST) 800-12, and NIST 800-14. SSA is currently working to address the risks associated with this finding through the Standardized Security Profile Project. These datasets should be secured over the next 3 years.

The following finding is specific to the "Percent of Supplemental Security Income (SSI) Debt in a Collection Arrangement" performance indicator. We found that SSA did not use a standard change control process for the Common Business Oriented Language (COBOL) Programs used in this indicator's calculation. The COBOL programs, SSIDBT29 and GOALTOT6, used to calculate the performance measure did not adhere to the SSA system development life cycle process, QA2/SRCOL. In addition, these programs are not stored in Endeavor but are maintained in personal libraries, where they are not subject to SSA's standard system development life cycle process. By not following the SSA standard change control process, changes to the COBOL programs may not be authorized, tested, or documented appropriately.

Assessment of Meaningful Performance Measurement

We found that SSA could report more meaningful, outcome-oriented results, by disclosing complementary measures such as the percentage of debt written off each year, percentage of new debt being collected each year, and aging reports. Although several of these indicators are included throughout the PAR, as presented, the performance indicators' did not directly disclose sufficient information to enable an independent reviewer to assess the overall status of debt collection efforts and progress.

We further found that SSA could develop a more meaningful indicator by only measuring debt that is eligible to be collected. The current indicators' denominator includes newly established debt and debt under appeal, the first and second principal debt categories, which are not eligible to be collected.

In our Management Letter resulting from the 2004 Financial Statement Audit, we found SSA did not completely disclose its efforts to improve debt collection in compliance with the Debt Collection Improvement Act (DCIA) of 1996. SSA disclosed several efforts to improve debt collection, however, the DCIA authorized SSA to implement several specific procedures to improve overall collection rates and granted Agencies the discretion to implement the Act in accordance with policies outlined by the Commissioner. SSA has determined that certain tools will be followed and phased in over the next 12-24 months. Clearly linking the indicator to other areas in the PAR and disclosing SSA's progress and efforts to improve debt collections using available tools would make the presentation of the indicator results more meaningful.

We found that SSA did not develop annual, incremental improvement goals for these performance indicators that supported a clear linkage to the strategic objective to "Prevent fraudulent and erroneous payments and improve debt management." SSA management stated that the Agency sets "maintenance goals" and seeks to achieve the same level of performance as the previous year. Although this indicator was not created as part of SSA's Program Assessment Rating Tool (PART) efforts, OMB PART guidance states, "Goals should be relative to an established baseline, have clear time frames and targets, and challenge program managers to continuously improve program performance." SSA management did not provide an explanation in the PAR to justify their decision to not establish incremental improvement goals as described in the OMB PART guidance.

CONCLUSION AND RECOMMENDATIONS

SSA should have adequate controls over the generation, calculation, and reporting of performance data to ensure a reliable measurement of performance. Equally important is the development of performance indicators that meaningfully provide an assessment of the outcome of efforts to improve collection of outstanding debt. Accordingly, for each performance indicator, we recommend SSA:

1. Improve the documentation of the performance indicators included in this report. The documentation should be complete and accurate to define all of the steps performed, accountability for measurement, and all internal controls in operation used to calculate and report the indicator results in the PAR. SSA management should annually update and take ownership of the documentation to enable an independent assessment that complies with OMB A-123 guidance.

2. Retain the detailed data used to calculate the performance indicators results that are reported in the PAR.

3. Continue to implement the Standardized Security Profile Project to restrict access to personnel that should not have the ability to directly modify, create or delete the datasets used to calculate the indicators' results.

4. Provide additional links to other sections of the PAR that discuss progress made (e.g. implementation of new debt collection tools) indicating increased alignment with laws and regulations that are related to the performance indicators' results.

5. Provide additional transparency of overall debt collection efforts, link the existing indicators to complementary debt collection measures available both in the PAR and internal management reports. This linkage should clearly disclose the effect of such information on the indicators' calculation and the overall progress of debt collection efforts. The PAR may also include a graphical representation, e.g. a pie chart, to identify annual changes in each debt collection category.

6. SSA should consider revising the existing performance measure calculation to include in its denominator only debt that is eligible to be collected. Newly established debt and debt under appeal, the first and second principal debt categories, are not eligible for collection. We suggest that the calculation's numerator include debt in a collection arrangement, the third principal debt category, and the denominator include both debt in a collection arrangement and debt not in a collection arrangement, the fourth principal debt category. This will create a more meaningful measure that demonstrates the percent of all debt that was eligible to be collected that resulted in a debt collection arrangement.

7. Enhance disclosure of SSA's decision not to establish incremental improvement goals that results in a departure from OMB guidance.

Specific to the performance indicator "Percent of Supplemental Security Income (SSI) Debt in a Collection Arrangement," we recommend SSA:

8. Ensure COBOL programs adhere to the SSA System Development Life Cycle. In addition, the programs should be maintained in a version control system in the event the program must be backed out and reinstalled.

AGENCY COMMENTS

The Agency agreed with four of our recommendations (numbers 1, 3, 5 and 8) and disagreed with four recommendations (numbers 2, 4, 6 and 7). For recommendation number 6, SSA considered our suggested change to the performance measure calculation. However, SSA decided to include all unresolved debt in the formula to be consistent with other Agency reports related to outstanding debt, such as the quarterly Treasury Report on Receivables to the Treasury Department. Similarly, for recommendation number 7, SSA is currently developing new performance measures as part of the President's Management Agenda initiative on improper payments.

PWC RESPONSE

We appreciate the Agency's comments and consideration of our recommendations. In regard to recommendation number 2, we recognize some of the technical challenges associated with maintaining such detailed data and we understand the Agency must weigh the cost and benefit of retaining source data. Although we will incorporate real-time audit approaches in ongoing performance measure reviews, these alternative audit steps will not totally negate the Agency's need to retain detailed documentation. As such, we continue to believe the Agency needs to develop methods to retain the detailed data used to calculate the performance indicator results reported in the PAR. The lack of data inhibits an independent verification of the performance metrics reported in the PAR.

For recommendation 4, we acknowledge that SSA's PAR contains some linkage to other sections of the PAR with information related to outstanding debt. However, we believe this linkage can be improved. To that end, we have revised recommendation 4 to reflect that we are recommending additional linkage.

With respect to recommendation 6, we consider this closed as SSA considered the recommendation and made a business decision to do otherwise. In the event that the newly developed performance indicators referenced in the Agency's response to recommendation 7 depart from OMB guidance for goal development, we recommend that SSA ensure sufficient disclosure of the business reasons for such departures are included in the PAR.

Appendices

APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Agency Comments

Appendix A
Acronyms

COBOL Common Business Oriented Language
DCFAM Deputy Commissioner of Finance, Assessment, and Management
DCIA Debt Collection Improvement Act
DI Disability Insurance
FY Fiscal Year
GAO Government Accountability Office
GPRA Government Performance and Results Act
MBR Master Beneficiary Record
MIICR Management Information Initial Claims Record
MSSICS Modernized Supplemental Security Income Claims Systems
NIST National Institute of Standards and Technology
OASI Old Age and Survivors Insurance
OASDI Old Age, Survivors, and Disability Insurance
OMB Office of Management and Budget
OQA Office of Quality Assurance and Performance Assessment
OSM Office of Strategic Management
PAR Performance and Accountability Report
PART Program Assessment Rating Tool
ROAR Recovery of Overpayments & Accounts Receivable
SSA Social Security Administration
SSI Supplemental Security Income
SSIPT Supplemental Security Income Processing Time
SSIRMS Supplemental Security Income Records Maintenance System
SSR Supplemental Security Record
U.S.C. United States Code

Appendix B
Scope and Methodology
We updated our understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing, including testing of source documentation, we performed the following:

Reviewed prior SSA, Government Accountability Office, Office of the Inspector General and other reports related to SSA's GPRA performance and related information systems.
Met with the appropriate SSA personnel to confirm our understanding of the performance indicator.
Flowcharted the process. (See Appendix C).
Tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).
Conducted and evaluated tests of the automated and manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.
Identified attributes, rules, and assumptions for each defined data element or source document.
Identified and extracted data elements from relevant systems and obtained source documents for detailed testing selections and analysis.
Recalculated the metric or algorithm of key performance indicators to ensure mathematical accuracy.
For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators appear to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes.

We followed all performance audit standards in accordance with generally accepted government auditing standards.

In addition to the steps above, we specifically performed the following to test the indicators included in this report:

PERCENT OF OUTSTANDING OLD AGE, SURVIVORS, AND DISABILITY INSURANCE (OASDI) DEBT IN A COLLECTION ARRANGEMENT

Performed an application review over the Title II systems (Modernized Claims System, Retirement, Survivors & Disability Insurance Accounting and Recovery of Overpayment, Accounting and Reporting (ROAR)).
Performed a general controls review over the environment that supports the Title II applications and ROAR.
Inquired of the interface process from the Master Beneficiary Record (MBR) to ROAR, including resolution of rejected records.
Inspected a selection of interfaces between the MBR/ROAR and performed a count of records.
Reviewed the code to create the DEBTCATC program for reasonableness.

PERCENT OUTSTANDING SUPPLEMENTAL SECURITY INCOME (SSI) DEBT IN A COLLECTION ARRANGEMENT

Performed an application review over the Title XVI systems (Modernized Supplemental Security Income Claims Systems and Supplemental Security Income Records maintenance System).
Performed a general controls review over the environment that supports the Title XVI applications.
Reviewed the Common Business Oriented Language (COBOL) programs 1 and 2 for reasonableness of the data extraction and summarization of the SSI Debt Goal Report.
Inspected users with update access to the program and the SSI Debt Goal Report.
Inquired of the process to perform program maintenance over the creation of the SSI Debt Goal Report.
Inspected a selection of program maintenances during the fiscal year and reviewed to ensure that maintenance followed the Federal Information System Controls Audit Manual.

Appendix C
Percent of Outstanding OASDI Debt in a Collection
Arrangement - 2004 Process Flowchart

Percent of Outstanding OASDI Debt in a Collection
Arrangement - 2004 Process Flowchart Narrative

The Title II overpayment is detected by Social Security Administration (SSA) systems or SSA Operations.
An overpayment record is established on the MBR and ROAR.
A notice is sent to the beneficiary and an agreement is established for repayment of the outstanding debt.
A program designed by Office of Quality Assurance and Performance Assessment called DEBTCATC is run against a back-up copy of the ROAR master file and produces monthly debt collection information which is used to calculate the performance measure. Using the DEBTCATC program, the amounts for each principal category of debt (new debt, debt under appeal, debt in a collection arrangement, and debt not in a collection arrangement) are summed and the portion in a collection arrangement is determined by dividing the amount in that category by the total amount of unresolved debt (i.e. the sum of the four principal debt Categories).
The DEBTCATC report is used as an update to a 12-month rolling average done by manual calculation. The 12-month rolling average is calculated by dropping off the percent of dollars in collection for the earliest month, adding the percent for the latest month and dividing the total by 12. The result of that calculation is the goal achievement for the current month. The process of calculating the goal is repeated each month.
DCFAM provides the information to OSM for the PAR.

Percent of Outstanding SSI Debt in a Collection Arrangement - 2004 Process Flowchart
Percent of Outstanding SSI Debt in a Collection Arrangement - 2004 Process Flowchart Narrative
The Title XVI overpayment is detected by SSA systems or SSA Operations.
An overpayment record is established on the SSR.
A notice is sent to the beneficiary and an agreement is established for repayment of the outstanding debt.
A COBOL program called SSIDEBT29 is run against all 20 national file segments of the SSR to determine the status of debts.
A second COBOL program is run to total the 20 national file segments and produces monthly debt collection information on the SSR National Debt Goal Report. The amounts for each principal category of debt (new debt, debt under appeal, debt in a collection arrangement, and debt not in a collection arrangement) are summed and the portion in a collection arrangement is determined by dividing the amount in that category by the total amount of unresolved debt (i.e. the sum of the four principal debt categories).
The SSI National Debt Goal report is used as an update to a 12-month rolling average done by manual calculation. The 12-month rolling average is calculated by dropping off the percent of dollars in collection for the earliest month, adding the percent for the latest month and dividing the total by 12. The result of that calculation is the goal achievement for the current month. The process of calculating the goal is repeated each month.
DCFAM supplies the information to the OSM for PAR.

Appendix D
Agency Comments

SOCIAL SECURITY

MEMORANDUM

December 19, 2005

To: Patrick P. O'Carroll, Jr.
Inspector General

From: Larry W. Dye/
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: Outstanding Debt" (A-02-05-15116)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report's recommendations are attached.

Please let me know if you have any questions. Staff inquiries may be directed to
Candace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL DRAFT REPORT, "PERFORMANCE INDICATOR AUDIT: OUTSTANDING DEBT" (A-02-05-15116)

Thank you for the opportunity to review and comment on the draft report.

Our responses to the specific recommendations are provided below.

Recommendation 1

Improve the documentation of the performance indicators (PI) included in this report. The documentation should be complete and accurate to define all of the steps performed, accountability for measurement, and all internal controls in operation used to calculate and report the PI results in the Performance and Accountability Report (PAR). SSA management should annually update and take ownership of the documentation to enable an independent assessment that complies with the Office of Management and Budget (OMB) A-123 guidance.

Response

We agree. As stated in the report on page 4, footnote 13, we provided updated documentation in April 2005 to adhere to this recommendation. We will continue to review our policies and procedures to ensure proper documentation for each fiscal year (FY).

Recommendation 2

Retain the detailed data used to calculate the PIs results that are reported in the PAR.

Response

We disagree. Satisfying this recommendation would require SSA to preserve and maintain, among other things, data transactions, source code, multiple versions of software and the operating system in use during the potential audit review period. Staff would then need to be available to reconstruct all this to support an audit. Further, OMB's Circular A-11, section 203(f) states "Performance data need not be perfect to be reliable, particularly if the cost and effort to secure the best performance data will exceed the value of any data so obtained."

During the course of the audit, we offered to have the PricewaterhouseCoopers (PwC) auditors observe the programmers as the goals reports were being generated. PwC also had the opportunity to conduct their own "real-time" calculation of the performance measure (at or about the time of the month when the performance measures were being produced) using the programs and specifications that they were provided. The PwC auditors choose not to do so.

Recommendation 3

Continue to implement the Standardized Security Profile Project (SSPP) to restrict access to personnel that should not have the ability to directly modify, create or delete the datasets used to calculate the PIs results.

Response

We agree. The Agency recently reached a critical juncture in the limitation of programmer access to its most high-risk systems, thus lifting a reportable condition assessment. SSA will continue to implement and monitor the progress of SSPP. Additionally, SSA will target, for implementation of SSPP, those systems related to the development of Government Performance and Results Act performance measures.

Recommendation 4

Clearly link discussions in other sections of the PAR that discuss progress made (e.g., implementation of new debt collection tools) indicating increased alignment with laws and regulations that are related to the PIs results.

Response

We disagree. We believe that the information in the PAR already contains the clear linkage that is being recommended. In the Debt Management summary on page 218 of the FY 2004 PAR, SSA's progress in implementing the various debt collection tools authorized by law are clearly discussed, as is the performance indicator.

Recommendation 5

Provide additional transparency of overall debt collection efforts; link the existing indicators to complementary debt collection measures available both in the PAR and internal management reports. This linkage should clearly disclose the effect of such information on the PIs calculation and the overall progress of debt collection efforts. The PAR may also include a graphical representation, e.g., a pie chart, to identify annual changes in each debt collection category.

Response

We partially agree. We agree that the Agency can better link the performance indicator to other relevant portions of the PAR and how it relates to the President's Management Agenda (PMA) initiative of "Improper Payments". We believe we already have an appropriate level of information supporting our debt collection performance indicators in the PAR, and do not believe it is necessary or appropriate to expand this external document to discuss internal management reports or add additional graphics on other measures.

Recommendation 6

Consider revising the existing PI calculation to include in its denominator only debt that is eligible to be collected. Newly established debt and debt under appeal, the first and second principal debt categories, are not eligible for collection. OIG suggests that the calculation's numerator include debt in a collection arrangement, the third principal debt category, and the denominator include both debt in a collection arrangement and debt not in a collection arrangement, the fourth principal debt category. This will create a more meaningful measure that demonstrates the percent of all debt that was eligible to be collected that resulted in a debt collection arrangement.

Response

We disagree. We considered this option when the measures were being developed, but it was decided to include all unresolved debt in the formula to be consistent with what the Agency reports elsewhere as outstanding debt (i.e., the quarterly Treasury Report on Receivables to the Treasury Department). Additionally, the measures were developed in consultation with OMB, which is fully aware and in agreement with them.

Recommendation 7

Enhance disclosure of SSA's decision not to establish incremental improvement goals that results in a departure from OMB guidance.

Response

We disagree. SSA is in the process of developing increased performance measures for debt collection as part of the PMA initiative on "Improper Payments".

Recommendation specific only to performance indicator, "Percent of Outstanding SSI Debt in a Collection Arrangement"

Recommendation 8

Ensure COBOL programs adhere to the SSA Systems Development Life Cycle. In addition, the programs should be maintained in a version control system in the event the program must be backed out and reinstalled.

Response

We agree. We will ensure that appropriate controls are established to make certain that no data is lost in the event the program must be backed out and reinstalled.

Overview of the Office of the Inspector General

The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Resource Management (ORM). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit

OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations

OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General

OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Resource Management

ORM supports OIG by providing information resource management and systems security. ORM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, ORM is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.