OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR
AUDIT:
DISABILITY DETERMINATION
SERVICES PROCESSING

May 2007

A-02-06-16110

AUDIT REPORT

Mission

By conducting independent and objective audits, evaluations and investigations, we inspire public confidence in the integrity and security of SSA's programs and operations and protect them against fraud, waste and abuse. We provide timely, useful and reliable information and advice to Administration officials, Congress and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

We strive for continual improvement in SSA's programs, operations and management by proactively seeking new ways to prevent and deter fraud, waste and abuse. We commit to integrity and excellence by supporting an environment that provides a valuable public service while encouraging employee development and retention and fostering diversity and innovation.

MEMORANDUM
Date: May 8, 2007

To: The Commissioner

From: Inspector General

Subject: Performance Indicator Audit: Disability Determination Services Processing (A-02-06-16110)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 15 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. Attached is the draft report presenting the results of two of the performance indicators PwC reviewed. For the performance indicators included in this audit, PwC's objectives were to:
Assess the effectiveness of internal controls and test critical controls over data generation, calculation, and reporting processes for the specific performance indicator.
Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.
Test the accuracy of results presented and disclosed in the Fiscal Year 2006 Performance and Accountability Report.
Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

This report contains the results of the audit for the following indicators:
Agency Decisional Accuracy Rate
Disability Determination Services Cases Processed per Workyear

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.

Patrick P. O'Carroll, Jr.
Attachment

MEMORANDUM

Date: May 3, 2007

To: Inspector General

From: PricewaterhouseCoopers, LLP

Subject: Performance Indicator Audit: Disability Determination Services Processing (A-02-06-16110)

OBJECTIVE

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

Our audit was conducted in accordance with generally accepted government auditing standards for performance audits. For the performance indicators included in this audit, our objectives were to:

1. Assess the effectiveness of internal controls and test critical controls over the data generation, calculation, and reporting processes for the specific performance indicator.

2. Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.

3. Test the accuracy of results presented and disclosed in the Fiscal Year
(FY) 2006 Performance and Accountability Report (PAR).

4. Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

BACKGROUND

We audited the following performance indicators as stated in the SSA FY 2006 PAR:

Performance Indicator FY 2006 Goal FY 2006 Reported Results
Agency Decisional Accuracy (ADA) Rate 97% 97%*
Disability Determination Services (DDS) Cases Processed per Workyear (PPWY) 262 241
*The performance data shown for FY 2006 is based on performance through June 2006. Actual end-of-year data will be reported in the FY 2007 PAR.

SSA administers the Old-Age and Survivors Insurance (OASI), Disability Insurance (DI) and Supplemental Security Income (SSI) programs. The OASI program, authorized by Title II of the Social Security Act, provides income for eligible workers and for eligible members of their families and survivors. The DI program, also authorized by Title II of the Social Security Act, provides income for eligible workers who have qualifying disabilities and for eligible members of their families before those workers reach retirement age. The SSI Program, authorized by Title XVI of the Social Security Act, was designed as a needs-based program to provide or supplement the income of aged, blind, and/or disabled individuals with limited income and resources.

To determine eligibility for both Title II and Title XVI programs, applicants must first file a claim with SSA. This is typically accomplished through an appointment in 1 of SSA's approximately 1,300 field offices (FO), through the SSA telephone network, or online via the Internet Social Security Benefit Application. Interviews with the applicants are conducted by FO personnel via the telephone or in person to determine the applicants' non-medical eligibility. If the applicant is filing for benefits based on disability, basic medical information concerning the disability, medical treatments, and identification of treating sources (e.g. a Doctor's office) is obtained.

Field office personnel input the applicant's information into the Modernized Claims System (MCS) for OASI and DI claims or the Modernized SSI Claims System (MSSICS) for SSI claims. A relatively minor number of OASI and DI claims are input through the SSA Claims Control System (SSACCS). SSACCS is used to process claims that cannot be fully processed through MCS. For example, when a Title II record is established, the MCS application allows for entry of up to 11 claimants on the specific record. Additional claimants to a single MCS record would need to be recorded on SSACCS. DI and SSI disability claims are sent to the State DDS office for review of medical information and a disability determination. The State DDS offices input case determinations into the National Disability Determination Services System (NDDSS).

The Office of Quality Control (OQC) routinely performs reviews of initial decisions for DI, SSI, or concurrent disability determinations made by DDS personnel (concurrent cases are cases in which there is a claim for both DI and SSI benefits simultaneously). Cases that meet sampling criteria are selected within NDDSS and sent to Disability Quality Branches (DQB) for review by medical consultants and disability examiners. The reviews are both medical and non-medical and replicate the DDS process to determine whether the correct disability determinations were made by DDS personnel. DQB review results are input into the Disability Case Adjudication and Review System (DICARS) which is then interfaced to the Management Information Services Facility (MISF) for reporting purposes.

RESULTS OF REVIEW

Our assessment of the two indicators included in this report did not identify any significant exceptions related to the meaningfulness of these indicators, the accuracy of presentation, or disclosure of information related to these indicators in the FY 2006 PAR.

Our assessment did identify issues with internal controls and data reliability. For the indicator "Agency Decisional Accuracy Rate," we noted that SSA programmer personnel in the Office of Quality Performance had direct data access that would allow them to update production performance indicator data. In addition, the programming logic used to create one of the system generated reports (used to calculate the agency decisional accuracy rate) was created and quality reviewed by the same programmer. The programming logic should have been independently reviewed to ensure it was developed to accurately meet the intent of the functional requirements. Due to these internal controls weaknesses, we found the data to be unreliable.

For the indicator "DDS Cases Processed per Workyear (PPWY)," we noted that an audit trail for transactions processed through the SSACCS application did not exist. In addition, management had not formally documented procedures to review and confirm the indicator calculation and results. Despite these internal control weaknesses, we were able to determine that the data used to calculate this performance indicator was reliable.

Indicator Backgrounds

Agency Decisional Accuracy rate

When DDS personnel complete their review of a case, the disability determination is input into NDDSS. On a daily basis, the Automated Sample Selection Process (ASSP) module of NDDSS automatically selects a sample of initial DI, SSI, or concurrent cases for review by DQB personnel. The cases selected by the ASSP are first screened for inclusion in the Quality Assurance (QA) review and then screened for inclusion in the Pre-effectuation Review (PER).

The QA review includes a review of initial determinations of Title II, Title XVI, and concurrent Title II/Title XVI cases. The PER includes a review of initial determinations of favorable determinations of Title II or concurrent Title II/Title XVI cases. By law, PERs must be performed on at least 50 percent of the favorable Title II or concurrent Title II/Title XVI disability determinations to identify deficiencies prior to effectuation. If a case meets the criteria to be selected for both a QA and PER review, it is reviewed once, but double-counted as both a QA and PER review. This process occurs on an ongoing basis as determinations are cleared by DDS personnel.

After sample selection, the case information is compiled into one file (the file contains both PER and QA samples) which is placed on the MISF. On a nightly basis, a series of mainframe batch jobs separate the file into 10 files that are segmented by each region. Each morning, users at the DQBs select the file on the MISF for their region and upload the file into DICARS. The DQB personnel perform a medical and
non-medical review of the case to determine if the disability determination on the case has been made correctly. When deficiencies are identified during the review, DQB personnel return the case to the originating DDS for correction. If the DDS disagrees with the correction to be made, it can rebut the decision. The DDS will notify the DQB of the rebuttal and if a decision is not agreed upon, the rebuttal will be documented and sent to Headquarters to decide on an outcome. If corrections are required, DDS personnel will correct the identified deficiencies and return the case to the DQB for completion of their review.

Upon completion of their review, DQB personnel input the results for each case into DICARS. Decisional errors that are corrected during the course of the DQB review are subtracted from the overall number of estimated errors. Each day, DQB personnel transfer a file containing all completed review results from DICARS to the MISF. On a nightly basis, a series of mainframe batch jobs compile the results of all regional DQB reviews from the previous day into a single file. In addition, a report is automatically printed at the DQB on a nightly basis that lists the QA and PER cases that were successfully transmitted to the MISF. If the report indicates that there were cases not transferred to the MISF due to errors, DQB personnel correct the specific cases so they will be included in the ADA rate result.

The DQB review is used to identify the percentage of correct determinations made by DDS personnel for cases meeting QA and PER criteria. On a quarterly basis, the "ADA Accuracy - National and Regional Rates" report is generated from the results file on the MISF. The report contains the fiscal Year-to-Date (YTD) accuracy rate up to the prior quarter end. Results are reported on a 3 month rolling basis to allow time for corrective actions to take place on cases reviewed. For example, results reported in October will be the accuracy rate for the months of April through June.

Performance Indicator Calculation

Agency Decisional Accuracy Rate
=
(1 - (Total Estimated Errors - Total Changed Decisions) / Total Disability Determinations) * 100

Findings

Internal Controls and Data Reliability

We identified two internal control issues. First, six members of SSA's OQC's programming personnel had the "All" access designation within the Top Secret security software to MISF datasets used to calculate the indicator results. This level of access allows users to create, delete and modify any of the data (or datasets) contained within the datasets we reviewed. Therefore, the data used to calculate the performance indicator could be inappropriately modified and could impact the results of this performance indicator. This level of access prevents SSA from ensuring the integrity of this production data. By allowing programming personnel to have the "All" access designation, SSA is not conforming to Office of Management and Budget Circular
A-130, Management of Federal Information Resources, Appendix III, Security of Federal Automated Information Resources, principles of "least privileged access" or segregation of duties.

Second, OQC management's quality review of the quarterly "ADA Accuracy - National and Regional Rates" programming logic (code) was performed in part by one of the programmers that developed the code used to create the report. This programmer is also one of the six individuals noted above who has the "All" access Top Secret designation to the MISF data used to generate the report. The quality review of the code and data access issue creates a segregation of duties conflict, as an independent review and verification of the reported initial results was not completed.

Additionally, we found that the MISF data used to record QA and PER results was not archived and maintained. SSA management stated that the detailed data was not maintained due to limited data storage space and lack of personnel resources. Therefore, we performed alternative testing procedures to assess the reliability of the indicator data presented in the PAR.
As a result of these tests, we are reasonably comfortable that the data reported in the PAR for this indicator are complete, accurate, and consistent. However, the data cannot be considered reliable since the two access control issues noted above created the potential for inappropriate alteration.

We did not identify any significant exceptions related to the accuracy of the presentation and disclosure of the information related to this indicator contained in the PAR, or to the meaningfulness of this indicator. However, due to the access and independent review control issues raised previously, we could not consider the data used to calculate the results of the indicator to be reliable.

Disability Determination Services (DDS) Cases Processed per Workyear (PPWY)

When a claim determination is made by DDS personnel, the status is entered into NDDSS and the case is closed. The data within NDDSS is automatically transferred to the Disability Operational Datastore (DIODS). Workyear information is input into DIODS by management at each DDS on a weekly basis. DDS managers are required to input information related to both direct and indirect time expended for all work in DIODS. The average number of cases processed by an individual DDS employee is captured on the DDS Staffing and Workloads Analysis (FD-15) Report, generated from DIODS.

The Office of Disability Determinations prints the FD-15 Report on a weekly basis and reviews the report for errors or anomalies. This report displays cases processed per workyear for the current week, rolling 4 weeks, quarter ending, cumulative, and adjusted cumulative cases processed. The cumulative YTD calculation includes all contractor hours related to processing of cases, and is directly used for reporting indicator results within the PAR. The adjusted YTD calculation includes all clerical and non-clerical contractual hours.

Performance Indicator Calculation

Disability Determination Services (DDS) Cases Processed per Workyear (PPWY)
=
Total Clearances
Annual Workyear Rate

Annual Workyear Rate is defined as direct and indirect time, including overhead time (time spent on training, travel, leave, holidays, et cetera).

Findings

Internal Controls and Data Reliability

We identified two internal control issues. First, an audit trail for transactions processed through SSACCS was not created or reviewed. Accordingly, management is not able to review and identify inappropriate or unauthorized transactions being processed through SSACCS.

Second, we found that there are no formally documented procedures outlining the process for managers to consistently review the FD-15 reports in a timely manner. This increases the risk that standard and timely procedures will not be followed when reviewing the reports. Lack of a standard and timely management review could result in inaccurate reporting of indicator results.

Additionally, we also found that the DIODS data used to classify the disability claims as clearances was not archived and maintained. SSA management stated that the detailed data was not maintained due to limited data storage space and lack of personnel resources. Therefore, we performed alternative testing procedures to assess the reliability of the indicator data presented in the PAR.

Specifically, we performed a detailed review of the code used to generate the indicator results (included on the FD-15 report). We concluded that the code is designed to calculate the indicator results as described by SSA management. In addition, we selected numerous cases from DIODS, and compared the case information to the corresponding records in the Supplemental Security Record and the Master Beneficiary Record. Also, we were able to observe the final calculation of this indicator on a real-time basis. We compared the final reported results of this indicator as reported in the PAR with the final data recorded on the FD-15 report (which includes final indicator results). Our testing resulted in no exceptions with the code, the data in DIODS, or the results recorded on the PAR.

We did not identify any significant exceptions related to the meaningfulness of this indicator or the accuracy of the presentation and disclosure of the information related to this indicator contained in the PAR. Also, based on the additional control testing we completed related to the DIODS data, as well as our review of the code that was used to calculate the indicator results, we were able to conclude that the data used in the calculation of the indicator results was reliable at the time of calculation.

RECOMMENDATIONS

Specific to the performance indicator, "Agency Decisional Accuracy Rate" we recommend SSA:

1. Ensure that the "least privileged access" principle is in place for SSA personnel that have the ability to directly modify, create or delete the datasets used to calculate the results of this indicator.

Specific to the performance indicator, "Disability Determination Services Cases Processed per Workyear" we recommend SSA:

2. Implement formal procedures to ensure that reviews of the FD-15 reports are performed in a timely manner.

3. Maintain an audit trail for SSACCS that captures the user identification, terminal, date and time the transaction was processed. Policies and procedures should be implemented requiring a review of the audit trail for inappropriate access or processing of transactions. In the event that the SSACCS application is replaced, SSA management should ensure that the replacement system is designed with the appropriate audit trail controls.

AGENCY COMMENTS

SSA agreed with the first two recommendations and disagreed with the third. In disagreeing with the third, SSA stated that SSACCS will be phased out; therefore, it is cost-prohibitive to maintain an audit trail for this system's transactions. However, SSA agreed that the replacement system should be designed with appropriate audit trail controls. The Agency's comments are included in Appendix D.

PwC RESPONSE

We appreciate the Agency's comments and consideration of our recommendations. While SSA management noted disagreement with our third recommendation, we believe management's narrative response actually shows agreement with it. We concur that SSA management should not change the current SSACCS system, if it intends to replace SSACCS with a new system in the near future. SSA stated in its response that it will build the appropriate audit trail into the new system, which was what we recommended.

Appendices

APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Agency Comments

Appendix A
Acronyms

ADA Agency Decisional Accuracy
ASSP Automated Sample Selection Process
DDS Disability Determination Services
DI Disability Insurance
DICARS Disability Case Adjudication and Review System
DIODS Disability Operational Datastore
DQB Disability Quality Branch
FD-15 DDS Staffing and Workload Analysis
FO Field Office
FY Fiscal Year
GPRA Government Performance and Results Act
MBR Master Beneficiary Record
MCS Modernized Claims System
MISF Management Information Services Facility
NDDSS National Disability Determination Services System
OASI Old-Age and Survivors Insurance
ODD Office of Disability Determinations
OQC Office of Quality Control
OS Office of Systems
PAR Performance and Accountability Report
PER Pre-effectuation Review
PPWY Processed per Work Year
QA Quality Assurance
SAOR State Agency Operations Report
SSA Social Security Administration
SSACCS Social Security Administration Claims Control System
SSI Supplemental Security Income
SSR Supplemental Security Record
U.S.C. United States Code
YTD Year-to-Date

Appendix B
Scope and Methodology
We updated our understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured, as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing, including testing of source documentation, we performed the following:

Reviewed prior SSA, Government Accountability Office, Office of the Inspector General and other reports related to SSA's GPRA performance and related information systems.
Reviewed applicable laws, regulations and SSA policy.
Met with the appropriate SSA personnel to confirm our understanding of the performance indicators.
Flowcharted the processes. (See Appendix C).
Tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).
Conducted and evaluated tests of the manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicators.
Identified attributes, rules, and assumptions for each defined data element or source document.
Recalculated the metrics of key performance indicators to ensure mathematical accuracy.
For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these items interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators appear to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes. We followed all performance audit standards in accordance with generally accepted government auditing standards. In addition to these steps, we specifically performed the following to test the indicators included in this report:

AGENCY DECISIONAL ACCURACY RATE

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following areas:
Completed application control reviews over the National Disability Determination Services System (NDDSS) and the Disability Case Adjudication and Review System (DICARS). An application control review includes testing access controls, data input, data output, data rejection, and data processing, as applicable.
Completed a review of the DICARS/ Management Information Services Facility (MISF) interface process.
Used a benefit specialist to review a sample of cases completed by the Disability Determination Services (DDS) centers to determine the completeness and accuracy of the disability determinations made.
Used a statistician to determine the adequacy of the sampling methodology used to select cases within NDDSS for Disability Quality Branch (DQB) review.
Selected a sample of cases and traced/agreed common data elements from the DICARS flat file to the claims folder to ensure that DQB properly entered/documented the proper data fields into DICARS.
Observed a sample of cases at three DDS locations to determine whether cases were locked from changes or effectuation after being selected for DQB review.
Determined the adequacy of the programming logic used by SSA to calculate the Agency Decisional Accuracy (ADA) Rate from the MISF.
Recalculated the MISF data for April 2006 and Fiscal Year 2006 and compared it to the ADA Rate in the "ADA Accuracy - National and Regional Rates" report.
Traced data from supporting reports to the indicator calculation total included in the Performance and Accountability Report (PAR).

DDS CASES PROCESSED PER WORKYEAR (PPWY)

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following area:
Completed application control reviews over the NDDSS and the Disability Operational Datastore (DIODS). An application control review includes testing access controls, data input, data output, data rejection, and data processing, as applicable.
Performed a comparison of the summary data from the State Agency Operations Report (SAOR) report and the Master Beneficiary Record (MBR) and Supplemental Security Record (SSR) to ensure accuracy and completion of the transfer of files from the MBR and SSR, through NDDSS and into DIODS.
Performed a walkthrough of the workyear input into DIODS at three separate DDS locations.
Inquired about the process to review the weekly DDS Staffing and Workload Analysis (FD-15) report and correct any errors/anomalies discovered.
Determined the adequacy of the programming logic used by SSA to calculate the FD-15 report from DIODS.
Traced data from supporting reports to the indicator calculation total included in the PAR.

Appendix C
Flowchart of Agency Decisional Accuracy Rate

Agency Decisional Accuracy Rate
Disability Determination Services (DDS) receives application for Disability Insurance (DI) or Supplemental Security Income (SSDI) benefits.
DDS inputs receipt of case in National Disability Determination Services System (NDDSS).
NDDSS receives claimant information from Modernized Claims System (MCS), Modernized Supplemental Security Income System (MSSICS), or Social Security Administration Claims Control System (SSACCS).
DDS gathers and reviews medical evidence in order to make a medical determination.
Claim is approved or denied; medical portion of the decision is adjudicated.
Case is closed in NDDSS.
Automated Sample Selection Process (ASSP) module applies sampling criteria to determinations in NDDSS to select sample population.
Determination is at sampling interval for Quality Assurance (QA) review?
Yes
No - Determination is favorable Title II or concurrent Title II/Title XVI for Pre-effectuation review (PER)?
Yes - Determination scores are above cutoff score?
Yes
No - Determination is not selected for PER review.
End
No - Determination is not selected for QA or PER review.
End
DDS notified electronically and all selected cases are locked to ensure that changes or effectuation do not take place.
DDS case folders sent to respective regional DQB.
A
Claim is reviewed by Disability Quality Branch (DQB) medical consultants and disability examiners by replicating the DDS process as closely as possible.
Review identifies substantive deficiency?
Yes - Reviewer corrects basic decisional errors and returns cases to DDS with directions for required corrective actions.
DDS agrees with deficiency?
Yes - DDS performs corrective action and returns case to DQB for verification.
DQB completes verification.
No - DDS rebuts finding
Deficiency upheld?
Yes - DDS performs corrective action and returns case to DQB for verification.
No - DDS and DQB notified and DQB rescinds deficiency.
No - B
B
Reviewer inputs review/results data into Disability Case Adjudication and Review System (DICARS) using the Federal Review Results form (screens in DICARS).
DICARS interfaces with the Management Information Services Facility (MISF) to pass QA data.
Data is compiled in Social Security Administration (SSA) Headquarters (HQ) and is the basis of Agency Decisional Accuracy (ADA) reports.
Corrected errors removed from total estimated errors.
ADA rate is reported in the PAR.

Flowchart of Disability Determination Services (DDS) Cases Processed Per Workyear (PPWY)

Disability Determination Services (DDS) Cases Processed Per Workyear (PPWY)
DDS inputs receipt of case in National Disability Determination Services System (NDDSS).
NDDSS receives claimant information from Modernized Claims System (MCS), Modernized Supplemental Security Income System (MSSICS), or Social Security Administration Claims Control System (SSACCS).
DDS gathers and reviews medical evidence to make a medical determination.
If medical information is not sufficient, a consultative examination is scheduled.
DDS inputs medical decision as reported on Form SSA-831.
Claim is approved or denied; medical portion of the decision is adjudicated.
Case is closed in NDDSS.
NDDSS interfaces with Disability Operational Datastore (DIODS) to provide processed claims data.
Personnel hours and staffing information (workyear) are input into DIODS.
DIODS calculates the cumulative claims processed per workyear for each state, region, and the Nation on a weekly basis for reporting to the DDS Staffing and Workload Analysis (FD-15) Report.
The Office of Disability Determinations (ODD) reviews the weekly FD-15 report to identify anomalies.
ODD notifies the Office of Systems (OS) of the anomalies/errors that need to be corrected.
OS corrects errors and the report is regenerated for ODD review.
Month end cumulative PPWY data from FD-15 report is used to update the Commissioners Tracking Report.
Is the goal being met?
Yes - Year-end PPWY number from FD-15 report is recorded in the Performance and Accountability Report (PAR).
No - Provide feedback to the Office of the Chief Strategic Officer.
Year-end PPWY number from FD-15 report is recorded in the PAR.

Appendix D
Agency Comments

SOCIAL SECURITY

MEMORANDUM

Date: April 18, 2007

To: Patrick P. O'Carroll, Jr.
Inspector General

From: Larry W. Dye /s/

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: Disability Determination Services Processing" (A-02-06-16110)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Please let me know if we can be of further assistance. Staff inquiries may be directed to Ms. Candace Skurnik, Director, Audit Management and Liaison Staff, at 410 965-4636.

Attachment:
SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL (OIG) DRAFT REPORT, "PERFORMANCE INDICATOR AUDIT: DISABILITY DETERMINATION SERVICES PROCESSING" (A-02-06-16110)

Thank you for the opportunity to review and comment on the draft report. We generally agree with the report's findings and the intent of the recommendations. Our responses to the specific recommendations are provided below. We are also providing technical comments to enhance the accuracy of the report.

Recommendation 1

The Social Security Administration (SSA) should ensure that the "least privileged access" principle is in place for SSA personnel that have the ability to directly modify, create or delete the datasets used to calculate the results of this indicator.

Response

We agree with this recommendation and are taking the following actions: We will delete one of the six Office of Quality Control (OQC) staff members who has the All" access designation within the Top Secret security software to the Management Information System Facility (MISF) datasets used to calculate the indicator results. All changes to data will be made through established screens and a report will be printed identifying all changes. Each report will be stored by management. As noted in our technical comments, a management review is done prior to the release of each report. We will move to formalize this review. Each report packet will be appended to indicate that a management reasonableness test was conducted to ensure that the numbers reported are correct. Documentation of this review will be annotated on the report cover sheet. Each time that we run the monthly, quarterly and fiscal year tables, we will make a copy of all the data on the MISF, and store this copy in the tape library at the National Computer Center. The retention period will be two years. In addition, we will create a text file and copy the contents onto a compact disk to be retained in the Division.

Recommendation 2

SSA should implement formal procedures to ensure that reviews of the FD-15 reports are performed in a timely manner.

Response

We agree. Formal procedures have been in place for many years to ensure that reviews of the FD-15 reports are performed in a timely manner. However, they were not previously documented. During the course of the audit, the Office of Disability Determinations established the documentation and it was provided to PricewaterhouseCoopers prior to the release of this draft. We believe that this should be noted in the report.

Recommendation 3

SSA should maintain an audit trail for the Social Security Administration Claim Control System (SSACCS) that captures the user identification, terminal, date and time the transaction was processed. Policies and procedures should be implemented requiring a review of the audit trail for inappropriate access or processing of transactions. In the event that the SSACCS application is replaced, SSA management should ensure that the replacement system is designed with the appropriate audit trail controls.

Response

We disagree. SSACCS will be phased out; therefore, it is cost-prohibitive to maintain an audit trail for this system's transactions. The Office of Management and Budget's Circular A-11, section 230.2e states "Performance data need not be perfect to be reliable, particularly if the cost and effort to secure the best performance data will exceed the value of any data so obtained." This directive applies in this situation. However, we agree that the replacement system should be designed with appropriate audit trail controls.

Technical Comments

Management also provided 2 technical comments that were addressed in the report.

Overview of the Office of the Inspector General
The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Resource Management (ORM). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit
OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations
OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General
OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Resource Management
ORM supports OIG by providing information resource management and systems security. ORM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, ORM is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.