To: The Commissioner

From: Inspector General

Subject: Performance Indicator Audit: Hearings and Appeals (A-15-07-17132)

OBJECTIVE

We contracted with PricewaterhouseCoopers, LLP, (PwC) to evaluate 13 of the Social Security Administration's (SSA) performance indicators established to comply with the Government Performance and Results Act. Attached is the final report presenting the results of one of the performance indicators PwC reviewed. For the performance indicator included in this audit, PwC's objectives were to:

Assess the effectiveness of internal controls and test critical controls over data generation, calculation, and reporting processes for the specific performance indicator.

Assess the overall reliability of the performance indicator's computer-processed data. Data are reliable when they are complete, accurate, consistent and not subject to inappropriate alteration.

Test the accuracy of results presented and disclosed in SSA's Fiscal Year 2007 Performance and Accountability Report.

Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

This report contains the results of the audit for the following indicator:

SSA hearings case production per workyear (PPWY).

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.

Patrick P. O'Carroll, Jr.

OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
HEARINGS AND APPEALS

April 2008

A-15-07-17132

AUDIT REPORT

Mission

By conducting independent and objective audits, evaluations and investigations, we inspire public confidence in the integrity and security of SSA's programs and operations and protect them against fraud, waste and abuse. We provide timely, useful and reliable information and advice to Administration officials, Congress and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

We strive for continual improvement in SSA's programs, operations and management by proactively seeking new ways to prevent and deter fraud, waste and abuse. We commit to integrity and excellence by supporting an environment that provides a valuable public service while encouraging employee development and retention and fostering diversity and innovation.

MEMORANDUM

Date: April 1, 2008

To: Inspector General

From: PricewaterhouseCoopers, LLP

Subject: Performance Indicator Audit: Hearings and Appeals (A-15-07-17132)

OBJECTIVE

The Government Performance and Results Act of 1993 (GPRA) requires that the Social Security Administration (SSA) develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

Our audit was conducted in accordance with generally accepted government auditing standards for performance audits. For the performance indicator included in this audit, our objectives were to:

1. Assess the effectiveness of internal controls and test critical controls over the data generation, calculation, and reporting processes for the specific performance indicator.

2. Assess the overall reliability of the performance indicator's computer-processed data. Data are reliable when they are complete, accurate, consistent and not subject to inappropriate alteration.

3. Test the accuracy of results presented and disclosed in SSA's Fiscal Year (FY) 2007 Performance and Accountability Report (PAR).

4. Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

BACKGROUND

We audited the following performance indicator as stated in the SSA FY 2007 PAR.

Performance Indicator
Goal
Reported Results
SSA hearings case production per workyear (PPWY)
106
101

The strategic objective to which this indicator is linked is "Manage Agency finances and assets to link resources effectively to performance outcomes." This indicator allows SSA and the public to directly relate the performance outcome of hearings processed to employee work-years.

SSA administers the Old-Age and Survivors Insurance (OASI), Disability Insurance (DI) and Supplemental Security Income (SSI) programs. The OASI program, authorized by Title II of the Social Security Act (Act), provides benefits for eligible workers and eligible members of their families and survivors. The DI program, also authorized by Title II of the Act, provides income for eligible workers who have qualifying disabilities and eligible members of their families before those workers reach retirement age. The SSI program, authorized by Title XVI of the Act, was designed as a needs-based program to provide or supplement the income of aged, blind, and/or disabled individuals with limited income and resources.

To determine eligibility for both Title II and XVI programs, applicants must first file a claim with SSA. This is typically accomplished through an appointment or walk-in visit to 1 of SSA's approximately 1,300 field offices, through the SSA telephone network, or online via the Internet Social Security Benefit Application. Interviews with the applicants are conducted by field office personnel via the telephone or in person to determine the applicant's non-medical eligibility. If an applicant is filing for benefits based on disability, basic medical information concerning the disability, medical treatments, and identification of treating sources is obtained.

After an applicant submits a claim, it will receive an initial determination of benefits. If a claimant disagrees with the initial determination, he/she can appeal within 60 days (plus 5 days for mailing purposes). The SSA appeals program provides four levels of appeal for a claimant:

Reconsideration;
Hearing;
Appeals Council (AC) Review; and
Lawsuit in Federal District Court.

Reconsideration

A complete review of the claim is performed by someone who was not part of the initial decision process. All of the evidence initially submitted by the claimant, and any new evidence, is reevaluated during the reconsideration process. Upon receiving the reconsideration decision, the claimant may request a hearing if he/she disagrees with the decision.

Hearing

A hearing is conducted by an administrative law judge (ALJ) who is independent of both the initial determination and the reconsideration decision. The ALJ reviews all information related to the claim and makes the hearing decision. If the claimant disagrees with an ALJ's hearing decision, the claimant may request an AC review.

AC Review

The AC evaluates all requests for review but can deny a request if it believes the hearing decision was correct. If the AC grants the request for review, it will either complete the review or return it to an ALJ for further review. If the claimant disagrees with the review decision or if the AC decides not to review the case, the claimant may file a lawsuit in a Federal District Court.

Lawsuit in Federal District Court

The Federal District Court may remand the court case to SSA's Commissioner for further consideration or dismissal. If remanded to the Commissioner, the AC, acting on the Commissioner's behalf, can make a decision or remand the case to an ALJ to make a decision.

SSA strategically measures the amount of cases and appeals processed to deliver high quality, citizen-oriented services. Their objective is to make correct decisions in the disability process as early as possible. A key measurement is the determination of the number of SSA hearing cases processed.

Employees in the Division of Cost Analysis (DCA) under the Deputy Commissioner for Budget, Finance and Management and the Office of Disability Adjudication and Review (ODAR) produce the reports used to calculate the number of SSA hearings cases PPWY. ODAR's Monthly Activity Report, Case Processing Management System (CPMS), Payroll Analysis Recap Report (PARR), Time and Attendance Management Information System (TAMIS), ODAR Bi-weekly Staffing report, and the Consolidated Caseload Analysis Report are created by ODAR. The Training Report and Control Workyears are from DCA. As of FY 2005, the PARR was used instead of the Time and Attendance Report. The indicator is calculated by dividing the number of cases processed by direct workyears.

The number of cases processed is reported on the Monthly Activity Report, which is generated by CPMS. This number is the sum of all hearings cases with a disposition date recorded in CPMS and is entered into the Electronic Cost Analysis System (CAS) spreadsheet for calculation.

The number of direct workyears expended by ODAR employees is calculated by dividing the number of actual work hours worked by the work hours in a year. Refer to the following formula.

Direct Workyears = (Regular + Overtime) - (Leave + Holidays + Training + Travel)
Work Hours in a Year

The following inputs are entered into the PPWY Calculation spreadsheet to determine the amount of actual work hours.

Regular and Overtime: Time worked by ODAR employees is recorded in the Mainframe Time and Attendance System (MTAS) and automatically transferred to TAMIS at the end of each pay period. Regular and overtime hours are reported on the Time and Attendance Report generated by TAMIS.

Leave: Leave hours taken by ODAR employees are recorded in MTAS and automatically transferred to the Payroll Operational DataStore. Leave hours used in the calculation are reported on the PARR.

Holidays: The number of official SSA paid holidays is used for this input.

Training: Time spent in training is tracked at the Hearings Office and Regional Office levels, by training forms, sign-in sheets and employee reporting. The Hearings Office sends the training information to the Regional Office, who then sends this information to ODAR.

Travel: The amount of time spent traveling by ALJs is estimated using the following formula.

Travel Time = 1.1 * Total working days in Month * (Total ALJs - Chief Judges)

The number of work hours in a year is defined by SSA as 2,080 hours (40 hours in a week x 52 weeks in a year). However, for the overtime figure used in the denominator calculation, work hours in a year was calculated using data from the Payroll Analysis and Recap Report (PARR) generated from the Payroll Operational Datastore. SSA uses a separate divisor for overtime because it contributes to total time differently than regular hours. Unlike regular hours, it is considered to be completely work time, with no associated leave hours. The calculation converts the overtime to the equivalent regular time to properly compute direct workyears.

After all components of the indicator are entered, formulas in the PPWY Calculation spreadsheet calculate the direct workyears. The direct workyears are entered into an Electronic CAS spreadsheet, and the final number of SSA hearings cases PPWY is calculated. The assigned DCA employee inputs the Electronic CAS spreadsheet into the CAS application, which produces the Pre-Input Cost Analysis (PICA) report. The performance indicator result for the number of SSA hearings cases PPWY is reported on the PICA.

Performance Indicator Calculation

SSA Hearings Cases Processed per Workyear (PPWY) = Number of SSA hearings cases processed Direct Workyears

RESULTS OF REVIEW

Our assessment of the indicator included in this report did not identify any significant exceptions related to the meaningfulness of the indicator, the accuracy of presentation, or disclosure of information related to the indicator in the FY 2007 PAR. We were able to recalculate the indicator results and found them to be accurate. In a January 2006 audit on this performance measure, we noted one issue of concern related to general controls that, to date, has not been adequately addressed. We also noted weaknesses with the security configuration of two UNIX operating systems.
Data Reliability

We were able to recalculate the interim and year-end indicator results and found them to be substantially accurate. Despite the internal control weaknesses noted below, we were able to determine that the data used to calculate this performance indicator were reliable.

Systems Control Issues

The CPMS application resides on a UNIX operating system to process data. In our current review of the CPMS application, which was performed at a point in time, we reviewed the security controls over two of these UNIX systems. We identified eight security and compliance issues. This review was conducted on the SSA-developed UNIX Risk Model configuration standard, National Institute of Standards and Technology (NIST) guidelines and the Defense Information Security Agency (DISA) Security Technical Implementation Guides (STIGS). We did note that SSA management has monitoring controls in place to compare security settings on its UNIX servers to the SSA Risk Model. This is performed on a monthly basis. If a variance is noted, SSA management will correct this variance. In addition, SSA management updates the SSA Risk Models for UNIX every 6 months.

General Controls Issues

We previously identified issues related to the general controls at the ODAR office space in Falls Church, Virginia. These issues were reported in our report, Performance Indicator Audit: Hearings and Appeals Process (A-15-05-15113).

During our general controls testing for the current audit, we again found that visitors to the ODAR space were not required to sign in upon entry. In addition, there were no security guards at the entrance of the ODAR space. Management stated that security guards are in place throughout the facility; however, during the fieldwork, we did not note the presence of any guards. It should be noted that the ODAR space is located in a multi-tenant, privately owned building, and ODAR management does not have complete control over the physical security of the building. In addition, to gain access to the ODAR space, a key card was required.

We recommended in the January 2006 report that ODAR, "Ensure all visitors were required to sign in upon entry to restrict visitor access to the OHA [Office of Hearings and Appeals] buildings." SSA's management provided the following response to these findings and recommendation in the January 2006 audit report.

We agree. The ODAR Headquarters building security could be improved. ODAR is working in conjunction with the Department of Justice to provide security enhancements at the ODAR facility in Falls Church, Virginia to bring the building in compliance with Level IV federal standards.

CONCLUSION AND RECOMMENDATIONS

We reaffirm our previous recommendation related to CPMS noted in the prior audit of this indicator. We continue to recommend that SSA take action to address the recommendation. (Refer to Appendix D for the prior audit recommendation.)

AGENCY COMMENTS

The Agency agreed with our prior recommendation. The Agency's comments are included in Appendix E.

Appendices
APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Number of Social Security Administration Hearings Cases Processed per Workyear - Flowchart
APPENDIX D - Prior Audit Recommendations
APPENDIX E - Agency Comments

Appendix A
Acronyms
Act Social Security Act
AC Appeals Council
ALJ Administrative Law Judge
CAS Cost Analysis System
CIS Care Intake Specialist
CPMS Case Processing and Management System
DCA Division of Cost Analysis
DI Disability Insurance
DISA Defense Information Security Agency
FY Fiscal Year
GAO Government Accountability Office
GPRA Government Performance and Results Act of 1993
MTAS Mainframe Time and Attendance System
NIST National Institute of Systems and Technology
OASI Old-Age and Survivors Insurance
ODAR Office of Disability Adjudication and Review
ODS Operational Datastore
PAR Performance and Accountability Report
PARR Payroll Analysis Recap Report
PICA Pre-Input Cost Analysis
PPWY Processed per Workyear
RSI Retirement and Survivors Insurance
SCT Senior Case Technician
SSA Social Security Administration
SSI Supplemental Security Income
STIGS Security Technical Implementation Guides
TAMIS Time and Attendance Management Information System
U.S.C. United States Code

Appendix B
Scope and Methodology
We updated our understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes through research and questions to SSA management. We also requested that SSA provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing, including testing of source documentation, we completed the following steps.

Reviewed prior SSA, Office of the Inspector General, and other reports related to SSA's GPRA performance and related information systems.
Reviewed applicable laws, regulations and SSA policy.
Met with the appropriate SSA personnel to confirm our understanding of the performance indicator.
Flowcharted the process (see Appendix C).
Tested key controls related to manual or basic computerized processes (for example, spreadsheets and databases).
Conducted and evaluated tests of the automated and manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.
Identified attributes, rules, and assumptions for each defined data element or source document.
Recalculated the metric or algorithm of the performance indicator to ensure mathematical accuracy.
Assessed the completeness and accuracy of the data to determine the data's reliability as they pertain to the audit objectives and intended use of the data.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes in SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine whether the performance indicator appeared to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes.

In addition to these steps, we specifically performed the following to test the performance indicator included in this report.

Inspected relevant policies and procedures as necessary.
Audited the design and effectiveness of SSA internal controls and the accuracy and completeness of the data related to the following areas:
Observed the input of the Hearing Request Date, Request Received Date and the Input of Hearing Disposition in the Case Processing and Management System (CPMS). This was performed for 40 cases - the entire population available at that time.
Performed a follow-up general computer control review as it relates to the CPMS.
Performed a limited application controls review of CPMS.
Determined the adequacy of programming logic used by SSA related to CPMS Management Information.
Reviewed each component of the workyear calculation for completeness and accuracy.
Traced data from supporting reports to the spreadsheets used to calculate the performance indicator.
Compared the spreadsheet results for the performance indicator for Fiscal Year 2007 to the number reported in the Performance and Accountability Report.

We conducted our work in accordance with generally accepted government auditing standards.

Appendix C
Number of Social Security Administration Hearings Cases Processed per Workyear - Flowchart

Number of Social Security Administration Hearings Cases Processed per Workyear - Flowchart continued

Number of Social Security Administration Hearings Cases Processed per Workyear - Flowchart Continued

Number of Social Security Administration Hearings Cases Processed per Workyear - Narrative

The initial decision of a claim is received by the claimant.
The claimant requests a reconsideration of the SSA's initial decision.
If SSA does not agree with the initial decision, it will reverse its initial decision. If SSA agrees with the initial decision, the claimant will receive SSA's reconsideration decision.
The claimant can now request a hearing, either through an FO or teleservice center.
In both cases, the claimant is instructed to file Form HA-501 and send the hearing request to an FO.
The FO forwards the case file to the HO for processing if the appeal was filed within the appeals period.
The CIS or SCT enters the case into the CPMS.
The CIS determines whether the case is eligible for early screening (that is, claimants who are over 55 years of age) and will assign to an ALJ on a rotational basis.
The ALJ can either decide to conduct a hearing or not conduct a hearing. If a hearing is not conducted, the ALJ can dismiss the case or pay the claim on record without conducting a hearing.
If a hearing is conducted, the case is explained, and the ALJ issues a decision.
In most instances, the decision will be entered into CPMS by support staff.
A clerk enters disposition date and mail date into CPMS.
A decision letter and a copy of the ALJ's decision are sent to the claimant.
HO database files are sent to the RO and combined in CPMS.
Regional database files are sent electronically to ODAR and combined in CPMS.
The MAR is generated by CPMS.
The MAR is combined for all regions to generate the CAR.
The MAR is posted to the SSA Intranet for ROs to review.
The number of SSA Hearings Processed is taken from the MAR.
Control workyears is calculated by DCA and emailed to ODAR.
Regular Time, Overtime, and Leave are calculated using Mainframe Time and Attendance System (MTAS).
Travel is calculated using the Travel Formula.
Training is totaled from HO/RO training reports using Webbass.
The monthly electronic CAS file is emailed to DCA.
The ODAR Workload file is created and reconciled to the Electronic CAS by a designated ODAR budget analyst.
A macro is run to create the PRN file for upload to CAS.
A macro is run to update the CAS input sheet with data from ODAR Workload file.
The PRN file is uploaded to CAS.
PICA is updated with the current month's input.
Check PICA against CAS input sheet to ensure proper upload of information.
Reconcile the number of hearings listed in the Receipts, Processed, and Workyears fields of the CAS spreadsheet to the numbers listed in the PICA.
Spreadsheet formulas calculate performance indicator Number of SSA Hearings PPWY.
Send to Director of DCA for review.
Director of DCA sends to OSM by the 15th of the month.
The Performance Indicator is reported in the PAR.

Appendix D
Prior Audit Recommendations
During a prior audit, we provided SSA with the following recommendation for the CPMS application that we continue to recommend:

1. Ensure that the CPMS UNIX systems are configured to be in compliance with the SSA UNIX Risk Model and Government guidelines from National Institute of Standards and Technology (NIST) and Defense Information Security Agency.

Appendix E
Agency Comments

SOCIAL SECURITY

MEMORANDUM

Date: March 31, 2008 Refer To: S1J-3

To: Patrick P. O'Carroll, Jr.
Inspector General

From: David V. Foster
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: Hearings and Appeals" (A-15-07-17132)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our response to the report findings and recommendation is attached.

Please let me know if we can be of further assistance. Staff inquiries may be directed to Ms. Candace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL (OIG) DRAFT REPORT, "PERFORMANCE INDICATOR AUDIT: HEARINGS AND APPEALS" (A-15-07-17132)

Thank you for the opportunity to review and provide comments on this draft report. We appreciate your efforts in conducting this review and thank you for collaborating with us in making the technical changes suggested to the results and findings section of the draft report. We believe the suggested change will fairly represent the state of UNIX monitoring, and we look forward to seeing these changes reflected in the final report.

Our response to the recommendation is as follows:

Recommendation 1

Ensure that the Case Processing and Management System (CPMS) and UNIX systems are configured to be in compliance with the SSA UNIX Risk Model and Government guidelines from the National Institute of Standards and Technology and Defense Information Security Agency.

Comment

We agree. SSA periodically updates its risk models to conform to accepted security principles. Additionally, we frequently sweep the UNIX environment to ensure compliance with published settings. Audits seldom match the timing of these activities, and we are confident our processes are identifying non-compliant servers.

Overview of the Office of the Inspector General
The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Resource Management (ORM). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit
OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations
OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General
OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Resource Management
ORM supports OIG by providing information resource management and systems security. ORM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, ORM is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.