Date: September 3, 2004

To: The Commissioner

From: Acting Inspector General

Subject: Performance Indicator Audit: President's Management Agenda Related Initiatives (A-15-04-14070)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 16 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. The attached final report presents the results of four of the performance indicators PwC reviewed. For each performance indicator included in this audit, PwC's objectives were to:

Test critical controls over the data generation and calculation processes for the specific performance indicator,

Assess the overall adequacy, accuracy, reasonableness, completeness, and consistency of the performance indicator and supporting data, and

Determine if each performance indicator provides meaningful measurement of the program and the achievement of its stated objectives.

This report contains the results of the audit for the following indicators:

"Get to Green" on all President's Management Agenda Initiatives,
Percent of Commercial Positions Competed or Converted,
Percent Improvement in the New Hire Retention Rate, and
Provide the Equivalent of 40 Hours of Training Annually to all Employees.

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965 9700.

Patrick P. O'Carroll, Jr.

OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
PRESIDENT'S MANAGEMENT AGENDA
RELATED INITIATIVES

September 2004

A-15-04-14070

AUDIT REPORT

Mission

We improve SSA programs and operations and protect them against fraud, waste, and abuse by conducting independent and objective audits, evaluations, and investigations. We provide timely, useful, and reliable information and advice to Administration officials, the Congress, and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

By conducting independent and objective audits, investigations, and evaluations, we are agents of positive change striving for continuous improvement in the Social Security Administration's programs, operations, and management and in our own office.

MEMORANDUM

Date: August 6, 2004

To: Acting Inspector General

From: PricewaterhouseCoopers LLP

Subject: Performance Indicator Audit: President's Management Agenda Related Initiatives (A-15-04-14070)

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

OBJECTIVE

For each performance indicator included in this audit, our objectives were to:

1. Test critical controls over the data generation and calculation processes for the specific performance indicator.

2. Assess the overall adequacy, accuracy, reasonableness, completeness, and consistency of the performance indicator and supporting data.

3. Determine if each performance indicator provides meaningful measurement of the program and the achievement of its stated objectives.

We audited the following performance indicators as stated in the SSA Fiscal Year (FY) 2003 Performance and Accountability Report (PAR):

Performance Indicator FY 2003 Goal FY 2003 Reported Results
"Get to Green" on all President's Management Agenda (PMA) Initiatives Receive a "green" for progress on 100% of PMA initiative plans. Received a "green" for progress on 100% of PMA initiative plans.
Percent of Commercial Positions Competed or Converted 15% 0.4%
Percent Improvement in the New Hire Retention Rate 84.6% 86.4%
Provide the Equivalent of 40 Hours of Training Annually to All Employees 40 hours 68 hours

BACKGROUND

The President's Management Agenda

The PMA was designed to "…address the most apparent deficiencies where the opportunity to improve performance is the greatest." The PMA commits Federal agencies to achieving immediate, concrete, and measurable results in the near term.
It not only focuses on remedies to problems generally agreed to be serious, but more importantly, commits to implement them fully. The five government-wide goals are described below:

1. The "Strategic Management of Human Capital" initiative requires agencies to prepare for massive impending retirements and remedy deficiencies in the skills of their employees.

2. "Competitive Sourcing" requires that agencies make sure the most efficient sources perform their commercial activities.

3. "Improved Financial Performance" enhances the quality of financial information so that agencies can ensure the integrity and efficiency of their operations.

4. "Expanded Electronic Government" improves the management of information technology and streamlines and simplifies the delivery of Government services through the use of the Internet.

5. "Budget and Performance Integration" enhances the quality of information on program results so that Government can make better decisions about its allocation of resources.

RESULTS OF REVIEW

"Get to Green" on All PMA Initiatives

FY 2003 Goal: 100 percent of initiative plans receive "green" for progress.
Actual FY 2003 Performance: Received a "green" for progress on 100 percent of PMA initiative plans.
SSA met the goal. The FY 2003 ratings for progress were "green" on the five PMA plans.
Indicator Background
MA Plan

To ensure accountability for performance and results, the President's Administration is using an Executive Branch Management Scorecard to track how well agencies are executing the management initiatives (i.e. status), and their progress at a given point in time (i.e. progress) against overall standards for success.

The Office of Management and Budget (OMB) is tasked with evaluating agencies' performance (status and progress) in "Getting to Green" on the five PMA initiatives.

At the end of each fiscal quarter, agencies are required to report to OMB on their accomplishments for that quarter and the planned actions for the next quarter. OMB reviews the quarterly PMA update reports and rates the agencies on their progress and status using the scorecard. SSA receives the scorecard from OMB approximately 2 weeks after the close of each quarter.

Status: Scores are based on five standards for success defined by the President's Management Council. OMB assigns status scores as follows:

Green - The Agency met all of the standards for success.
Yellow - The Agency achieved some but not all of the criteria.
Red - The Agency had any number of serious flaws.

Progress: OMB assesses progress on a case-by-case basis against planned deliverables established for the five initiatives. OMB assigns progress scores as follows:

Green - Implementation is proceeding according to plan.
Yellow - Some slippage or other issues requiring adjustment by the Agency to achieve the initiative objectives on a timely basis.
Red - Initiative is in serious jeopardy; unlikely to realize objectives absent significant management intervention.

Findings

We found the data SSA provided to OMB was accurate and complete. However, we found that SSA had not documented policies and procedures related to the formal process to collect, review and provide PMA information to OMB.

The PMA initiatives represent high-level strategic directions that cover three out of the four SSA Strategic Goals:
" Service
" Stewardship
" Staff

Furthermore, the indicator is an indirect measure of efforts to achieve the results linked to SSA's strategic objective "Efficiently manage Agency finances and assets, and effectively link resources to performance outcomes" rather than specific results or outcomes.

Percent of Commercial Positions Competed or Converted

FY 2003 Goal: 15 percent.
Actual FY 2003 Performance: 0.4 percent.
SSA did not meet its goal. To reach the 15 percent goal, SSA must compete all the functions identified in the Agency's competitive sourcing plan for 2002/2003.

Indicator Background

The policy of the Federal Government is to rely on the private sector for needed commercial services. To ensure that the American people receive maximum value for their tax dollars, commercial activities should be subject to the forces of competition. The agencies shall:

1. Identify all activities performed by Government personnel as either commercial or inherently governmental.

2. Use a streamlined or standard competition to determine if Government personnel should perform a commercial activity.

This indicator was calculated as follows:
Percent of Commercial
Positions Competed or Converted =
Number of Federal Government positions competed or converted to commercial positions in FYs 2001, 2002, and 2003.
Number of Federal positions eligible for competition or conversion from the FY 2000 Federal Activities Inventory Reform (FAIR) Act Inventory.

This performance indicator represents the percentage of SSA's commercial activities competed with commercial sources, or directly converted to contract, by the end of each FY. SSA calculated this performance indicator in accordance with OMB guidelines.

Findings

SSA relies on OMB A-76 guidance as documentation for the Competitive Sourcing processes. As a result, SSA has not formally documented its procedures related to the development of their Competitive Sourcing Plan (CSP). The process for developing the CSP involves collecting, validating, and then reporting information from numerous sources within the SSA.

SSA did not clearly articulate in the FY 2003 PAR that the numerator for this performance indicator's calculation is the composite number of Federal Government positions competed or converted to commercial positions in FYs 2001, 2002, and
2003. As the FY 2003 PAR is currently documented, the reader would infer that the performance indicator numerator is the Federal Government positions competed or converted in FY 2003 only.

The FY 2003 PAR did not articulate the benefits (for example, cost savings) of competing or converting commercial positions and how this supported SSA's strategic goals and objectives. It did however describe the programs in place for achieving the goal including specific action plans for FY 2004.

Percent Improvement in the New Hire Retention Rate

FY 2003 Goal: 84.6 percent.
Actual FY 2003 Performance: 86.4 percent.
SSA met its goal. SSA surpassed this goal by continuing to improve the Agency's processes for interviewing job applicants and by increasing the focus on early development of new hires, including mentoring.

Indicator Background

The baseline figure of 84.3 percent for the new hire retention rate was obtained from the SSA's New Hire Attrition and Retention Report, which identified an average 2-year attrition rate of 15.7 percent. This average rate was based on attrition of SSA new hires that were employed during FYs 1998, 1999, and 2000. The 2-year attrition period was defined as the FY of hire plus the subsequent FY. Data for this indicator is gathered from the Human Resources Management Information System (HRMIS), which is SSA's internal human resources application. This indicator was calculated as follows:

Percent improvement in retention rate = The number of total new hires in FY 2001 less the new hires who left the Agency during FY 2001 and FY 2002

The number of total new hires in FY 2001

In the New Hire Attrition and Retention Report, new hires are defined as persons hired into permanent positions. Temporary hires were not included in the calculation because, by definition of being temporary, they are not expected to remain with the Agency. Conversions from temporary to permanent positions were also not included as these persons are not "new" hires to the Agency, and are only changing their employment status.

Findings

Best practices show that organizations (both public and private) use employee retention as a key performance indicator for evaluating overall performance. This indicator is a quantitative indicator of the 2-year period retention rate of new hires. It adequately supports one of SSA's objectives related to staff, specifically the objective of "Recruit, develop, and retain a high performing workforce." It is a direct outcome performance indicator that assesses the level of success in SSA's effort to retain staff.

However, we found that the description and calculation of this performance indicator was incomplete. By design, SSA excluded some of the full time equivalent positions from the final indicator calculation. The impact of including these positions within the calculation would have an immaterial effect on the final calculation; however, these exclusions were not disclosed in the PAR.

Other refinements could be made to this performance indicator. For example, the indicator could be more specific. "Percent improvement in the new hire retention rate" did not clearly specify a percentage rate increase between the measurement periods. The current indicator is only stating the retention rate at a point in time.

Our recalculation found a potential overstatement of the number of terminations. SSA included 474 terminated employees in its calculation as reported in the FY 2003 PAR. PwC was only able to review supporting documentation for a total of 471 terminated employees. This is a difference of three employees or one tenth of 1 percent. Although this difference is immaterial to the final calculation of the indicator, SSA was unsuccessful in locating the detailed list of individual employees included in the reported results to reconcile and explain the data difference. The current process provides summary level reports that do not provide specific employee information.

In addition, PwC was unable to review the computer program for the standardized reports used to calculate the indicator results. SSA recreated the computer program for our testing. We were unable to reconcile the results of the new program with the results contained in the FY 2003 PAR.

Finally, items in the HRMIS suspense file were not being cleared in a timely manner. During our review of the interface process, we found 33 percent of items (19 out of 58) listed on the suspense file were not resolved within a week's time period. This delay could result in inaccurate reporting for this indicator.

Provide the Equivalent of 40 Hours of Training Annually to All Employees

FY 2003 Goal: Provide the equivalent of 40 hours of training per employee.
Actual FY 2003 Performance: Provided an average of 68 hours of training per employee.
SSA met its goal. SSA provides employees with career enhancement and individual growth opportunities through its OnLine University (OLU).

Indicator Background

The Office of Human Resources had recently implemented an OLU. The OLU is an Internet site that employees can access from any computer that has Internet connectivity. The purpose of the OLU is to enhance the ongoing training and development opportunities for all employees by improving access to quality training resources. Courses are self-paced and completed entirely online.

All SSA employees are eligible to take OLU courses; employees may take up to four courses in a 6-month period (eight courses a year) with 90 days to complete each course. For the purpose of this performance indicator, each course equals 4 hours of training for a total of 32 hours per employee, per year.

In addition to hours of training made available to employees through the OLU, training sessions are broadcast periodically through SSA's Interactive Video Teletraining (IVT). In FY 2003, seven 1-hour Automation Skills courses were broadcast via IVT. For the purpose of this performance indicator, employees are allowed to take two of these courses a month for a total of 24 hours per employee, per year.

Seven general topic IVT courses were broadcast in FY 2003. For the purpose of this performance indicator, employees were allowed to take one course a month for a total of 12 hours per employee, per year.

Findings

This indicator measures the number of training hours offered to employees and not the number of training hours actually received by employees. In addition, the indicator does not measure the adequacy or reasonableness of the training received.

The indicator includes elective training that might not be related to employees' current or future job responsibilities. The hours reported did not include all hours of training made available to SSA employees. The number of hours only included general interest courses offered through the OLU and a selection of automation IVT courses. It did not include job-specific training, new hire training, training required for certification, or any other job enhancement courses offered throughout the year.

The rationale for how this indicator directly supports the Agency's strategic goal, "Recruit, develop and retain a high-performing workforce," was not clear. Information reported in the PAR did not:

Disclose SSA's efforts to encourage employees to take training or SSA's efforts to monitor and improve the quality of training.

Disclose the meaning of 'provided,' as the Agency is only tracking training made available to employees, not the actual number of training hours each employee received during the year.

Indicate that this is elective training only, not job-specific training.

Articulate the definition of "equivalent hours."

The usefulness of this indicator could be improved through interim reporting to help SSA determine whether to increase or decrease efforts expended in employee development.

The number of hours reported in the PAR included an overstatement of the number of training hours offered through IVT. During our review of the general topic and automation skills course content, we found that the material was redundant. Only seven automation skills courses and seven general courses qualified for inclusion in this indicator. The course curriculum for those courses was not modified in any way for additional broadcasts. While employees were offered the courses multiple times, taking the same material multiple times would not enrich the employee. Accordingly, we believe that these offerings represent 14 hours rather than 36 hours of training.

PwC was informed that the IVT course announcements were made available to all employees through a schedule published on SSA's Intranet. However this schedule is updated monthly and a history of IVT courses offered is not maintained.

Finally, documentation describing the method used to compile and calculate the number of training hours made available to SSA employees was not provided. We found that SSA had not documented policies and procedures related to a formal process for collection of data or the calculation of the number of training hours made available to SSA employees.

RECOMMENDATIONS

We recommend SSA:

1. Enhance the discussion of the linkage of the performance indicators to the Agency's strategic goals and objectives.

2. Ensure that the performance indicator titles, definitions, and goals are explicit, complete, and consistent.

3. Improve documentation by maintaining descriptions of how the performance indicator goals were established, documenting the policies and procedures used to prepare and disclose the results of the performance indicators, and maintaining a complete audit trail for the results of the performance indicators.

4. Completely and accurately calculate the actual results of the performance indicators in accordance with the original goal.

Specific to the performance indicator, "Provide the Equivalent of 40 Hours of Training Annually to All Employees," we recommend SSA:

5. Revise the indicator to measure the amount of job related training actually received by employees, and the effectiveness of the training in improving the employees' job related capabilities.

AGENCY COMMENTS

SSA agreed with the recommendations in this report. The text of SSA's comments can be found in Appendix D.

PwC RESPONSE

SSA agreed with the recommendations noted in this report. However, it disagreed with the findings related to documenting policies and procedures for both the "Get to Green" and "Percent of Commercial Positions Competed or Converted" indicators. The Agency stated that SSA follows the OMB guidelines for reporting the information related to
"Get to Green," and SSA follows the OMB A-76 guidelines for calculating the percent of commercial positions competed or converted. We recommend SSA develop and consistently follow detailed written internal procedures, outlining the steps taken to compute and report the results of the performance indicators. Refer to recommendation number 3.

Additionally, SSA disagreed with the "Percent of Commercial Positions Competed or Converted" finding regarding the lack of clarity within the FY 2003 PAR that the numerator for this calculation is a composite number. We recommend that the PAR be updated to ensure that the performance indicator titles, definitions, and goals are explicit and internally consistent.

SSA indicated in its response to "Provide the Equivalent of 40 Hours of Training Annually to All Employees" performance indicator that "We are pleased that the review found that we met our goal and that in FY 2003 we provided an average of 68 hours of training per employee." It should be clarified that the PAR stated the Agency met its goal, not PwC. As discussed in this report, PwC found inconsistent data definitions, invalid data, and incomplete documentation to support the amounts reported. As a result, it was not possible for PwC to recalculate the average hours of training provided to SSA employees.

Appendices
APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Agency Comments

Appendix A
Acronyms
COSS Commissioner
DCOSS Deputy Commissioner
CSP Competitive Sourcing Plan
DCFAM Deputy Commissioner, Finance, Assessment, and Management
FAIR Federal Activities Inventory Reform
FPPS Federal Personnel Payroll System
FY Fiscal Year
GPRA Government Performance and Results Act
HRMIS Human Resource Management Information System
IVT Interactive Video Teletraining
MEO Most Efficient Organization
OB Office of Budget
OCS Office of Competitive Sourcing
OLU OnLine University
OMB Office of Management and Budget
PAR Performance and Accountability Report
P.L. Public Law
PMA President's Management Agenda
PwC PricewaterhouseCoopers
PWS Performance Work Statement
SSA Social Security Administration
U.S.C. United States Code

Appendix B
Scope and Methodology
We updated our current understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing including testing of source documentation, we performed the following as applicable:

Reviewed prior SSA, Government Accountability Office, and other reports related to SSA GPRA performance and related information systems.
Met with the appropriate SSA personnel to confirm our understanding of each individual performance indicator.
Flowcharted the processes (see Appendix C).
Where applicable, we tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).
Conducted and evaluated tests of the automated and manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.
For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.
Identified and extracted data elements from relevant systems and obtained source documents for detailed testing selections and analysis.
Identified attributes, rules, and assumptions for each defined data element or source document.
Tested the adequacy, accuracy, reasonableness, consistency, and completeness of the selection.
Recalculated the metric or algorithm of key performance indicators to ensure mathematical accuracy.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators being used appear to be valid and appropriate

given our understanding of SSA's mission, goals, objectives and processes. We followed all performance audit standards. In addition to the steps above, we specifically performed the following to test the indicators included in this report:

"GET TO GREEN" ON ALL PRESIDENT'S MANAGEMENT AGENDA (PMA) INITIATIVES

Reviewed the Office of Management and Budget (OMB) PMA criteria and applied such criteria to data reviewed.
Verified the OMB rating received by SSA on the Government results web site: http://www.results.gov/agenda/scorecard.html.

PERCENT OF COMMERCIAL POSITIONS COMPETED OR CONVERTED

Assessed the reliability of the data by reviewing the number of positions competed or converted as reported by SSA in its 2003 Performance and Accountability Report (PAR) to the source documents.
Reviewed the OMB Circular No. A-76 criteria for calculating positions competed or converted and applied such criteria to data reviewed.
Recalculated and verified the percent of positions competed or converted in 2001, 2002, and 2003 as shown in the PAR.

PERCENT IMPROVEMENT IN THE NEW HIRE RETENTION RATE

Reviewed The New Hire Attrition and Retention Report 1998-2000.
Compared and reconciled the number of new hires and terminations to SSA's Fiscal Year (FY) 2003 PAR with SSA data files.
Compared key dates in the data files to check for appropriate inclusion (i.e. hire date and termination date).
Compared terminated employees to new hire list to ensure only new hires from FY 2001 were included in the termination calculation.
Reviewed the list of FY 2001 new hires and FY 2001 and FY 2002 terminations to ensure no duplicates were included.
Performed an application control review over the Human Resource Management Information System (HRMIS).
Used programming specialist to determine the adequacy of the programming logic used to calculate the indicator.
Traced a selection of new hires to source documentation to verify authorization of employment.

Traced a selection of terminations to source documentation to verify authorization of dismissal
PROVIDE THE EQUIVALENT OF 40 HOURS OF TRAINING ANNUALLY TO ALL EMPLOYEES

Assessed the reliability of the data by inquiring of appropriate personnel as to courses offered.
Reviewed the OnLine University offerings at http://www.vcampus.com/ssaolu.
Obtained a list of all courses included in hours of calculation.
Compared and reconciled the number of courses offered to those numbers reported.
Recalculated and verified the number of hours made available to SSA employees.

Appendix C
Flowchart of "Get to Green" on all PMA Initiatives

"Get to Green" on all President's Management Agenda (PMA) Initiatives:

Information is provided to the Office of Budget (OB) by Lead Executive.
Information provided by Lead Executive is reviewed by the OB for completeness.
OB determines if the information is adequate.
If the information provided by Lead Executives is not adequate, the data is returned to components for further review or clarification.
If the information provided by Lead Executives is adequate, a draft report is prepared by OB.
The draft report is submitted to OB management for review.
If OB management determines the report is not adequate, it is returned for additional information and resolution with OB staff, or the contacts in the respective Lead Deputy offices.
If OB management determines the report is adequate, the draft report is submitted for Deputy Commissioner, Finance, Assessment, and Management (DCFAM) review.
If DCFAM determines the report is not adequate, it is returned for additional information and resolution with OB staff, or the contacts in the respective Lead Deputy offices.
If DCFAM determines the report is adequate, it is presented to the Commissioner (COSS) and Deputy Commissioner (DCOSS) for review.
If COSS and DCOSS determine the report is not adequate, it is returned for additional information and resolution with OB staff, or the contacts in the respective Lead Deputy offices.
If COSS and DCOSS determine the report is adequate, SSA provides update on progress and status to OMB.
OMB reviews the quarterly PMA update report.
OMB provides a draft assessment on progress and status to the SSA through a draft scorecard update.
A conference call with components, OB and Office of Management and Budget (OMB) is held to discuss draft scorecard update and clarify questions.
OMB / Social Security Administration (SSA) make revisions to the draft scorecard update and finalize the scorecard.
OMB provides final scorecard to SSA.
SSA reports on the PMA ratings in the performance and accountability report.

Flowchart of Percent Commercial Positions Competed or Converted

Percent Commercial Positions Competed or Converted

The Competitive Sourcing Plan (CSP) is presented to the components and the decision is made to compete or convert positions.

If the decision is made to compete positions, components identify the positions to be competed from those on the CSP and begin pre-planning. An official announcement is made to the unions, at the same time, an announcement is sent to FedBizOpps.gov. A Performance Work Statement (PWS) is developed. The PWS is sent to the SSA contracting office for approval. Once the PWS is approved by contracting, it is sent for solicitation. At the same time the PWS is sent for solicitation, the Social Security Administration (SSA) begins work on the agency tender, Most Efficient Organization (MEO). Bids are submitted from various vendors as well as the MEO. SSA performs an evaluation of all bids submitted to ensure the bid meet the PWS. Only bids which meet the PWS are enter into software provided by the Office of Management and Budget (OMB), (WINCOMPARE). WINCOMPARE identifies the lowest bidder. The bids are reviewed for other predefined criteria to consider besides price. The bids are reevaluated based on the predefined criteria and a winner is chosen. The contract is awarded to the winning bid.

If the decision is made to convert positions, the component submits a direct conversion proposal to the Office of Competitive Sourcing (OCS). OCS reviews this proposal and enters it into WinCOMPARE. If OCS does not approve of the proposal, questions / clarifications are submitted to the component. If OCS agrees with the decision to convert positions, the convention proposal is submitted for executive approval. If the convention proposal is awarded executive approval, the contract is awarded.

A transition plan is developed.

The contract is implemented.

The number of positions competed or converted are divided by the number of convertible positions listed on the 2000 Federal Activities Inventory Reform (FAIR) Act Inventory to get the percentage of positions competed or converted.

The percentage is reported on the Performance and Accountability Report (PAR).

Flowchart of the Percent Improvement in New Hire Retention Rate

Percent Improvement in New Hire Retention Rate

New employees hired (100 series) and terminated employees (300 series) are entered into Federal Personnel Payroll System (FPPS).
FPPS interfaces nightly with Human Resource Management Information System (HRMIS) and populates the HRMIS data file.
Automated reports are run to capture all new hires from 2001 and the RLNEWHIR report is produced (report of all 2001 New Hires).
Automated reports are run to capture all 2001 new hires who were terminated in 2001 and 2002. Only employees hired under qualifying pay plans are included. The RLNHSE98 is produced (report of all 2001 New Hires terminated in 2001 and 2002).
The total from report RLNHSE98 is divided by the total from RLNEWHIR.
The percent improvement in the new hire retention rate is reported on the Performance and Accountability Report (PAR).
Flowchart of Provide the Equivalent of 40 hours of Training Annually to All Employees

Provide the Equivalent of 40 hours of Training Annually to All Employees

The number of OnLine University (OLU) course hours, Interactive Video Teletraining (IVT) Automation Skills course hours and IVT general topic course hours offered are totaled.
The number of hours is reported in the Performance and Accountability Report as the number of training annually provided to all employees.

Appendix D
Agency Comments

SOCIAL SECURITY

MEMORANDUM 33296-24-1158

Date: July 30, 2004
To: Patrick P. O'Carroll, Jr.
Acting Inspector General

From: Larry W. Dye
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report "Performance Indicator Audit: President's Management Agenda Related Initiatives" (A-15-04-14070)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Please let me know if you have any questions. Staff inquiries may be directed to Candace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL (OIG) DRAFT REPORT "PERFORMANCE INDICATOR AUDIT: PRESIDENT'S MANAGEMENT AGENDA (PMA) RELATED INITIATIVES" (A-15-04-14070)

Thank you for the opportunity to review and comment on the draft report. We are pleased that the review found that we met or exceeded our goal on 3 of the 4 indicators. We are also pleased to report that, through our continuing efforts to improve our performance measurement and accountability processes, we have already taken action to address some of the areas identified as needing improvement. Our comments below address the findings for the four specific indicators.

"Get to Green" on all Presidential Management Agenda Initiatives

We are pleased that the review found that: 1) we met our goal; 2) the data provided was accurate and complete; and 3) the initiatives represent high-level strategic directions that cover 3 of our 4 strategic goals. However, we disagree with the finding that we have not documented policies and procedures related to the formal process to collect, review and provide PMA information to the Office of Management and Budget (OMB). The formal process for tracking progress of implementing PMA's as outlined by OMB calls for agencies to develop quarterly status reports in each of the categories. The internal documentation of how we provide information to OMB to make that judgment is secondary to OMB's systems and procedures. We continue to comply with OMB's reporting requirements in this area and we have recently made substantial progress toward meeting the actual goal of getting to green. OMB's scorecard as of June 30, 2004 (PDF file attached) shows that we are green on progress for all five PMA categories and green on status for three of the five.

Percent of Commercial Positions Competed or Converted

We disagree with the finding that we have not formally documented our procedures related to the development of the Competitive Sourcing Plan (CSP). We continue to follow OMB guidelines in its A-76 Circular that govern Agency activities. Copies of the documentation have been collected in one binder so that anyone working in this area has a reference immediately available and, therefore, we did not find it necessary to reinterpret the rules that we must follow. We also disagree with the finding that we did not articulate in the FY 2003 Performance and Accountability Report (PAR) that the numerator for this calculation is a composite number. The data definition as articulated in the PAR is consistent with established A-76 procedures. In the past we have consistently had our Federal Activities Inventory Reform Act (FAIR) inventory and Competitive Sourcing Plan approved by OMB using the government-wide guidelines.

Percent Improvement in the New Hire Retention Rate

We are pleased that the report found that we met and surpassed our goal. Regarding the finding that the description and calculation was incomplete, we believe the report should explain the basis and rationale for the intentional "by design" exclusion. Specifically, we excluded the wage grade (WG) positions since the annual new hire rate would be compared to the baseline previously established, we concluded that the data parameters should be consistent with the data parameters used in the baseline calculation and that did not include the WGs. We are pleased that you found that the impact of excluding those positions within the calculation had no material effect on the final calculation.

As part of our ongoing activities to improve the performance measurement and accountability process, we have: 1) improved our data collection and posting processes; and 2) implemented documentation maintenance and developed record keeping systems; however, we also maintain that this continues to be a valid measure.

Provide the Equivalent of 40 Hours of Training Annually to All Employees

We are pleased that the review found that we met our goal and that in FY 2003 we provided an average of 68 hours of training per employee. We acknowledge that there have been problems with the data definitions, data validity, and associated documentation. This is among a number of "Staff" measures that are being totally restructured for the FY 2005/2006 Annual Performance Plans (APP). We will consider PwC's comments as we develop new measures. Our response to the specific recommendation (#5) is provided below.

Recommendation 1

SSA should enhance the discussion of the linkage of the performance indicators to the Agency's strategic goals and objectives.

Response

We welcome external input, suggestions, and the contributions the Inspector General has made, and continues to make, in assisting us in developing a comprehensive performance measurement process. We too are committed in our efforts toward documenting relevant performance measures, ensuring our performance data are complete, and implementing adequate internal controls over our data systems and processes. We will consider and adopt where appropriate the suggestions for enhancements as we develop future APP's and PAR's.

Recommendation 2

SSA should ensure that the performance indicator titles, definitions, and goals are explicit, complete, and consistent.

Response

See response for recommendation #1.

Recommendation 3

SSA should improve documentation by maintaining descriptions of how the performance indicator goals were established, documenting the policies and procedures used to prepare and disclose the results of the performance indicators, and maintaining a complete audit trail for the results of the performance indicators.

Response

See response for recommendation #1.

Recommendation 4

SSA should completely and accurately calculate the actual results of the performance indicators in accordance with the original goal.

Response

See response for recommendation #1.

Recommendation 5

SSA should revise the indicator to measure the amount of job related training actually received by employees, and the effectiveness of the training in improving the employees' job related capabilities.

Response

We agree. It should be noted that for our largest workload component (Operations), data is available from the work measurement systems that allow us to compute the number of work-years spent on both new staff and ongoing training. The FY 2003 data from that system confirm that Operation's employees actually received more than 40 hours of training. For the revised FY-2005/FY-2006 APP we will propose new training performance goals that identify and provide training on job-specific competencies for specific mission-critical positions.

Overview of the Office of the Inspector General

The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Executive Operations (OEO). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit

OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations

OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General

OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Executive Operations

OEO supports OIG by providing information resource management and systems security. OEO also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, OEO is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.