Skip to content
Social Security Online
Office of the Inspector General
OIG Seal image
Blank Spacer Image

Audit Report - A-02-97-10201


Office of Audit

Benchmarking Payment Accuracy Performance Measures   (A-02-97-10201)   10/31/97 

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

FINDINGS

ORGANIZATIONS ISSUING REPETITIVE PAYMENTS MONITORED THE ACCURACY OF PAYMENTS

ORGANIZATIONS USED TWO METHODS TO MEASURE THE ACCURACY OF THEIR PAYMENTS

MOST ORGANIZATIONS MEASURED BOTH THE AMOUNT OF DOLLARS IN ERROR AND THE NUMBER OF CASES WITH ERRORS

ORGANIZATIONS TREATED OVERPAYMENTS AND UNDERPAYMENTS DIFFERENTLY

TWO TYPES OF PAYMENT ACCURACY RATES WERE USED

MOST ORGANIZATIONS DID NOT PROJECT THE COSTS OF ERRORS

MOST ORGANIZATIONS WERE REQUIRED BY A GOVERNMENTAL OR PROFESSIONAL ORGANIZATION TO MEASURE THE ACCURACY OF THEIR PAYMENTS

SSA’S PAYMENT ACCURACY REVIEW PROCESS IS SIMILAR TO MANY OF THE ORGANIZATIONS CONTACTED, BUT A FEW DIFFERENCES WERE NOTED

CONCLUSIONS AND RECOMMENDATIONS

APPENDICES

A - Description of Participating Organizations

C - Major Contributors to this Report

EXECUTIVE SUMMARY  

OBJECTIVE

The objective of this evaluation was to benchmark private and public business practices in measuring payment accuracy.

BACKGROUND

The Government Performance and Results Act (GPRA), enacted in August 1993, seeks to improve public confidence by systematically holding Federal agencies accountable for achieving program results. Agencies must set performance goals, measure performance against the goals, and report publicly on their performance. GPRA also calls for agencies to have strategic plans in place by September 30, 1997. Additionally, GPRA requires agencies to provide a basis for comparing results and for verifying and validating measured values. Similarly, Executive Order 12862 requires benchmarking customer service standards against the best in business. This benchmarking exercise, by examining the procedures and performances of other organizations in monitoring payment accuracy, will address both the GPRA and Executive Order concerns.

The Social Security Administration (SSA) annually conducts payment accuracy reviews of its title II Old-Age, Survivors, and Disability Insurance; and title XVI Supplemental Security Income (SSI) programs, including the Index of Dollar Accuracy (IDA) reviews and Stewardship reviews. These reviews are used to determine SSA`s payment accuracy rate through a quality review of a selection of sampled cases. The IDA measures payment accuracy for title II initial claims, title XVI initial claims, and field office (FO) redeterminations. Initial claims include original entitlements and related retroactive benefit decisions. The title II review does not examine payments made to individuals due to disabilities. The title XVI review does not include reviews of the initial disability decisions made by State Disability Determination Services.

The Stewardship reviews of title II and title XVI programs are not limited to just initial cases or redeterminations. The Stewardship reviews are based on a sample drawn from the entire universe of Retirement and Survivors Insurance (RSI) beneficiaries who received title II payments and from the entire universe of SSI recipients who received title XVI payments. The Stewardship reviews identify the overall payment accuracy, as well as the rate of overpayments and underpayments made by SSA.

FINDINGS

ORGANIZATIONS ISSUING REPETITIVE PAYMENTS MONITORED THE ACCURACY OF PAYMENTS

ORGANIZATIONS USED TWO METHODS TO MEASURE THE ACCURACY OF THEIR PAYMENTS

MOST ORGANIZATIONS MEASURED BOTH THE AMOUNT OF DOLLARS IN ERROR AND THE NUMBER OF CASES WITH ERRORS

ORGANIZATIONS TREATED OVERPAYMENTS AND UNDERPAYMENTS DIFFERENTLY

TWO TYPES OF PAYMENT ACCURACY RATES WERE USED

MOST ORGANIZATIONS DID NOT PROJECT THE COSTS OF ERRORS

MOST ORGANIZATIONS WERE REQUIRED BY A GOVERNMENTAL OR PROFESSIONAL ORGANIZATION TO MEASURE THE ACCURACY OF THEIR PAYMENTS

SSA’S PAYMENT ACCURACY REVIEW PROCESS IS SIMILAR TO MANY OF THE ORGANIZATIONS CONTACTED, BUT A FEW DIFFERENCES WERE NOTED

CONCLUSIONS AND RECOMMENDATIONS

The practices used by private and public organizations have highlighted methods that could help SSA measure payment accuracy more efficiently and effectively. We recommend that SSA consider the following changes to its procedures:

Report case payment accuracy rates.

Report a netted and non-netted payment accuracy dollar rate.

Eliminate life-cycle accuracy rates as a performance measure.

Reconsider sample sizes.

AGENCY COMMENTS

SSA did not agree with our first three recommendations. The Agency stated that the addition of either a case payment accuracy rate or a non-netted payment accuracy dollar rate would misrepresent the extent of errors in the payment process. Additionally, the Agency believed that the life-cycle accuracy rate presented a more realistic measurement of the impact of payment errors. SSA noted in response to our final recommendation that it had recently reduced sample sizes and would continue to do so as long as statistical precision was not compromised.

OIG RESPONSE

OIG believes that the addition of both a case payment accuracy rate and a non-netted payment accuracy dollar rate would provide a clearer indication of how accurately the Agency disburses payments to its customers. While we agree that the life-cycling accuracy rate does show a projected cost to the Agency, we do not believe that it is an accurate performance measure of current payment accuracy. Lastly, we appreciate the Agency’s efforts to reduce the sample sizes for the Stewardship and IDA reviews and encourage consideration of further reductions.

INTRODUCTION

OBJECTIVE

The objective of this evaluation was to benchmark private and public business practices in measuring payment accuracy.

BACKGROUND

GPRA, enacted in August 1993, seeks to improve public confidence by systematically holding Federal agencies accountable for achieving program results. Agencies must set performance goals, measure performance against the goals, and report publicly on their performance. GPRA also calls for agencies to have a strategic plan in place by September 30, 1997. These plans are to contain a discussion of outcome-related goals and objectives, how the goals and objectives are to be achieved, and the program evaluations used to establish and revise the goals and objectives.

GPRA also requires agency heads to submit annual reports on program performance to the President and Congress beginning March 31, 2000, and requires the Office of Management and Budget to require each agency to prepare an annual performance plan which, beginning with Fiscal Year 1999, will be included in a Federal Government’s performance plan for the overall budget. The required annual performance plans must have: 1) objective, quantifiable, and measurable performance goals; 2) performance indicators to be used in measuring the relevant outputs, service levels and outcomes; and 3) a means for comparing, verifying, and validating data.

Within SSA, the Office of Strategic Management (OSM) is the lead component responsible for developing the Agency`s strategic and performance plans. The OSM is currently updating the Agency`s strategic plan and leading the effort to develop the Agency`s first performance plan. The actual task of developing and presenting a performance plan to the Agency`s Executive Staff has been given to an intercomponent work group. At this time, the work group has completed a draft performance plan that will be presented to the Executive Staff. Three of the performance measures contained in the draft are measures of payment accuracy in SSA`s three programs: Old-Age and Survivors Insurance, Disability Insurance, and SSI.

In September 1993, the National Performance Review recommended that all Government agencies develop and publish customer service standards. President Clinton responded to that recommendation with Executive Order 12862, which calls for the Federal Government to be customer-driven. To achieve this goal, each Federal agency is required to have customer service standards equal to the "best in business."

Executive Order 12862 requires benchmarking customer service standards against the best in business. Additionally, GPRA requires agencies to provide a basis for comparing results and for verifying and validating measured values. This benchmarking exercise, by looking at the procedures and performances of other organizations in monitoring payment accuracy, will address both the Executive Order and GPRA concerns.

SSA annually conducts payment accuracy reviews of its title II and title XVI programs, including the IDA reviews and Stewardship reviews. These reviews are used to determine SSA`s payment accuracy rate through quality reviews of a selection of sampled cases.

IDA Reviews

SSA annually conducts IDA reviews for title II initial claims and title XVI initial claims and FO redeterminations. Initial claims include original entitlements and related retroactive benefit decisions. The title II review does not examine payments made to individuals due to disabilities. The title XVI review does not include reviews of the initial disability decisions made by State Disability Determination Services.

The title II review examines initial entitlement decisions for RSI claims. The RSI program provides benefits to retired workers, their families, and dependent survivors. SSA processes over 3 million such claims a year. The review is used to identify and make recommendations for changes needed to address issues of payment errors. The reviews develop two payment accuracy rates. There is the initial payment accuracy rate and a life-cycle accuracy rate, which shows the dollar accuracy of adjudicative decisions over the expected life of the beneficiary.

The title II review examines a sample of payments selected through the following method of probability sampling. Counties are clustered into groups to minimize travel costs. Each cluster is assigned a probability of selection based on Regional Office of Program and Integrity Reviews’ (ROPIR) desire to visit the counties in that cluster. For example, a cluster with counties that minimizes a certain ROPIR’s travel costs will be assigned a higher probability of selection (90 to 95 percent) than the clusters of counties that would involve higher travel costs. Given the higher probability of selection, the desired clusters are more likely to be selected, but all clusters have the potential to be selected. The cases are chosen from the 48 contiguous States and the District of Columbia. SSA selects approximately 2,500 cases a year for this review.

The title XVI IDA review examines payments for the SSI program, which provides a minimal level of income for the aged, blind, and disabled with limited income and resources. Each year, SSA processes approximately 2 million initial SSI claims and 1 million FO redeterminations with benefit payments totaling over $20 billion annually. As with the title II review, the title XVI review is used to identify and correct payment errors. Dollar payment accuracy rates are derived from the data collected for this review.

In the title XVI IDA review, a stratified cluster design is used to select the sample. Four strata are used, which are defined by the travel costs for ROPIR staff. The first strata contains FOs which involve the least amount of travel expense. Approximately 80 percent of the sample is drawn from the first strata. The remaining 20 percent of the sample is drawn from strata II to IV, which have higher travel costs. Each year, the title XVI review examines approximately 1,000 initial awards, 1,000 FO redeterminations with a change, and 1,000 FO redeterminations without a change.

The process used for both the title II and title XVI IDA reviews are similar. In both cases, quality assurance staff review a case to determine if the benefit payment amount was based on accurate information and was calculated correctly. All of the beneficiaries or recipients in the sample under review are contacted to verify information within the case file. The title II beneficiaries are either called or visited, while title XVI recipients are all visited by a quality assurance staff member.

Stewardship Reviews

SSA uses another review to measure the accuracy of payments made to its beneficiaries and recipients. The Stewardship reviews of title II and title XVI programs are not limited to initial cases or redeterminations. The Stewardship reviews are based on a sample drawn from the entire universe of RSI beneficiaries who received a payment and SSI recipients who received a payment. The Stewardship reviews identify the payment accuracy for overpayments and underpayments.

METHODOLOGY

This review was a benchmarking exercise which examined private and public business processes. Thirty-one organizations were contacted; nine were private organizations and the remainder were State or Federal agencies. All of the organizations made repetitive payments or monitored organizations that made repetitive payments. An effort was made to interview only organizations that made repetitive payments such as those made by SSA. The organizations contacted made payments to individuals who were retired, disabled, or in need of income assistance. The organizations provided the following type of payments: State income assistance, unemployment, State or Federal pensions, private pensions, private disability, Federal disability, and private annuity payments. Many of the organizations were identified through the International Benchmarking Clearinghouse of the American Productivity and Quality Center, of which SSA is a member. The private businesses are very well-known entities and could easily be classified as the best in business.

A detailed discussion guide was used to interview the individuals responsible for the payment accuracy reviews within their organizations. The discussion guide addressed the processes used by the organizations to measure the accuracy of the repetitive payments made to customers, the steps used to analyze the information reviewed, and the formulas used to determine payment accuracy rates. The discussion guide also obtained demographic information about each organization. Eight of the organizations also submitted reports and other documentation related to their payment accuracy reviews.

The data from the discussion guides was entered into a data base and also reviewed manually. Similarities and differences in the review processes used by organizations were identified. The practices used by the organizations were also compared to the practices used by the Office of Program and Integrity Reviews in conducting the IDA and Stewardship reviews. Part of the analysis was aided by SAS, a statistical computer software.

Our review was conducted from December 1996 to April 1997. It was conducted in accordance with the Quality Standards for Inspections issued by the President`s Council on Integrity and Efficiency. 

FINDINGS

ORGANIZATIONS ISSUING REPETITIVE PAYMENTS MONITORED THE ACCURACY OF PAYMENTS

We spoke with 28 organizations that issue repetitive payments and 3 organizations that regulate or monitor organizations that issue repetitive payments. Of the 31 organizations contacted, 22 were Government agencies and 9 were private businesses. The Government agencies consisted of State public assistance agencies, State unemployment departments, State insurance departments, State and Federal pension divisions, and a Federal disability insurance division. The private businesses consisted of insurance companies, financial and investment firms, and the pension divisions of large organizations.

All of these organizations monitored the accuracy of payments released to customers. However, they reviewed payment accuracy with different frequency. Half of the organizations measured the accuracy of their payments on a monthly basis. This was the most common time frame reported by the organizations. Other reviews were conducted on a weekly, quarterly, or yearly basis.

ORGANIZATIONS USED TWO METHODS TO MEASURE THE ACCURACY OF THEIR PAYMENTS

The organizations contacted reported using two distinct methods in measuring the accuracy of payments. Two-thirds of the organizations we spoke with recalculated the amount of the benefit that their customers should have received and compared it to the amount that the customers were actually paid. The other third reviewed information from their computer systems used for making the repetitive payments. Many of these organizations performed a reconciliation from month to month with the balances provided by their payment system.

The basic model used to recalculate a customer’s correct payment amount in order to compare it to the actual payment made was quite similar for the organizations that used this methodology. In these organizations, audit or quality control staff manually recalculated the amount of the payment a customer should have received. Once this figure was computed, it was compared to the amount of money already paid to a customer which was provided through the organization’s computer system or a canceled check.

The organizations that recalculated the amount of a customer’s payment started with the information available within the customer’s case folder or computer file. For some organizations, the information within the file was sufficient to do the recalculation. Many (58 percent), however, verified information that was missing or unclear in the client’s files. They contacted individuals, such as employers, bank officials, and school officials, to verify information that affected the amount a customer was eligible to receive. These organizations also determined whether the customer was eligible to receive a payment at all. Almost all of the organizations that manually recalculated the payment amount due a customer also redetermined the overall eligibility of the customer.

The organizations that relied on their computer systems to measure payment accuracy used different methods. Half of the organizations that relied solely on their computer files performed a reconciliation of their computer files from month to month. They took the balance of all payments made for the previous month, added any new payments, subtracted any terminated payments, and compared that result to the current month’s balance on their computer system.

Other organizations that relied on their computer systems to measure payment accuracy examined a few case files to determine the payments due to the customers. They compared these few figures to the amounts in the computer system that generated the payments. They were checking the controls within the computer system to ensure that it processed payments accurately. One Government organization sent letters to their customers asking them to verify that they received the amount shown as payable by the computer payment system.

Private businesses were more likely than Government agencies to review their computer system than to manually recalculate payments and compare the recalculation to a payment already made. Two-thirds of the private businesses reviewed computer files and system controls. Only 18 percent of the Government agencies used this methodology.

Most of the Organizations Reviewed a Sample of Payments

Eighty-four percent of the organizations reviewed a sample of payments when measuring their payment accuracy. Most of these organizations chose a random sample of cases. The size of the samples differed for each organization, but most of them were equal to or less than 500 cases. Overall, 70 percent of the samples consisted of 500 or fewer cases. The remaining samples, for the most part, were between 1,000 and 2,000 cases. There were 2 organizations with samples larger than 2,000 cases. One organization, a private financial company, reported reviewing 133,000 cases when measuring the accuracy of payments issued to customers.

All of the Government agencies selected a sample of cases for their payment accuracy reviews. The private businesses were almost evenly split between reviewing the entire universe or reviewing a sample of payments. Fifty-six percent of these organizations reviewed the entire universe of payments and the remaining 44 percent reviewed only a sample of payments.

Thirty-one percent of the organizations that used a sample reported excluding some cases from review after they had been chosen for the sample. Cases were generally excluded due to the inability of the organizations to contact the customer under review. For example, organizations excluded cases because the customer refused to cooperate with the review process. Others were excluded because the customer could not be located to provide additional information. Also, customers that moved out of State during the review period were excluded by some State agencies. Finally, some organizations excluded cases that had checks sent back uncashed.

Some cases were excluded from review even before a sample was drawn. The reason a claim would be excluded from review was that no payment was made. Most of the organizations we spoke with (81 percent) only reviewed a claim if a payment was made.

Almost half of the organizations that used samples stratified their samples to ensure that certain subgroups were represented. The organizations desired to examine specific subgroups such as recently approved initial claims, geographic regions and counties, and disability claims. Stratification ensured that organizations had an adequate number of these type of cases within their samples to examine and project back to the universe of all payments. Generally, strata were taken for groups that would have been too small for analysis if a simple random sample was drawn from the universe.

Amount of Staff Hours Needed to Conduct the Payment Accuracy Reviews Differed among Organizations Contacted

The amount of staff hours needed to complete a payment accuracy review varied from organization to organization. One of the organizations needed only 1 hour to complete its payment accuracy review, while another organization reported having used 24,000 staff hours. Generally, organizations that manually recalculated their customers’ authorized payment amounts and compared those to actual payments sent to the customers, used more staff hours than organizations that relied on computer systems to monitor payment accuracy. The former organizations tended to need hundreds or thousands of staff hours while the latter organizations reported having used 40 staff hours or less to complete their payment accuracy reviews.

The method used to measure the accuracy of payments appears to be related to the number of staff hours needed to complete a payment accuracy review. Organizations that reviewed computer systems’ controls needed fewer hours to do so than organizations that manually recalculated payment amounts. This is true even though they were not necessarily reviewing smaller samples. Most of the organizations that reviewed their computer systems to measure payment accuracy examined samples of hundreds or thousands. Most of the organizations that manually recalculated payment amounts reviewed samples of under 125 cases.

MOST ORGANIZATIONS MEASURED BOTH THE AMOUNT OF DOLLARS IN ERROR AND THE NUMBER OF CASES WITH ERRORS

Most of the organizations contacted (67 percent) reported having measured both case and dollar errors when they reviewed payment accuracy. Thirteen percent measured only case errors, 10 percent measured only dollar errors, and 10 percent did not measure either case or dollar errors. The organizations that did not measure errors used their computer systems to reconcile the current month’s balance of payments to the previous month’s balance. They were looking only to see if the overall balance was correct and were not concerned with individual dollar or case errors.

Pie chart of Percent of Organizations That Measured Dollar and Case Errors

One State agency reported that the measurement of dollar errors could identify which areas were in most need of improvement. It particularly measured the causes of overpayments, in an attempt to isolate and correct errors in the payment system. One person from a private insurance company stated why he thought it was important to record the number of cases in error. He said that one practice he found ineffective is, ". . . concealing the errors you make in the large volume you do. If you only have a $10,000 error and you paid out a million dollars, you look good. But, you may have made hundreds of errors and you are really not doing all that well."

ORGANIZATIONS TREATED OVERPAYMENTS AND UNDERPAYMENTS DIFFERENTLY

In most of the organizations we spoke with, it was possible for customers to have both an overpayment, where they were paid more than they should have, and an underpayment, where they were paid less than they should have, during the period being reviewed. A majority of these organizations (52 percent) treated each instance of an underpayment or overpayment as separate errors. For example, if a customer was overpaid $100 and also underpaid $100, there would be a dollar error of $200.

The remaining offices (48 percent) did not look at an overpayment or underpayment as two separate errors. In these organizations, if one customer had both an overpayment and an underpayment within the review period, all overpayment errors were added and underpayment errors were subtracted to create one net error dollar amount. For example, if a customer was overpaid $100 and also underpaid $100, there would be no dollar error since the two figures netted out.

Private businesses were more likely than Government agencies to report overpayments and underpayments as separate errors. Sixty percent of private businesses who had customers with both an overpayment and an underpayment in a single claim reported overpayments and underpayments as separate errors. Forty-five percent of the Government agencies reported these types of payments as separate errors.

TWO TYPES OF PAYMENT ACCURACY RATES WERE USED

All of the organizations that performed manual reviews of cases in reviewing payment accuracy created payment accuracy rates. Conversely, all but two of the organizations that relied on a computer system to review payment accuracy did not compute accuracy rates. The organizations that computed a payment accuracy rate used two basic formulas. One of the formulas focused on the amount of dollars paid in error and the other focused on the number of errors made in the handling of customers cases.

Eighty-three percent of the organizations that computed accuracy rates used a dollar payment accuracy rate. The basic formula used in evaluating the amount of dollars in error was the total amount of dollars in error divided by the total amount of dollars in the sample (or universe if the entire universe was being examined). This formula provided the percentage of dollars in error. This number subtracted from the number one gave the percentage of dollars paid correctly.

The organizations contacted had different thresholds on what constituted a dollar error, but all were low. About half of the organizations contacted considered a payment in error if it was one cent different than it should have been. All but one of the remaining organizations considered a payment in error if it was one dollar or five dollars different from what the payment should have been. The last organization had an error threshold of $25.

Some organizations reported calculating more than just the overall dollar payment accuracy rate. Organizations also looked at specific types of dollar errors. They examined the percentage of dollars in the sample that were overpayments and those that were underpayments separately. The organizations that combined, or netted, overpayments and underpayments together did not compute these separate dollar error rates.

Thirty-nine percent of the organizations that computed payment accuracy rates used a case payment accuracy rate. The basic formula used in evaluating the number of errors was the total number of payment errors divided by the total number of cases in the sample (or universe if the entire universe was being examined). This formula provided the case error rate. The case error rate subtracted from the number one yielded the case payment accuracy rate.

Organizations reported the need for specific information related to the type of errors involved in the process of making payments to customers. They calculated the percentage of errors caused by their employees versus the percentage of errors caused by the customers. (Customers were often required to report personal information to determine eligibility for payment.) They also examined the percentage of payments not made in a timely fashion, the percentage of payments not received by the customer, the percentage of payments not stopped in time resulting in an overpayment, and the percentage of cases with incorrect information regardless if the information caused a payment error or not. These figures were collected to monitor and improve specific aspects of the payment process.

The number of organizations using each type of payment rate was not mutually exclusive. Some of the organizations (22 percent) that used payment accuracy rates reported using both a dollar payment accuracy rate and a case payment accuracy rate (see graph on next page). These organizations believed that both rates were necessary to ensure the proper use of funds and to ensure a high level of quality customer service.

Pie chart of Percent of Organizations Using Dollar and Case Rates

MOST ORGANIZATIONS DID NOT PROJECT THE COSTS OF ERRORS

Eighty-three percent of the organizations did not project how much an error found during the review would have cost the organization if it had not been corrected. The organizations that reported projecting such costs did so for varying lengths of time. One organization projected how much errors cost the organization for each year. Another organization calculated the cost of errors, but only up to the time they were found during the review. They went back and identified when the error began and then calculated the costs for the organization from that time to the time the error was discovered. None of the organizations calculated how much errors cost for a period greater than a year.

MOST ORGANIZATIONS WERE REQUIRED BY A GOVERNMENTAL OR PROFESSIONAL ORGANIZATION TO MEASURE THE ACCURACY OF THEIR PAYMENTS

Three-quarters of the organizations contacted were required by some governmental or professional body to monitor the accuracy of payments sent to customers. Government agencies were twice as likely as private businesses to be required to measure their payment accuracy. Eight-six percent of the Government agencies were required to conduct payment accuracy reviews, while 44 percent of the private businesses were required to conduct reviews. While these governmental or professional bodies required many of the organizations to measure payment accuracy, not all mandated which process to use.

Sixty-one percent of the organizations required to measure accuracy had specific procedures to follow that were dictated by an outside organization. The organizations that mandated procedures dictated the type of measure to be used, the sample size, and the procedures to be used during the review. The remaining organizations that were required to measure payment accuracy were left to their own discretion on how to conduct their review.

SSA’S PAYMENT ACCURACY REVIEW PROCESS IS SIMILAR TO MANY OF THE ORGANIZATIONS CONTACTED, BUT A FEW DIFFERENCES WERE NOTED

As noted in the background section of this report, SSA conducts payment accuracy reviews for title II and title XVI payments. Similar to many of the organizations contacted, SSA selects a sample of payments and manually reviews and recalculates each customer’s payment amount to compare it to what actually has been paid. Another similarity is the use of stratification to ensure that certain subgroups are large enough for analysis in the sample reviewed. Also, SSA does exclude some cases from review as did some of the organizations contacted for this study.

SSA Reports Only a Dollar Accuracy Rate

SSA reports on the percentage of dollars paid correctly through dollar payment accuracy rates. While not all organizations reported the use of a case payment accuracy rate, three-fourths of the private businesses that calculated accuracy rates did. Some of the businesses that used case payment accuracy rates stated that they used this information to let their staff know how they were performing and to train them in areas that consistently had errors.

SSA Combines Overpayments and Underpayments as One Error

In calculating the overall amount of dollars in error, SSA combines, or nets, overpayments and underpayments within a case and reports one error dollar amount per case. A majority of the organizations interviewed (52 percent) who could have claims with both an overpayment and an underpayment during the review period and a majority of the private businesses (60 percent) with such claims did not net such payments. They reported the overpayments and underpayments in each case as separate errors. While not an overwhelming majority, many businesses believed it is important to identify all dollar errors separately rather than netting. These private businesses reported that identifying such errors separately provided information that helped improve all aspects of the payment process.

SSA Projects the Costs of Errors Through "Life-Cycling;" Most Organizations Do Not Make Projections

SSA uses a process known as "life-cycling" to estimate the magnitude of an error made during the initial claims process. The life-cycle accuracy rate reported by SSA in its title II IDA reviews projects the costs of errors over an actuarial life expectancy of 17 years. The actuarial projection of 17 years is an average model based on the type of error, when the error should be corrected and the estimated life of a beneficiary. Most organizations (83 percent) contacted for this review reported that they do not make projections on how much errors cost if they went uncorrected. Those that did project the costs of errors did not project out to the potential life of the customer.

SSA Reviews Are More Time Consuming and Its Samples Are Larger Than Other Organizations

SSA’s IDA and Stewardship reviews are more time consuming than other organizations’ reviews. SSA required approximately 80,000 staff hours to complete the 1995 title XVI IDA review alone. The average number of staff hours for the organizations reviewed was 1,963 hours, with a median of 300 hours. The average is much larger than the median due to 2 large outliers of 8,870 and 24,000 hours. Regardless, the time needed for one of the IDA reviews is three times larger than the highest number of hours needed by the organizations we contacted.

SSA may require more hours because it reviews a larger sample than most of the organizations contacted. Overall, 70 percent of our respondents’ samples consisted of 500 or fewer cases. The remaining samples, for the most part, were between 1,000 and 2,000 cases. There were 2 organizations with samples larger than 2,000 cases. They reviewed samples of 16,000 and 133,000 respectively. SSA reviews samples of approximately 2,500 and 3,000 cases for the title II and the title XVI IDA reviews, respectively, and another 1,500 cases for each of the Stewardship reviews.

CONCLUSIONS AND RECOMMENDATIONS

GPRA was established to ensure that Government agencies have adequate information to measure and improve their performance. GPRA seeks to improve Federal program effectiveness by focusing on results and service quality and to help Federal managers improve service delivery by providing them with information about program results and service quality. Additionally, it seeks to improve public confidence by providing data on program performance.

Benchmarking business practices reveals the processes that other organizations use and value as part of their operations. A review of many organizations’ policies and procedures helps to show the standards currently being used. This information is valuable as a point of comparison. In its proposed strategic plan, SSA highlights benchmarking other organization’s practices as a useful method to achieve its goal to make SSA program management the best in business.

Our review of 31 organizations’ practices highlighted the types of information the organizations believe are necessary and useful in measuring the accuracy of their payments. Their practices help to establish some standards on the type of performance data used by organizations to help ensure the provision of correct payment amounts to individuals. Their measurements of payment accuracy also identify problems in the payment process.

The practices used by private and public organizations have highlighted methods that could help SSA measure payment accuracy more efficiently and effectively. We recommend that SSA consider the following changes to its procedures.

Report Case Payment Accuracy Rates: SSA has performance measures that are related to payment accuracy. One of these measures, providing proper stewardship of funds, can be measured through SSA’s dollar payment accuracy rate. A measure of the dollars properly spent provides the public with an indication of SSA’s ability to properly manage title II and title XVI funds.

An additional performance measure that could provide a clear display of the level of service customers are receiving is a case payment accuracy rate. The use of this performance measure would provide an indication of SSA’s ability to meet its goal of providing world-class customer service. Similarly, the absence of a case payment accuracy rate deprives the Agency of a useful measurement of payment performance. This is particularly true since the dollar payment accuracy rate currently does not give an accurate indication of the number of errors. This is especially true when the netting of overpayments and underpayments is used. Netting effectively masks the effect of some errors.

A measurement of the total number of errors can highlight areas that are in particular need of improvement, or identify staff who require additional training. While the dollar payment accuracy rate provides a measure of the Agency’s stewardship of SSA funds, the case payment accuracy rate would show the Agency’s ability to provide quality service. Such an indicator would help assure the public that SSA is providing service that is equal to the best in business.

Report a Netted and Non-netted Payment Accuracy Dollar Rate: SSA currently nets, or combines, overpayments and underpayments within a case and reports one error dollar amount per case. These dollar amounts are used in calculating SSA’s dollar payment accuracy rates. This measure does not indicate the total amount of dollars that were paid incorrectly. Reporting overpayment and underpayment amounts separately would provide a clear indication of the total amount of dollars erroneously paid.

While the use of data with netted dollar errors indicates the overall amount of dollars that were released appropriately, it alone does not provide decision makers and the public with a performance measure that indicates the accuracy of the process used to make payments. SSA has created performance measures for the percent of payments without overpayments and those without underpayments in the Business Plan: Fiscal Years 1998-2002. These measures provide useful information and should be included with other payment accuracy performance measures in all publications reporting on the accuracy of the payment process.

Eliminate Life-Cycle Accuracy Rates as a Performance Measure: Only 17 percent of the organizations reported projecting the cost of errors and none of them reported projecting to the actuarial life of the error. At most, the amount of an error was projected for the year in which it occurred or up to the time it was identified. SSA should reconsider the value of life-cycling as a measurement of performance. While the life-cycle payment accuracy rate can provide information on the magnitude of a payment error made today over the projected life of a claimant, it does not indicate how many dollars were paid erroneously today or how many errors occurred in the payment process. Life-cycling information can be useful in indicating which reasons for errors have the greatest effect over time, but it is not an accurate performance measure of current payment accuracy.

Reconsider Sample Sizes: SSA devotes significantly more staff hours to measure payment accuracy than other organizations. Sample size may play a role in this. Almost all of the organizations contacted used smaller samples than SSA and, accordingly, required less staff hours to complete their payment accuracy reviews. This is true even though many of these organizations use methods similar to SSA’s, including contacting the paid customer during the review. Even given the significant amount of payments SSA makes, its use of large samples in the IDA and Stewardship reviews may not be necessary.

SSA has taken action to reduce the sample sizes reviewed, particularly in the title II and the title XVI IDA reviews. The samples have been cut approximately in half over the last 2 years. Even further reductions may be appropriate. The additional precision gained by large sample sizes reduces significantly once samples get larger than a few hundred observations, regardless of the size of the universe. SSA may be able to save resources in this area by reducing sample sizes.

The data used for SSA’s payment accuracy reviews are used for other analyses. For example, the data for the Stewardship reviews are used to determine profiling for SSI redeterminations. SSA should review all of the needs for the payment accuracy data and consider any methods that will allow for a reduction in the samples used for payment accuracy measurement.  

AGENCY COMMENTS

SSA did not agree with our first three recommendations. The Agency stated that the addition of either a case payment accuracy rate or a non-netted payment accuracy dollar rate would misrepresent the extent of errors in the payment process. Additionally, the Agency responded that the life-cycle accuracy rate presented a more realistic measurement of the impact of payment errors. SSA noted in response to our final recommendation that it had recently reduced the sample sizes of both the SSI Stewardship and the SSI IDA reviews. The Agency stated that it will continue to attempt to reduce sample sizes as long as statistical precision is not compromised. (See Appendix B for the full text of the Agency’s comments.)  

OIG RESPONSE

OIG believes that the addition of both a case payment accuracy rate and a non-netted payment accuracy dollar rate would provide a clearer indication of how accurately the Agency disburses payments to its customers. While the current netted dollar rate does measure proper stewardship of funds, a case payment accuracy rate and a non-netted payment accuracy rate would measure the actual number of payment errors and the total amount of dollars in error, respectively. We believe, as do many of the organizations we spoke with, that this type of information is necessary to monitor and improve the payment process. While we agree that the life-cycling accuracy rate does show a projected cost to the Agency, we do not believe that it is an accurate performance measure of current payment accuracy. Lastly, we appreciate the Agency’s efforts to reduce the sample sizes for the Stewardship and IDA reviews and encourage consideration of further reductions. 

APPENDICES

APPENDIX A 

DESCRIPTION OF PARTICIPATING ORGANIZATIONS

Thirty-one organizations were interviewed for this benchmarking exercise.

Twenty-eight of the organizations issued repetitive payments to customers and three monitored organizations that issued repetitive payments to customers. All of the organizations reviewed the accuracy of payments made to customers.

Twenty-two of the organizations were Government agencies and nine were private businesses. The organizations were contacted since they monitored payments that were similar to SSA payments, i.e., payments were paid due to retirement or disability. The governmental organizations contacted consisted of State public assistance agencies, State unemployment departments, State insurance departments, State and Federal pension agencies, a Federal disability division, and a national social security organization. The private businesses consisted of insurance companies, financial and investment firms, and pension divisions of large organizations. These businesses were well-known organizations that are equal to the best in business.

Twenty-nine percent of the organizations served customers on a national level and 19 percent served customers on an international level. The remaining organizations were State organizations that served customers within their State. The organizations served from 4,500 to 60,000,000 customers per year. The average number of customers served was 4,677,763. Most of the organizations (71 percent) issued payments on a monthly basis. The remaining organizations issued payments to their customers on a weekly or bi-monthly basis.

Three-quarters of the organizations reported that the amount of a payment was subject to change after a customer started receiving their payments. The reasons for changes were generally related to changes in the customers’ living arrangements, work status, number of dependents, or health status.

The number of payments released each payment cycle ranged from 700 to 3,450,193. The average number of payments was 595,666. The average amount of a payment for each organization ranged from $75 to $1,184. The mean payment amount for all of the organizations was $532. The total amount of dollars released per payment cycle was between $750,000 and $2,400,000,000, with an average of $352,191,426.  

APPENDIX C

 MAJOR CONTRIBUTORS TO THIS REPORT

Office of the Inspector General

Scott Patterson, Director, Evaluations and Technical Services
Tim Nee, Acting Deputy Director
Arthur Treglia, Senior Auditor

  Link to FirstGov.gov: U.S. Government portal Privacy Policy | Website Policies & Other Important Information | Site Map
Need Larger Text?
  Last reviewed or modified