Skip to content
Social Security Online
Office of the Inspector General
OIG Seal image
Blank Spacer Image

Audit Report - A-02-96-02202


Office of Audit

Cataloging Social Security’s Customer Service Monitoring - A-02-96-02202 - 9/24/97

TABLE OF CONTENTS

EXECUTIVE SUMMARY

INTRODUCTION

FINDINGS

CONCLUSION AND RECOMMENDATIONS

APPENDICES

Appendix B - Major Contributors to This Report

EXECUTIVE SUMMARY

OBJECTIVE

The objective of this evaluation was to catalog the Social Security Administration’s (SSA) efforts to monitor its service to the public and to identify monitoring gaps, duplicative efforts, and opportunities for improving coordination and quality.

BACKGROUND

One of SSA’s goals is to provide "world-class service." To that end, numerous SSA components conduct customer service monitoring reviews. Because we were unable to identify any catalogs or summaries of all such reviews, we attempted to create a listing of reviews. Additionally, we looked at factors that affect the quality of reviews, such as review standards and staff training.

In order to identify all the customer service reviews that SSA components conducted in Fiscal Year (FY) 1996, inquiries were sent to all Deputy Commissioners. The Deputy Commissioners were asked if any customer service reviews were conducted by their components and, if so, the purpose, cost, and frequency of each review. We received a response from each of the Deputy Commissioners. We then contacted the individual components identified as being involved in customer service reviews to obtain more specific information such as the professional background of the reviewers, the source of the request for the review, and the controls in place to ensure quality reviews.

FINDINGS

  • SSA Conducted a Significant Number of Customer Service Reviews
  • The Reviews Lacked Uniform Standards
  • The Training and Professional Backgrounds of Reviewers Varied
  • There Was No Formal Process for Coordination of Reviews Within SSA
  • The Reported Staff Hours Required to Complete Reviews Within SSA Varied Greatly

CONCLUSION AND RECOMMENDATIONS

While SSA has numerous initiatives to monitor its service to the public, these initiatives are not coordinated and some lack the rigor of professional standards. Additionally, two prior Office of the Inspector General (OIG) reviews and SSA’s Customer Service Executive Team’s independent contractor found that SSA’s sample sizes are often excessively large. Given the extent of resources committed to these initiatives, we recommend that SSA:

  • Develop a coordination mechanism to ensure that review resources are efficiently used (this is consistent with the recommendations of the Customer Service Executive Team’s contractor).
  • Perform cost-benefit evaluations of the sample sizes for customer service reviews conducted by the Agency in order to limit the staff hours required to perform those reviews.
  • Institutionalize existing Agency-wide professional standards for conducting customer service reviews.
  • Develop and publicize consistent training requirements for staff assigned to conduct customer service reviews in order to ensure the integrity of the reviews.

AGENCY COMMENTS

SSA acknowledged the need to better coordinate and integrate the Agency’s service monitoring activities, and has taken steps to meet this challenge. Earlier this year, SSA retitled the Chief Policy Officer position as the "Senior Advisor to the Commissioner on Customer Service Integration." This was done to reinforce the position’s role as principal contact and coordinator on both reinvention and customer service activities. Additionally, the Agency’s Customer Service Executive Team was reconvened to develop an integrated market measurement system.

In response to our recommendation on performing cost-benefit evaluations of sample sizes for reviews, SSA stressed that the Office of Management and Budget (OMB) reviews and approves all survey methodologies before they are undertaken by SSA. Further, SSA states that often a larger sample size is needed to provide breakdowns of responses by various subgroups.

In response to our recommendation that SSA institutionalize existing professional standards for conducting reviews, SSA acknowledged that, while these standards do exist, it will take steps to ensure greater visibility and understanding of the standards it employs.

OIG RESPONSE

Although both SSA and OMB review the statistical methods used for each customer service review, previous OIG reports and the report by the Customer Service Executive Team’s independent contractor have found that SSA’s sample sizes are often excessively large. We feel that the large size of some samples corresponds to the use of excessive resources. Accordingly, we continue to believe that SSA needs to ensure that it carefully evaluates the sample sizes of each review to ensure the most efficient use of resources.

OIG affirms its recommendation for establishing standards where there are currently none. The Agency should attempt to establish one set of standards for customer service reviews to help ensure the use of consistent standards and the production of valid and comparable results.

Back to top

INTRODUCTION

OBJECTIVE

The objective of this evaluation was to catalog SSA’s efforts to monitor its service to the public and to identify monitoring gaps, duplicative efforts, and opportunities for improving coordination and quality.

BACKGROUND

OIG is conducting a series of inspections to evaluate how SSA monitors service to the public. The first two reports, How The Private Sector Monitors Customer Satisfaction (Department of Health and Human Services, Office of Inspector General, OEI-02-94-00060), issued in April 1994, and Monitoring Social Security`s Customer Service (OEI-02-94-00780), issued in April 1995, were benchmarking exercises that described how private "world-class" organizations assessed customer satisfaction and the quality of customer services. Based on the findings of the first two reports, we have begun to look at SSA’s practices in monitoring service to the public and its use of comment cards to collect agency-level, customer-based data. The use of comment cards to collect customer-based data is described in a separate report, The Social Security Administration’s Use of the Customer Comment Card, A-02-96-02203, issued in April 1997.

Other reviews by OIG raised questions about SSA’s practices in conducting customer service reviews. OIG’s report, Monitoring Field Office Waiting Time (OEI-02-95-00110), issued in December 1995, concluded that a dramatic reduction in the sample size used in SSA’s ongoing field office (FO) waiting time study would achieve efficiency with a negligible loss in the precision of the study results. Another OIG report, Audit of the Office of Program and Integrity Reviews’ Special Studies (A-13-96-51142), issued in February 1997, also noted deficiencies in statistical practices. Additionally, the report of the consultant to the Customer Service Executive Team found that sample sizes for customer service reviews were excessively large.

SSA’s Customers

SSA has pledged to provide world-class service to its many different customers. Clearly, in terms of raw numbers of encounters and political sensitivity, the individual is SSA’s most identifiable customer. SSA serves the public through 1,300 FOs and 33 teleservice centers (TSC). FOs serve over 24 million visitors annually and answer an unknown number of telephone calls. SSA’s 800 number receives about 70 million calls annually.

SSA’s Performance Standards

Many of SSA’s components have customer service delivery performance standards. Some examples of these are access to the 800 number within 5 minutes of the customer initiating the call, and providing service to FO visitors within 10 minutes of their scheduled appointment. SSA’s performance standards are published in four documents--the Customer Service Pledge, the Social Security Strategic Plan, the Accountability Report, and the General Business Plan. Many of SSA’s components independently developed these standards and their uncoordinated efforts may have led to gaps in coverage or overlapping efforts. Currently, some of the Agency`s performance standards are not being monitored, while others have multiple monitoring efforts. Failure to coordinate efforts may have resulted in inefficient use of monitoring resources.

There are administrative and legislative directives that directly affect how SSA monitors its performance in serving the public. They are Executive Order 12862, "Setting Customer Service Standards," which is part of the National Performance Review, and the Government Performance and Results Act.

As noted above, SSA’s performance standards are published in a number of documents. While not contradictory nor inconsistent, these documents were developed at different times and for different purposes. The Social Security Strategic Plan, first published in September 1991 and updated in 1997, was created as part of the unified planning system to create a vision of SSA for the 21st century. The General Business Plan, also part of the unified planning process, was published in February 1995 as a centralized statement of SSA’s business processes and operational plans. The development of the Accountability Report was encouraged by the Chief Financial Officers Act. The Customer Service Pledge was developed in 1993 in response to President Clinton’s Executive Order 12862, "Setting Customer Service Standards."

The Customer Service Pledge has eight performance standards, as follows:

  • We will provide service through knowledgeable employees who will treat you with courtesy, dignity, and respect every time you do business with us.
  • We will provide you with our best estimate of the time needed to complete your request and fully explain any delays.
  • We will clearly explain our decisions so you can understand why and how we made them and what to do if you disagree.
  • We will make sure our offices are safe, pleasant, and our services accessible.
  • When you make an appointment we will serve you within 10 minutes of the scheduled time.
  • If you request a new or replacement Social Security card from one of our offices, we will mail it to you within 5 working days of receiving all information we need. If you have an urgent need for the Social Security number, we will tell you the number within 1 working day.
  • When you call our 800 number, you will get through within 5 minutes of your first try.

SSA has also identified the following standard that it realizes is important to customers but that cannot be met at the present time:

  • When you first apply for disability benefits, you will get a decision within 60 days.

The 1997 Social Security Strategic Plan contains five service-delivery goals.

  • To promote valued, strong, and responsive Social Security programs through effective policy development, research and program evaluation.
  • To deliver customer-responsive world-class service.
  • To make SSA program management the best in business, with zero tolerance for fraud, abuse, and mismanagement.
  • To be an employer that values and invests in each employee.
  • To strengthen public understanding of the Social Security programs.

The General Business Plan has a section on "Service Standards & Performance Measures," but references only the eight standards in the customer service pledge in this section. It further references seven core business services and six service-delivery interfaces in other sections of the Plan. The Plan states that ". . . all existing measures of service delivery and overall Agency performance are being re-examined from the customer satisfaction perspective." This review will lead to the development of a revised performance management system for the Agency. Similarly, the Accountability Report lists performance measures both in the Overview and in a chapter on Program and Financial Performance Measures. This report contains the Agency’s financial statement and is updated annually.

In May 1997, SSA Executive Staff completed a series of meetings to revise the Agency’s Strategic Plan (ASP). The revised ASP has five new goals. Because of the revised ASP, the performance plan published in the FY 1998-2002 Business Plan is also being revised.

Back to top

PRELIMINARY REVIEW

Based upon a review of the use of SSA`s FY 1995 generic clearances required by the Paperwork Reduction Act, we identified 11 different offices that are collecting customer-based data. Most are conducting ongoing monitoring studies. (See Table 1.)

Table 1

SSA Components Currently Involved
in Monitoring Customer Service

 OFFICE

ACRONYM
FREQUENCY
PURPOSE
Office of the Commissioner
OC
Ongoing
Planning
Office of Communications
OCOMM
One Time
Monitoring
Office of Central Records Operations
OCRO
Ongoing
Monitoring
Office of Disability
OD
One Time
Planning
Office of the Inspector General
OIG
Ongoing
Evaluating
Office of International Policy
OIP
One Time
Monitoring
Office of Policy
OP
One Time
Monitoring
Office of Program and Integrity Reviews
OPIR
Ongoing
Monitoring
Office of Public Service, Operations and Support
OPSOS
Ongoing
Monitoring
Office of Research and Statistics
ORS
Ongoing
Monitoring
Office of Workforce Analysis
OWA
Ongoing
Monitoring

Because of the number of components conducting reviews, we are concerned that SSA may not be applying uniform evaluation standards in all of its research/ evaluation components.

METHODOLOGY

In order to identify all of the customer service reviews that SSA components conducted in FY 1996, inquiries were sent to all Deputy Commissioners. They were asked if any customer service reviews were conducted by their components and, if so, the purpose, cost, and frequency of each review. We received responses from every Deputy Commissioner. We then contacted each component identified to obtain more specific information, such as the professional backgrounds of the reviewers, the source of the request for the review, and the controls in place to ensure quality reviews.

Our review was conducted from April 1996 to February 1997. This evaluation was performed in accordance with the Quality Standards for Inspections issued by the President’s Council on Integrity and Efficiency.

Back to top

FINDINGS

SSA CONDUCTED A SIGNIFICANT NUMBER OF CUSTOMER SERVICE REVIEWS

Twenty-one reviews of customer service were performed by SSA in FY 1996. The methodologies used included focus groups, telephone or mail surveys, and observational studies. These reviews looked at telephone service (both local and 800 numbers), the application and disability determination processes, FO waiting times, customer and employer satisfaction, and electronic service delivery. It should be noted that, in addition to the reviews, SSA uses operational data to augment its customer service information.

Five components performed the reviews, with two of the components responsible for almost three quarters of the reviews. The Office of Program and Integrity Reviews was the predominant reviewer with nine, the Office of Communications was next with seven, followed by OIG with three, and the Office of Human Resources and the Office of Operations with one each.

The majority of requests for reviews came from within SSA. Aside from three studies requested by Congress, requests came from the Commissioner, the Chief Policy Officer (now the Senior Advisor to the Commissioner on Customer Service Integration), the Senior Financial Executive, and various Deputy Commissioners. Most (14) were considered 1-time reviews, while 7 are ongoing.

THE REVIEWS LACKED UNIFORM STANDARDS

While most respondents reported having controls or procedures in place to ensure quality reviews, only one used externally-developed procedures, such as those provided by the General Accounting Office (GAO), OMB, or a professional society. The majority of respondents used internally-developed controls, such as reviewing all questionnaires and data entry to ensure accuracy. There was a lack of uniform standards and those used were developed by individual components to best meet their own needs. Additionally, one study conducted by individual FOs was decentralized; therefore, there was no uniform quality control in place. Only OIG used external controls. Its evaluations are guided by the President’s Council on Integrity and Efficiency Inspection Standards and its audits are guided by the GAO Government Auditing Standards.

THE TRAINING AND PROFESSIONAL BACKGROUNDS OF REVIEWERS VARIED

The professional backgrounds of SSA`s reviewers varied greatly. Nearly all were social insurance specialists and a few were management analysts. The position description for social insurance specialists indicates that they are responsible for analyzing and reporting on the quality of Retirement and Survivors Insurance operations and nonmedical aspects of Disability Insurance operations and/or Supplemental Security Income operations. Their tasks include: conducting beneficiary interviews; analyzing quality review data; identifying weaknesses in the process; and preparing reports on regional, area, district office, program service center, and TSC effectiveness.

The position description for management analysts indicates that they are responsible for measuring the effectiveness of Agency legislation, policies, and procedures; assessing achievement of program goals; evaluating the economy and efficiency of operations; and identifying vulnerabilities to fraud, waste, and abuse in a selected program or issue area. Their tasks include: planning extensive pre-inspection research; coordinating the development and training of team members; offering direction to team members conducting on-site visits; reviewing, analyzing, and evaluating quantitative and narrative data submitted by the team; and developing a final report incorporating outside comments.

Most of the employees conducting the reviews have received general on-the-job training in the various aspects of reviews. This training included, but was not limited to, survey methods, report writing, statistics, graphic design, interviewing techniques, and questionnaire construction. We were not able to determine whether the training received by the reviewers would equate to the professional standards indicated for these reviews.

THERE WAS NO FORMAL PROCESS FOR COORDINATION OF REVIEWS WITHIN SSA

Customer service reviews are done independently by the SSA components identified previously and there is no reported coordination of these efforts. In fact, most respondents were not aware of any efforts within SSA to coordinate its various customer service reviews. However, three respondents reported that the recent creation of an executive team within SSA may address uncoordinated monitoring efforts. This team, the Customer Service Executive Team, was formed to update SSA’s customer service standards and performance measures, and to support its strategic planning. The team developed a Statement of Work that SSA used to obtain the assistance of an outside contractor to provide an independent assessment of SSA`s current input program and current standards and performance measures, and to make suggestions for improvements. The contractor submitted a report of its findings in April 1997.

There is another SSA initiative which may enhance coordination of customer service reviews. In February 1997, SSA changed the title of its Chief Policy Officer to "Senior Advisor to the Commissioner on Customer Service Integration." A responsibility of this position is to serve as the focal point in SSA for coordinating customer service initiatives.

THE REPORTED STAFF HOURS REQUIRED TO COMPLETE REVIEWS WITHIN SSA VARIED GREATLY

The staff hours required to complete a review within SSA varied greatly. The range was from a low of 20 staff hours to a high of 14,600 staff hours. The median number of hours required to complete a review was approximately 1,000 staff hours. (See Table 2.)

Table 2

Staff Hours by Review in FY 1996

TITLE OF REVIEW
STAFF HOURS
COMPONENT
800# Evaluation - Caller Recontact
14,600
OPIR
Field Office Wait Time
7,200
OHR
Initial Awardees Satisfaction
6,700
OPIR
Initial Denials Satisfaction
5,200
OPIR
Redetermination Satisfaction
5,200
OPIR
Automated Service - "Perfect Access"
3,550
OPIR
Employer Satisfaction
2,500
OPIR
SI PEBES
2,080
OPIR
Local Satisfaction Surveys
1,300
OPERATIONS
12th Annual Customer Satisfaction
700
OIG
Kiosk Evaluation
500
OPIR
Benchmarking Notices
500
OIG
Test of Satisfaction Scales
380
OIG
System-to-System Access
220
OCOMM
Disability Packets
200
OCOMM
Public Information Material
180
OCOMM
Provider Packets
150
OCOMM
Payment Cycling II
150
OCOMM
Posters for Deaf and Hard of Hearing
20
OCOMM
Review Team Focus Groups
N/A
OCOMM

Back to top

CONCLUSION AND RECOMMENDATIONS

While SSA has numerous initiatives to monitor its service to the public, these initiatives are not coordinated and some lack the rigor of professional standards. Additionally, two prior OIG reviews and the Customer Service Executive Team’s independent contractor found that SSA’s sample sizes are often excessively large. Given the extent of resources committed to these initiatives, SSA should:

  • Develop a coordination mechanism to ensure that review resources are efficiently used (this is consistent with the recommendations of the Customer Service Executive Team’s contractor).
  • Perform cost-benefit evaluations of the sample sizes for customer service reviews conducted by the Agency in order to limit the staff hours required to perform those reviews.
  • Institutionalize existing Agency-wide professional standards for conducting customer service reviews.
  • Develop and publicize more consistent training requirements for staff assigned to conduct customer service reviews in order to ensure the integrity of the reviews.

AGENCY COMMENTS

SSA acknowledges the need to better coordinate and integrate the Agency’s service monitoring activities, and has taken steps to meet this challenge. Earlier this year, SSA retitled the Chief Policy Officer position to the "Senior Advisor to the Commissioner on Customer Service Integration." This was done to reinforce the position’s role as principal contact and coordinator on both reinvention and customer service activities. Additionally, the Agency’s Customer Service Executive Team was reconvened to develop an integrated market measurement system.

In response to our recommendation on performing cost/benefit evaluations of sample sizes for reviews, SSA stressed that its survey process does include evaluations of Agency benefit and that OMB reviews and approves all survey methodologies before they are undertaken. Further, SSA stated that often a larger sample size is needed to provide breakdowns of responses by various subgroups.

In response to our recommendation that SSA institutionalize existing professional standards for conducting reviews, SSA acknowledged that while these standards do exist it needs to take steps to ensure greater visibility and understanding. Also, SSA believes that the training requirements for the surveys conducted by the Agency are adequate. The full text of SSA’s comments, except for technical comments, is provided in Appendix A.

OIG RESPONSE

Although both SSA and OMB review the statistical methods used for each customer review, previous OIG reports and the report by the Customer Service Executive Team’s independent contractor have found that SSA’s sample sizes are often excessively large. We feel that the large size of some samples corresponds to the use of excessive resources. Accordingly, we continue to believe that SSA needs to ensure that it carefully evaluates the sample sizes of each review to ensure the most efficient use of resources.

OIG affirms its recommendation for establishing standards where there are currently none. The Agency should attempt to establish one set of standards for customer service reviews to help ensure the use of consistent standards and the production of valid and comparable results. Additionally, the training provided to the reviewers should equate to the professional standards indicated for these reviews, rather than be limited to the general on-the-job training currently provided.

Back to top

APPENDIX

  APPENDIX B

 MAJOR CONTRIBUTORS TO THIS REPORT

Office of the Inspector General

E. Scott Patterson, Director, Evaluations and Technical Services
Timothy F. Nee, Acting Deputy Director
Tracey K. Rennie, Evaluator-in-Charge
Robert Blake, Senior Auditor
Evan Buckingham, Senior Evaluator

  Link to FirstGov.gov: U.S. Government portal Privacy Policy | Website Policies & Other Important Information | Site Map
Need Larger Text?
  Last reviewed or modified