The objective of this evaluation was to catalog the Social Security
Administration’s (SSA) efforts to monitor its service to the
public and to identify monitoring gaps, duplicative efforts, and
opportunities for improving coordination and quality.
BACKGROUND
One of SSA’s goals is to provide "world-class service." To
that end, numerous SSA components conduct customer service monitoring
reviews. Because we were unable to identify any catalogs or summaries
of all such reviews, we attempted to create a listing of reviews.
Additionally, we looked at factors that affect the quality of reviews,
such as review standards and staff training.
In order to identify all the customer service reviews that SSA components
conducted in Fiscal Year (FY) 1996, inquiries were sent to all Deputy
Commissioners. The Deputy Commissioners were asked if any customer
service reviews were conducted by their components and, if so, the
purpose, cost, and frequency of each review. We received a response
from each of the Deputy Commissioners. We then contacted the individual
components identified as being involved in customer service reviews
to obtain more specific information such as the professional background
of the reviewers, the source of the request for the review, and the
controls in place to ensure quality reviews.
FINDINGS
SSA Conducted a Significant Number of Customer Service Reviews
The Reviews Lacked Uniform Standards
The Training and Professional Backgrounds of Reviewers Varied
There Was No Formal Process for Coordination of Reviews Within
SSA
The Reported Staff Hours Required to Complete Reviews Within
SSA Varied Greatly
CONCLUSION AND RECOMMENDATIONS
While SSA has numerous initiatives to monitor its service to the
public, these initiatives are not coordinated and some lack the rigor
of professional standards. Additionally, two prior Office of the
Inspector General (OIG) reviews and SSA’s Customer Service Executive
Team’s independent contractor found that SSA’s sample sizes
are often excessively large. Given the extent of resources committed
to these initiatives, we recommend that SSA:
Develop a coordination mechanism to ensure that review resources
are efficiently used (this is consistent with the recommendations
of the Customer Service Executive Team’s contractor).
Perform cost-benefit evaluations of the sample sizes for customer
service reviews conducted by the Agency in order to limit the staff
hours required to perform those reviews.
Institutionalize existing Agency-wide professional standards for
conducting customer service reviews.
Develop and publicize consistent training requirements for staff
assigned to conduct customer service reviews in order to ensure
the integrity of the reviews.
AGENCY COMMENTS
SSA acknowledged the need to better coordinate and integrate the
Agency’s service monitoring activities, and has taken steps
to meet this challenge. Earlier this year, SSA retitled the Chief
Policy Officer position as the "Senior Advisor to the Commissioner
on Customer Service Integration." This was done to reinforce
the position’s role as principal contact and coordinator on
both reinvention and customer service activities. Additionally, the
Agency’s Customer Service Executive Team was reconvened to develop
an integrated market measurement system.
In response to our recommendation on performing cost-benefit evaluations
of sample sizes for reviews, SSA stressed that the Office of Management
and Budget (OMB) reviews and approves all survey methodologies before
they are undertaken by SSA. Further, SSA states that often a larger
sample size is needed to provide breakdowns of responses by various
subgroups.
In response to our recommendation that SSA institutionalize existing
professional standards for conducting reviews, SSA acknowledged that,
while these standards do exist, it will take steps to ensure greater
visibility and understanding of the standards it employs.
OIG RESPONSE
Although both SSA and OMB review the statistical methods used for
each customer service review, previous OIG reports and the report
by the Customer Service Executive Team’s independent contractor
have found that SSA’s sample sizes are often excessively large.
We feel that the large size of some samples corresponds to the use
of excessive resources. Accordingly, we continue to believe that
SSA needs to ensure that it carefully evaluates the sample sizes
of each review to ensure the most efficient use of resources.
OIG affirms its recommendation for establishing standards where
there are currently none. The Agency should attempt to establish
one set of standards for customer service reviews to help ensure
the use of consistent standards and the production of valid and comparable
results.
The objective of this evaluation was to catalog SSA’s efforts
to monitor its service to the public and to identify monitoring gaps,
duplicative efforts, and opportunities for improving coordination
and quality.
BACKGROUND
OIG is conducting a series of inspections to evaluate how SSA monitors
service to the public. The first two reports, How The Private
Sector Monitors Customer Satisfaction (Department of Health and
Human Services, Office of Inspector General, OEI-02-94-00060), issued
in April 1994, and Monitoring Social Security`s Customer Service (OEI-02-94-00780),
issued in April 1995, were benchmarking exercises that described
how private "world-class" organizations assessed customer
satisfaction and the quality of customer services. Based on the findings
of the first two reports, we have begun to look at SSA’s practices
in monitoring service to the public and its use of comment cards
to collect agency-level, customer-based data. The use of comment
cards to collect customer-based data is described in a separate report, The
Social Security Administration’s Use of the Customer Comment
Card, A-02-96-02203, issued in April 1997.
Other reviews by OIG raised questions about SSA’s practices
in conducting customer service reviews. OIG’s report, Monitoring
Field Office Waiting Time (OEI-02-95-00110), issued in December
1995, concluded that a dramatic reduction in the sample size used
in SSA’s ongoing field office (FO) waiting time study would
achieve efficiency with a negligible loss in the precision of the
study results. Another OIG report, Audit of the Office of Program
and Integrity Reviews’ Special Studies (A-13-96-51142),
issued in February 1997, also noted deficiencies in statistical practices.
Additionally, the report of the consultant to the Customer Service
Executive Team found that sample sizes for customer service reviews
were excessively large.
SSA’s Customers
SSA has pledged to provide world-class service to its many different
customers. Clearly, in terms of raw numbers of encounters and political
sensitivity, the individual is SSA’s most identifiable customer.
SSA serves the public through 1,300 FOs and 33 teleservice centers
(TSC). FOs serve over 24 million visitors annually and answer an
unknown number of telephone calls. SSA’s 800 number receives
about 70 million calls annually.
SSA’s Performance Standards
Many of SSA’s components have customer service delivery performance
standards. Some examples of these are access to the 800 number within
5 minutes of the customer initiating the call, and providing
service to FO visitors within 10 minutes of their scheduled appointment.
SSA’s performance standards are published in four documents--the Customer
Service Pledge, the Social Security Strategic Plan, the Accountability
Report, and the General Business Plan. Many of SSA’s
components independently developed these standards and their uncoordinated
efforts may have led to gaps in coverage or overlapping efforts.
Currently, some of the Agency`s performance standards are not
being monitored, while others have multiple monitoring efforts. Failure
to coordinate efforts may have resulted in inefficient use of monitoring
resources.
There are administrative and legislative directives that directly
affect how SSA monitors its performance in serving the public. They
are Executive Order 12862, "Setting Customer Service Standards," which
is part of the National Performance Review, and the Government Performance
and Results Act.
As noted above, SSA’s performance standards are published in
a number of documents. While not contradictory nor inconsistent,
these documents were developed at different times and for different
purposes. The Social Security Strategic Plan, first published
in September 1991 and updated in 1997, was created as part of the
unified planning system to create a vision of SSA for the 21st century.
The General Business Plan, also part of the unified planning
process, was published in February 1995 as a centralized statement
of SSA’s business processes and operational plans. The development
of the Accountability Report was encouraged by the Chief Financial
Officers Act. The Customer Service Pledge was developed in
1993 in response to President Clinton’s Executive Order 12862, "Setting
Customer Service Standards."
The Customer Service Pledge has eight performance standards,
as follows:
We will provide service through knowledgeable employees who will
treat you with courtesy, dignity, and respect every time you do
business with us.
We will provide you with our best estimate of the time needed
to complete your request and fully explain any delays.
We will clearly explain our decisions so you can understand why
and how we made them and what to do if you disagree.
We will make
sure our offices are safe, pleasant, and our services accessible.
When
you make an appointment we will serve you within 10 minutes of
the scheduled time.
If you request a new or replacement Social
Security card from one of our offices, we will mail it to you
within 5 working
days of receiving all information we need. If you have an urgent
need for the Social Security number, we will tell you the number
within 1 working day.
When you call our 800 number, you will
get through within 5 minutes
of your first try.
SSA has also identified the following standard that it realizes
is important to customers but that cannot be met at the present time:
When you first apply for disability benefits, you will get a decision
within 60 days.
The 1997 Social Security Strategic Plan contains five service-delivery
goals.
To promote valued, strong, and responsive Social Security programs
through effective policy development, research and program evaluation.
To deliver customer-responsive world-class service.
To make SSA program management the best in business, with zero
tolerance for fraud, abuse, and mismanagement.
To be an employer that values and invests in each employee.
To strengthen public understanding of the Social Security programs.
The General Business Plan has a section on "Service
Standards & Performance Measures," but references only the
eight standards in the customer service pledge in this section. It
further references seven core business services and six service-delivery
interfaces in other sections of the Plan. The Plan states that ".
. . all existing measures of service delivery and overall Agency
performance are being re-examined from the customer satisfaction
perspective." This review will lead to the development of a
revised performance management system for the Agency. Similarly,
the Accountability Report lists performance measures both
in the Overview and in a chapter on Program and Financial Performance
Measures. This report contains the Agency’s financial statement
and is updated annually.
In May 1997, SSA Executive Staff completed a series of meetings
to revise the Agency’s Strategic Plan (ASP). The revised ASP
has five new goals. Because of the revised ASP, the performance plan
published in the FY 1998-2002 Business Plan is also being revised.
Based upon a review of the use of SSA`s FY 1995 generic clearances
required by the Paperwork Reduction Act, we identified 11 different
offices that are collecting customer-based data. Most are conducting
ongoing monitoring studies. (See Table 1.)
Table 1
SSA Components Currently Involved
in Monitoring Customer Service
OFFICE
ACRONYM
FREQUENCY
PURPOSE
Office of the Commissioner
OC
Ongoing
Planning
Office of Communications
OCOMM
One Time
Monitoring
Office of Central Records Operations
OCRO
Ongoing
Monitoring
Office of Disability
OD
One Time
Planning
Office of the Inspector General
OIG
Ongoing
Evaluating
Office of International Policy
OIP
One Time
Monitoring
Office of Policy
OP
One Time
Monitoring
Office of Program and Integrity Reviews
OPIR
Ongoing
Monitoring
Office of Public Service, Operations and Support
OPSOS
Ongoing
Monitoring
Office of Research and Statistics
ORS
Ongoing
Monitoring
Office of Workforce Analysis
OWA
Ongoing
Monitoring
Because of the number of components conducting reviews, we are concerned
that SSA may not be applying uniform evaluation standards in all
of its research/ evaluation components.
METHODOLOGY
In order to identify all of the customer service reviews that SSA
components conducted in FY 1996, inquiries were sent to all Deputy
Commissioners. They were asked if any customer service reviews were
conducted by their components and, if so, the purpose, cost, and
frequency of each review. We received responses from every Deputy
Commissioner. We then contacted each component identified to obtain
more specific information, such as the professional backgrounds of
the reviewers, the source of the request for the review, and the
controls in place to ensure quality reviews.
Our review was conducted from April 1996 to February 1997. This
evaluation was performed in accordance with the Quality Standards
for Inspections issued by the President’s Council on Integrity
and Efficiency.
Twenty-one reviews of customer service were performed by SSA in
FY 1996. The methodologies used included focus groups, telephone
or mail surveys, and observational studies. These reviews looked
at telephone service (both local and 800 numbers), the application
and disability determination processes, FO waiting times, customer
and employer satisfaction, and electronic service delivery. It should
be noted that, in addition to the reviews, SSA uses operational data
to augment its customer service information.
Five components performed the reviews, with two of the components
responsible for almost three quarters of the reviews. The Office
of Program and Integrity Reviews was the predominant reviewer with
nine, the Office of Communications was next with seven, followed
by OIG with three, and the Office of Human Resources and the Office
of Operations with one each.
The majority of requests for reviews came from within SSA. Aside
from three studies requested by Congress, requests came from the
Commissioner, the Chief Policy Officer (now the Senior Advisor to
the Commissioner on Customer Service Integration), the Senior Financial
Executive, and various Deputy Commissioners. Most (14) were considered
1-time reviews, while 7 are ongoing.
While most respondents reported having controls or procedures in
place to ensure quality reviews, only one used externally-developed
procedures, such as those provided by the General Accounting Office
(GAO), OMB, or a professional society. The majority of respondents
used internally-developed controls, such as reviewing all questionnaires
and data entry to ensure accuracy. There was a lack of uniform standards
and those used were developed by individual components to best meet
their own needs. Additionally, one study conducted by individual
FOs was decentralized; therefore, there was no uniform quality control
in place. Only OIG used external controls. Its evaluations are guided
by the President’s Council on Integrity and Efficiency Inspection
Standards and its audits are guided by the GAO Government Auditing
Standards.
The professional backgrounds of SSA`s reviewers varied greatly.
Nearly all were social insurance specialists and a few were management
analysts. The position description for social insurance specialists
indicates that they are responsible for analyzing and reporting on
the quality of Retirement and Survivors Insurance operations and
nonmedical aspects of Disability Insurance operations and/or Supplemental
Security Income operations. Their tasks include: conducting beneficiary
interviews; analyzing quality review data; identifying weaknesses
in the process; and preparing reports on regional, area, district
office, program service center, and TSC effectiveness.
The position description for management analysts indicates that
they are responsible for measuring the effectiveness of Agency legislation,
policies, and procedures; assessing achievement of program goals;
evaluating the economy and efficiency of operations; and identifying
vulnerabilities to fraud, waste, and abuse in a selected program
or issue area. Their tasks include: planning extensive pre-inspection
research; coordinating the development and training of team members;
offering direction to team members conducting on-site visits; reviewing,
analyzing, and evaluating quantitative and narrative data submitted
by the team; and developing a final report incorporating outside
comments.
Most of the employees conducting the reviews have received general
on-the-job training in the various aspects of reviews. This training
included, but was not limited to, survey methods, report writing,
statistics, graphic design, interviewing techniques, and questionnaire
construction. We were not able to determine whether the training
received by the reviewers would equate to the professional standards
indicated for these reviews.
Customer service reviews are done independently by the SSA components
identified previously and there is no reported coordination of these
efforts. In fact, most respondents were not aware of any efforts
within SSA to coordinate its various customer service reviews. However,
three respondents reported that the recent creation of an executive
team within SSA may address uncoordinated monitoring efforts. This
team, the Customer Service Executive Team, was formed to update SSA’s
customer service standards and performance measures, and to support
its strategic planning. The team developed a Statement of Work that
SSA used to obtain the assistance of an outside contractor to provide
an independent assessment of SSA`s current input program and
current standards and performance measures, and to make suggestions
for improvements. The contractor submitted a report of its findings
in April 1997.
There is another SSA initiative which may enhance coordination of
customer service reviews. In February 1997, SSA changed the title
of its Chief Policy Officer to "Senior Advisor to the Commissioner
on Customer Service Integration." A responsibility of this position
is to serve as the focal point in SSA for coordinating customer service
initiatives.
The staff hours required to complete a review within SSA varied
greatly. The range was from a low of 20 staff hours to a high of
14,600 staff hours. The median number of hours required to complete
a review was approximately 1,000 staff hours. (See Table 2.)
While SSA has numerous initiatives to monitor its service to the
public, these initiatives are not coordinated and some lack the rigor
of professional standards. Additionally, two prior OIG reviews and
the Customer Service Executive Team’s independent contractor
found that SSA’s sample sizes are often excessively large. Given
the extent of resources committed to these initiatives, SSA should:
Develop a coordination mechanism to ensure that review resources
are efficiently used (this is consistent with the recommendations
of the Customer Service Executive Team’s contractor).
Perform cost-benefit evaluations of the sample sizes for customer
service reviews conducted by the Agency in order to limit the staff
hours required to perform those reviews.
Institutionalize existing Agency-wide professional standards for
conducting customer service reviews.
Develop and publicize more consistent training requirements for
staff assigned to conduct customer service reviews in order to
ensure the integrity of the reviews.
AGENCY COMMENTS
SSA acknowledges the need to better coordinate and integrate the
Agency’s service monitoring activities, and has taken steps
to meet this challenge. Earlier this year, SSA retitled the Chief
Policy Officer position to the "Senior Advisor to the Commissioner
on Customer Service Integration." This was done to reinforce
the position’s role as principal contact and coordinator on
both reinvention and customer service activities. Additionally, the
Agency’s Customer Service Executive Team was reconvened to develop
an integrated market measurement system.
In response to our recommendation on performing cost/benefit evaluations
of sample sizes for reviews, SSA stressed that its survey process
does include evaluations of Agency benefit and that OMB reviews and
approves all survey methodologies before they are undertaken. Further,
SSA stated that often a larger sample size is needed to provide breakdowns
of responses by various subgroups.
In response to our recommendation that SSA institutionalize existing
professional standards for conducting reviews, SSA acknowledged that
while these standards do exist it needs to take steps to ensure greater
visibility and understanding. Also, SSA believes that the training
requirements for the surveys conducted by the Agency are adequate.
The full text of SSA’s comments, except for technical comments,
is provided in Appendix A.
OIG RESPONSE
Although both SSA and OMB review the statistical methods used for
each customer review, previous OIG reports and the report by the
Customer Service Executive Team’s independent contractor have
found that SSA’s sample sizes are often excessively large. We
feel that the large size of some samples corresponds to the use of
excessive resources. Accordingly, we continue to believe that SSA
needs to ensure that it carefully evaluates the sample sizes of each
review to ensure the most efficient use of resources.
OIG affirms its recommendation for establishing standards where
there are currently none. The Agency should attempt to establish
one set of standards for customer service reviews to help ensure
the use of consistent standards and the production of valid and comparable
results. Additionally, the training provided to the reviewers should
equate to the professional standards indicated for these reviews,
rather than be limited to the general on-the-job training currently
provided.
E. Scott Patterson, Director, Evaluations and Technical Services
Timothy F. Nee, Acting Deputy Director
Tracey K. Rennie, Evaluator-in-Charge
Robert Blake, Senior Auditor
Evan Buckingham, Senior Evaluator
www.socialsecurity.gov
