A-05-08-18041 - Alternate Format

MEMORANDUM

Date: March 12, 2009

To: The Commissioner

From: Inspector General

Subject: Contract for the Benefit Offset National Demonstration Project with Abt Associates, Incorporated (A-05-08-18041)

The attached final report presents the results of our audit. Our objectives were to (1) review the services provided by Abt Associates, Incorporated, under Contract Number SS00-04-60110 and the related costs charged to the Social Security Administration for adherence to the negotiated contract terms and applicable regulations and (2) ensure the Agency received the goods and services for which it contracted.

Please provide within 60 days a corrective action plan that addresses each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.

Patrick P. O'Carroll, Jr.

OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

CONTRACT FOR
THE BENEFIT OFFSET
NATIONAL DEMONSTRATION PROJECT
WITH ABT ASSOCIATES, INCORPORATED

March 2009

A-05-08-18041

AUDIT REPORT

Mission

By conducting independent and objective audits, evaluations and investigations, we inspire public confidence in the integrity and security of SSA's programs and operations and protect them against fraud, waste and abuse. We provide timely, useful and reliable information and advice to Administration officials, Congress and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

We strive for continual improvement in SSA's programs, operations and management by proactively seeking new ways to prevent and deter fraud, waste and abuse. We commit to integrity and excellence by supporting an environment that provides a valuable public service while encouraging employee development and retention and fostering diversity and innovation.

Executive Summary
OBJECTIVE

The objectives of our audit were to (1) review the services provided by Abt Associates, Incorporated, (Abt) under Contract Number SS00-04-60110 and the related costs charged to the Social Security Administration (SSA) for adherence to the negotiated contract terms and applicable regulations and (2) ensure SSA received the goods and services for which it contracted.

BACKGROUND

SSA contracted with Abt to develop the Benefit Offset National Demonstration (BOND) project to test alternate methods of treating work activity in the Title II disability program. The contract states that after the design is completed, subsequent phases of this demonstration are to be awarded to the same contractor on a sole-source basis for the project's implementation, data collection, and evaluation and management. This award will be contingent upon successful performance of the design phase. A $2.4 million cost-plus-fixed-fee contract was awarded to Abt with a period of performance from September 2004 to September 2006. Because of contract modifications and additional task orders, the contract was extended to September 2008 at a total cost of approximately $10.6 million.

To inform the BOND design phase, SSA awarded four separate contracts for a total cost of $3.9 million in what was called the Four-State Pilot. The contracts were awarded to Connecticut, Utah, Vermont and Wisconsin. The Pilot was to be a 2-year demonstration project expected to end in April 2007. However, numerous contract modifications have extended the contract through April 2009 at an estimated cost of $8.4 million.

RESULTS OF REVIEW

We found that Abt generally adhered to the terms of the contract and delivered the services and final design options that SSA asked for under the contract. However, we have several concerns about the delays and costs associated with the design phase. While SSA is required by law to continue with the demonstration project, we are concerned that the multiple modifications extended the contract period from 2 to 4 years, and the design phase costs increased to $10.6 million, or $8.2 million more than initially expected. We believe approximately $5.3 million, or half of the total costs, could have been put to better use had the contract been better focused and completed within the initial timeframe. The delay of the BOND design phase also led to additional costs under the Four-State Pilot, or $4.5 million more than initially expected. As a result, during our audit period, the obligated cost for BOND and the Four-State Pilot was approximately $19 million, considerably more than the planned combined costs of $6.3 million.

Contract Initial Design Costs1
Obligated Costs1 Increase
(Percent)2
BOND $2.4 million $10.6 million 342
Four-State Pilot $3.9 million $8.4 million 115
Totals $6.3 million $19.0 million 202
Note 1: Design costs also include pre-implementation costs added by SSA management.
Note 2: Percent increases do not total since each calculation is specific to each set of costs.

Prior SSA management demonstrated inadequate oversight of the contract's planning, scope and expenditures. While current SSA management attempted to streamline the BOND project's design, financial monitoring of the project was inadequate. For instance, while the Office of Management and Budget requires enhanced controls over cost-reimbursement contracts, in our review of invoices, we were unable to determine whether the contractor over- or under-billed for a specific task due. Furthermore, the contract was not monitored in a way that allowed for quickly detecting or avoiding cost overruns for tasks. Moreover, we found that SSA did not perform timely interim contractor performance evaluations, as required by the contract and recommended by the Federal Acquisition Regulation (FAR).

In terms of contract coordination, the first BOND design was submitted to SSA management before the Four-State Pilot participants formally presented their feedback to the contractor. We believe that earlier communication between BOND and the Four-State Pilot regarding the project's design would have enhanced the Pilot's usefulness to the BOND contractor. Finally, we found that the BOND project's current sole-source contract lacks clear separation of duties by allowing the contractor to evaluate its own performance.

RECOMMENDATIONS

To enhance oversight of the BOND project and future contracts of a similar nature, we recommend that SSA:

1. Before awarding the next phase of BOND, ensure a detailed cost-benefit analysis of the BOND design phase has been conducted and documented to support an Agency decision to continue with this contractor. As a part of the analysis, the Agency should have (a) reviewed the current status of the project, (b) assessed what value the Agency received for the costs already incurred, (c) determined whether the entire BOND project can be completed at a reasonable cost, and (d) documented the Agency's justification for the continued use of the current sole-source contractor as well as the type of contract being used.

2. Establish a greater degree of management oversight by strengthening contract oversight roles and responsibilities and more clearly defining contract requirements.

3. Improve financial monitoring by implementing the procedures necessary to (a) identify the work for which the contractor is requesting payment and (b) maintain documentation to prove to an outside party that value was received for the money expended.

4. Conduct and document interim and final evaluations of the contractor's performance in accordance with the contract and as recommended by the FAR.

5. Ensure timely and meaningful participation occurs with other parties associated with the demonstration project, such as the States involved in the Four-State Pilot, to enhance the total project.

6. Ensure the evaluation phase of the BOND contract is conducted by an independent party.

AGENCY COMMENTS

With the exception of Recommendations 3 and 6, SSA generally agreed with our report and recommendations. SSA stated it had made changes to improve the overall management of the project as it moves toward implementation. Regarding Recommendation 3, SSA disagreed with the wording of this recommendation and believes it refers to its ability to monitor the work for which the contractor requested payment, not just recordkeeping. SSA stated that different contractors report and bill differently. SSA noted that, in this contract, the combination of the monthly invoices, the monthly progress reports, and the close communication between the Contract Officer Technical Representative and the contractor's project director provided enough information to determine that the contractor was billing correctly for each task. Regarding Recommendation 6, SSA disagreed with our recommendation and believes any future contract, with appropriate built-in safeguards, can provide adequate mitigation against conflicts of interest. SSA further indicated that the FAR allows the Agency to avoid, neutralize, or mitigate conflicts of interest. SSA believes it has appropriate mitigation strategies. See Appendix H of this report for SSA's full comments.

OIG RESPONSE

We commend SSA for the actions already taken to improve oversight of the BOND contract. We continue to believe Recommendation 3 would enhance the level of financial monitoring and minimize the risk that a contractor is over- or under-billing for specific tasks. The current financial monitoring process places too much reliance on the contractor and does not allow for independent reconciliation of costs back to the contract tasks and deliverables. We also believe Recommendation 6, regarding the involvement of an independent party during the evaluation phase, would ensure a clear separation of duties. Assigning the evaluation phase to the same contractor performing the design and implementation work could potentially compromise an independent assessment of the BOND project in appearance or fact. In a time of increased scrutiny of taxpayer funds and calls for greater oversight of Federal programs and contracts, we believe the Agency needs to adopt stronger controls over a contract estimated to cost taxpayers approximately $100 million. In the report, we also cite FAR warnings about the risks of cost-plus-fixed-fee contracts as well as the Office of Management and Budget's requirements for greater management oversight when contracts involve inherently governmental functions. Given these concerns, we believe the Agency should take extra steps to ensure an independent evaluation of the BOND project.

Table of Contents
Page
INTRODUCTION 1
RESULTS OF REVIEW 3
Status of the Design Contract 3
Design Contract Costs 3
Design Contract Timeliness 3
Design Contract Tasks and Design Modifications 4
Cost of Full Implementation 7
Evaluation Phase 7
Contract Costs, Deliverables and Oversight 8
Role of the Four-State Pilot 11
Management of the Bond Design Contract 12
CONCLUSIONS AND RECOMMENDATIONS 14

APPENDICES

APPENDIX A - Acronyms
APPENDIX B - Background on the Benefit Offset National Demonstration Design
APPENDIX C - Contracting Duties
APPENDIX D - Scope and Methodology
APPENDIX E - Benefit Offset National Demonstration Contract Timeline
APPENDIX F - Benefit Offset National Demonstration Contract Modifications
APPENDIX G - Benefit Offset National Demonstration Tasks and Deliverables
APPENDIX H - Agency Comments
APPENDIX I - OIG Contacts and Staff Acknowledgments

Introduction
OBJECTIVE

The objectives of our audit were to (1) review the services provided by Abt Associates, Incorporated, (Abt) under Contract Number SS00-04-60110 and the related costs charged to the Social Security Administration (SSA) for adherence to the negotiated contract terms and applicable regulations and (2) ensure that SSA received the goods and services for which it contracted.

BACKGROUND

SSA contracted with Abt to develop the multi-site Benefit Offset National Demonstration (BOND) project that tests alternate methods of treating work activity in the Title II disability program. The contract states that after the design is complete, subsequent phases of this demonstration are to be awarded to the same contractor on a sole-source basis for the implementation, data collection and evaluation and management of the project. This award will be contingent upon successful performance of the design phase. The cost-plus-fixed-fee contract was awarded for $2.4 million, with a period of performance from September 30, 2004 to September 29, 2006. With contract modifications and additional task orders, the total contract award was approximately $10.6 million and extended to September 29, 2008.

SSA selected Abt to assist with the design of all phases of the BOND project. The demonstration was to test the effects of replacing the current benefit reduction facing Disability Insurance (DI) beneficiaries who return to work with a more gradual $1 reduction in benefits for every $2 in earnings (the benefit offset). According to the contract, this benefit offset was to be tested alone and in combination with employment support models intended to provide further assistance to DI beneficiaries returning to work. The estimated impact of the benefit offset was to be applicable nationwide and should represent the collective effects of the benefit offset on a wide range of local circumstances and groups.

To inform the BOND design phase, SSA awarded four separate contracts for a total cost of $3.9 million as the Four-State Pilot. The contracts were awarded to Connecticut, Utah, Vermont and Wisconsin. The Pilot was to be a 2-year demonstration project from April 2005 to April 2007. However, numerous contract modifications have extended the contract through April 2009 at an estimated cost of $8.4 million.

Our audit focused on how effectively and efficiently SSA and the contractor managed the contract. As a part of this audit, we interviewed SSA and Abt staff associated with the contract, interviewed State representatives from the associated Four-State Pilot, and reviewed the contract and related documentation (vendor invoices and task and delivery order information) supporting payments to Abt covering the period from September 2004 through January 2008 (see Appendix D for our full methodology).

Results of Review

The contractor generally adhered to the terms of the contract and delivered the services and final design options that SSA requested under the contract. However, multiple modifications extended the contract period from 2 to 4 years, and obligated costs for the design phase increased to $10.6 million, or $8.2 million more than initially expected. In addition, the delay of the BOND design phase led to additional costs under the Four-State Pilot, or $4.5 million more than initially expected. In terms of contract management, prior SSA management demonstrated inadequate oversight of the contract's planning, scope and expenditures. And while current SSA management attempted to streamline the BOND project's design, financial monitoring of the project was inadequate. Furthermore, prior and current SSA management did not monitor the contract in a way that allowed for quickly detecting or avoiding cost overruns for tasks. Moreover, we found that SSA management had not performed timely contractor performance evaluations, as required by the contract and recommended by the FAR. In terms of contract coordination, we believe that earlier communication between BOND and the Four-State Pilot regarding the project's design would have enhanced the Pilot's usefulness to the BOND contractor. Finally, we found that the sole-source contract lacked clear separation of duties by allowing the contractor to evaluate its own performance.

STATUS OF THE DESIGN CONTRACT

We considered a number of factors in assessing the status of the design phase contract, including (1) contract costs, (2) contract timeliness, (3) contract tasks and modifications, (4) full implementation costs, and (5) plans for the evaluation phase of the contract.

Design Contract Costs

The BOND project's design phase contract costs had increased significantly. While the initial contract was awarded in September 2004 for approximately $2.4 million, later modifications increased obligated costs by approximately 340 percent to about $10.6 million. Hence, the contract exceeded the initial estimate by at least $8.2 million.

Design Contract Timeliness

The contract was awarded on September 30, 2004 with a projected completion date of September 29, 2006. The final contract modification added new tasks and deliverables and extended the contract period of performance through September 29, 2008. As a result, the design contract was completed 2 years after the planned completion date.

Design Contract Tasks and Design Modifications

The number of tasks under the BOND design contract had tripled since the initial contract, although SSA management later eliminated some of the added tasks. The September 2004 contract included eight tasks and associated deliverables. As of August 2008, the contract had undergone major changes resulting in 16 additional tasks as well as new costs associated with project management. These additional contract modifications occurred at a number of points in the process, including
(1) before the due date of the initial contract, (2) after the contract moved into implementation, and (3) after later management adjustments to the original design.

Modifications Before Contract Due Date

SSA added two new pieces to the design contract before September 2006, when the contract was to be completed. One modification, added in September 2005, related to early intervention with pilot participants. The second modification, added in August 2006, added health benefits to the design and extended the contract period to December 2006. Both of these modifications, if approved by management as part of the final design, would have significantly increased the cost of implementing BOND.

Modifications to Move into Implementation

In January 2007, a final design and a walk-through of the full design was provided to SSA's managers. As a result of the design completion, a new modification covering pre-implementation tasks was funded in January 2007. Under Modification #7, SSA management directed the contractor to perform a series of pre-implementation tasks to provide a bridge between the design work they had performed and the implementation tasks they will perform once the demonstration is awarded. The new tasks were to lay the groundwork for various implementation activities including

incorporating One-Stop Service Centers into the demonstration;
developing Work Support Organizations (WSO);
securing the services of a fiscal agent to administer the independence accounts;
developing the data warehouse and benefit calculator;
developing public information and outreach material; and
developing a data security plan for the project that was to be approved and in place before the recruitment of project participants.

Although the need for pre-implementation work was anticipated at the time of the original contract, steps were not taken to identify the costs and tasks before the decision to proceed to the implementation phase. To illustrate, below is an excerpt from the contract officer's January 5, 2007 award letter for Modification #7.

The (contract) statement of work states "should SSA decide to proceed with the next phase, the contractor may also be required to integrate the transition from the design phase of the contract to the implementation and evaluation phases". As currently written, the statement of work does not specify all of the tasks needed for the pre-implementation stage. Therefore, the new modification is necessary to provide funding to add Pre-Implementation Tasks to the contract. This modification adds new tasks (10-18) and deliverables into the statement of work and extends the period of performance for the new deliverables through June 30, 2007.

The Project Officers' Contracting Handbook warns "If the statement of work does not state exactly what is wanted, or does not state it precisely, it will generate contract administration problems for both the project officer and the contract officer. Ambiguous statements of work often result in unsatisfactory contractor performance, delays, disputes, and higher contract costs."

Later Modification by Management

In February 2007, management changes at SSA led to further modification of the design contract. For example, health benefits, early intervention, and WSO pieces were eliminated to streamline the design. In addition, the target population for BOND was expanded from Title II-only beneficiaries to concurrent beneficiaries. Below is an excerpt from an October 11, 2007 SSA project officer email concerning Modification #13.

We have stripped the project down to a benefit offset and possibly some additional changes to SSA's rules. That means that not only will we not be providing health insurance, we will not be providing any of the additional work supports that constituted much of the previous design.…In addition, we have informed [the contractor] that the project now will include not just [Title II] disability beneficiaries but concurrent beneficiaries as well, which is a significant change.

In our December 2007 meeting with SSA and BOND contractor officials, participants attributed these changes in contract scope to two factors: (1) SSA management turnover and (2) a task later becoming insignificant or too complex. At the same meeting, contract staff stated that 14 percent of total project resources, or $938,198, had been allocated to "pre-implementation planning tasks." The pre-implementation deliverables associated with these tasks included a number of planning documents and detailed specifications, draft recruitment materials and Memorandums of Understanding, findings from focus groups, a public information strategy and initial public information materials for potential demonstration participants. At the same December 2007 meeting, contractor staff stated that because of changes in the model in January 2007, many of the deliverables associated with the $938,198 could not be used. As a result, the decision by SSA's previous management to fund implementation activities before final design approval (and formal initiation of the implementation phase contract) led to unnecessary expenditures and increased project costs.

Cost of Full Implementation

Over the life of the design phase, the estimated cost of full implementation of the BOND project has varied greatly, though management attempted to reduce its complexity and overall costs, beginning in the spring of 2007. While SSA's project officer indicated that the latest estimate was $100 million, a January 2007 contractor estimate provided a range of costs that went as high as $1.6 billion. In April 2007, SSA and the contractor modified the project design and estimated implementation costs to range from approximately $242 to $309 million. By May 2007, the estimates were reduced further and ranged from $82 to $198 million. The implementation cost estimates have varied widely because of the numerous modifications that were added and then deleted, such as health benefits, early intervention, and work support.

Evaluation Phase

The contract states, "After the design is completed, subsequent phases of this demonstration will be awarded to the same contractor on a sole-source basis for the implementation, data collection and evaluation and management of the project." This evaluation will occur both during and after the implementation phase and assess the project's outcomes and implementation process.

Although current management favors the award of the implementation and evaluation phases to the current contractor on a sole-source basis, we are concerned that this model lacks clear separation of duties and may introduce a conflict of interest by not engaging an independent party to evaluate the BOND project. The FAR also states that contracts should prevent the existence of conflicting roles that might bias a contractor's judgment and prevent unfair competitive advantage. This evaluation piece represents an area where open competition could produce a more reliable evaluation report. At a minimum, we believe a full and independent evaluation of the process needs to be performed by a party independent of the contractor. In December 2007, we shared our concerns about the model with SSA management. In an April 14, 2008 memorandum to the Commissioner, the Deputy Commissioner for Retirement and Disability Policy recommended that the evaluation duties remain with the current contractor on a sole-source basis. In the memorandum SSA noted:

OIG concern on this issue is understandable. It is critical that the evaluation of this important project produces objective and credible results. However, we believe that changing the contract structure at this point in time would result in risk to the successful completion of the project as well as potential financial risk to the Agency….

CONTRACT COSTS, DELIVERABLES AND OVERSIGHT

Our review of the contract's approximately $5 million in expenditures from September 2004 through January 2008 found the costs generally adhered to the negotiated contract terms and applicable regulations and were approved by appropriate SSA staff, although we could not always link the expenditure to the specific contract task. During this period, the contractor expended approximately $5 million in funds to support work associated with the BOND design. The project officer and the contract officer had approved invoices associated with the approximately $5 million in expenditures through December 2008 (see Table 1).

Table 1: BOND Contract Costs
(as of December 2007)
Contract Number SS00-04-60110
Type of Services Provided Amount
Direct Labor Costs $ 926,438
Fringe Benefits $ 389,097
Overhead $ 613,193
Other Direct Labor:
Communications and Information
Technology
$ 83,282
Other Costs $ 8,974
Printing $ 4,503
Purchased Services $ 21,248
Travel $ 34,517
Consultant Costs $ 126,782
Subcontractor $ 1,900,122
Handling Charges $ 73,105
General and Administrative Costs $ 448,208
Fixed Fee $ 370,358
TOTAL $ 4,999,827

Identifying Deliverables

We found that the contractor services provided and the costs charged to SSA generally adhered to the terms of the contract, and that SSA received the goods and services for which it contracted based on the project officer's acceptance of the contractor's deliverables. However, during our review of the invoices, we were unable to identify most of the contract tasks and deliverables associated with the costs. For example, Modification #7 extended the contract period to June 30, 2007, increased the contract by $2,081,570, and added pre-implementation tasks. However, we were unable to determine which invoice(s) were associated with the tasks funded by this modification.

Our discussions with the project officer and the contract officer confirmed that the current contract does not price the tasks on a per-line basis. The project officer indicated that invoices associated with cost-reimbursement contracts are not required to identify the contract tasks and deliverables associated with the costs. Instead, the project officer and contract officer rely on the contractor's budget information, which is separated by task in progress reports. However, without maintaining its own records regarding tasks and deliverables, or at least requiring more specific information on the submitted invoices, SSA lacks full assurance that the contractor did not over- or under-bill for a specific task. This is even more of an issue with this particular cost-reimbursement contract given the multiple modifications that added and subtracted tasks and deliverables.

Performance Reports

The FAR recommends interim evaluations of contractor performance. The FAR also requires final evaluations of contractor performance. The FAR further states that the performance evaluation and report must be shared with the contractor, who has an opportunity to respond before the contract officer finalizes the performance report. While an interim evaluation of the contractor's performance was to be conducted under the contract, our audit found that the project officer and contract officer had not evaluated the contractor's performance after the 2-year period when the project was initially set to end nor at the time of our review, approximately 4 years into the contract.

In September 2008, SSA's contract officer completed the interim evaluation on the design contract. The final evaluation on the design contract was conducted and completed soon after. The contract officer stated that no plans had been made to conduct an interim evaluation of the contractor's performance because of the delays and changes made to the BOND contract. Although we realize there were delays with the design, we believe the contractor's performance could have been evaluated earlier.

The failure to perform interim and final evaluations of the contractor's performance increases the risk that potential performance problems will remain unnoticed and impact future phases of the project.

ROLE OF THE FOUR-STATE PILOT

In April 2005, SSA awarded $3.9 million for a Four-State Pilot involving Connecticut, Utah, Vermont and Wisconsin. The demonstration was to test the effects of replacing the current benefit reduction facing DI beneficiaries who return to work with a more gradual $1 reduction in benefits for every $2 in earnings (the benefit offset). This Four-State Pilot was to be a 2-year demonstration project to assist with the BOND design phase and was expected to end in March or April 2007. However, numerous contract modifications have extended the contract through April 2009 at an estimated cost of $8.4 million, or $4.5 million more than initially expected. This extension was in tandem with the extension of the BOND design phase, allowing the four States to continue developing their evaluation data and provide technical advice to SSA and the contractor.

We believe the Four-State Pilot represented a useful model for similar projects, though earlier communications with the States regarding the project's design would have enhanced its usefulness to SSA. In March 2007, after the final BOND design had been submitted to former SSA management, SSA and the BOND contractor held a conference with the Four-State Pilot managers to identify and analyze the most significant lessons learned by the four States. The conference participants provided input on the BOND design and discussed potential pitfalls. For example, some State participants counseled against too much complexity in the early intervention piece of the implementation. We interviewed officials from the States involved in the project in February 2008. One of the officials we interviewed indicated that if the States had met with SSA and the BOND contractor sooner, less time would have been spent on a design that was not practical because of its complexity. In addition, our review of the contract documents show limited interaction between the BOND contractor and the four States.

MANAGEMENT OF THE BOND DESIGN CONTRACT

As noted earlier, the BOND design phase contract, as well as the associated Four-State Pilot, has experienced significant delays and cost increases since the start of each project (see Table 2). The numerous modifications to the BOND project indicate that current and former SSA management has been indecisive regarding the requirements and costs of this contract. The expanded tasks and numerous modifications (including modifications of modifications) to the BOND contract indicate either a lack of planning or a lack of direction during the design phase. In addition, the current cost reimbursement sole-source contract does not encourage either cost containment or timely delivery since (1) all costs are ultimately reimbursed by SSA and (2) the same contractor is implementing its own design. Finally, assigning the evaluation phase to the same contractor performing the design and implementation work does not ensure an independent review of the BOND project and its results.

Table 2: History of Costs Associated with BOND and the Four-State Pilot
(as of June 2008)

Office of Management and Budget (OMB) Policy Letter 93-1 offers guidance to ensure that good management practices are followed. This Policy Letter establishes Government-wide policy, assigns responsibilities and provides guiding principles for Executive Departments and agencies in managing the acquisition and use of services. In terms of management oversight of service contracts, the letter states "Agency officials need to be able to make sound judgments on what the requirements should be, the estimated costs, and whether the contractor is performing according to the contract terms and conditions." The OMB Policy Letter further notes that "Agency officials must also provide an enhanced degree of management controls and oversight when contracting for functions that closely support the performance of inherently Governmental functions."

We acknowledge that SSA underwent a change in management during this contract, and many of the issues discussed in this review relate to prior management. For example, most of the major additions to the design (and overall cost of implementation), such as health benefits, early intervention, and work support organization activities, were added before the involvement of current management. Moreover, the premature expenditures for pre-implementation activities came at a time when the prior management had approved the design. Finally, the meeting between the BOND contractor and the Four-State Pilot participants should have occurred before the arrival of new management since this would have enhanced the final design submitted in January 2007.

With this said, we still have concerns about recent changes. After the change in management, the BOND design's scope was expanded to include concurrent beneficiaries, which was a significant change from the earlier model. Moreover, the design phase was extended and completed as of September 29, 2008. Further, the evaluation phase of the BOND project was still part of the sole-source contract at the time of our review and SSA management has not decided to change this structure.

Though SSA estimated the design phase would have been completed in September 2006 at a cost of $2.4 million, as we noted earlier, the current obligated costs associated with the design phase are now approximately $10.6 million. We believe that approximately half of these costs, or $5.3 million, could have been put to better use had SSA management properly planned and monitored this contract. In light of the history of changing requirements and increasing costs, we believe management must provide an enhanced degree of management controls and oversight to ensure timely implementation of the project.

Conclusions and Recommendations

The contractor generally adhered to the terms of the contract and delivered the services and final design options that SSA asked for under the contract. While SSA is required by law to continue with the demonstration project, we have several concerns about the delays and costs associated with the design phase that should be reviewed by Agency managers before moving into the implementation phase. During this period, prior SSA management provided inadequate oversight of the contract concerning the planning, scope and expenditures, and current SSA management did not address financial monitoring issues. For example, the contract was not monitored in a way that allowed for quickly detecting or avoiding cost overruns for tasks, nor was the contractor's performance timely evaluated. In terms of contract coordination, the initial BOND design was submitted to former SSA management before the Four-State Pilot formally presented its feedback to the contractor. Earlier communication between BOND and the Four-State Pilot regarding the project's design would have been more useful to the contractor. Finally, we found that the current sole-source contract lacks clear separation of duties by allowing the contractor to evaluate its own performance.

To enhance oversight on the BOND project and future contracts of a similar nature, we recommend that SSA:

2. Establish a greater degree of management oversight by strengthening contract oversight roles and responsibilities and more clearly defining contract requirements.

3. Improve financial monitoring by implementing the procedures necessary to (a) identify the work for which the contractor is requesting payment and (b) maintain documentation to prove to an outside party that value was received for the money expended.

4. Conduct and document interim and final evaluations of the contractor's performance in accordance with the contract and as recommended by the FAR.

5. Ensure timely and meaningful participation occurs with other parties associated with the demonstration project, such as the States involved in the Four-State Pilot, to enhance the total project.

6. Ensure the evaluation phase of the BOND contract is conducted by an independent party.

AGENCY COMMENTS

With the exception of Recommendations 3 and 6, SSA generally agreed with our report and recommendations. SSA stated it had already made changes to improve the overall management of the project as it moves toward implementation. Regarding Recommendation 3, SSA disagreed with the wording of this recommendation and believe it refers to its ability to monitor the work for which the contractor requested payment, and not just recordkeeping. SSA stated that different contractors report and bill differently. SSA noted that, in this contract, the combination of the monthly invoices, the monthly progress reports, and the close communication between the Contract Officer Technical Representative and the contractor's project director provided enough information to determine the contractor was billing correctly for each task. Regarding Recommendation 6, SSA disagreed with our recommendation and believes any future contract, with appropriate built-in safeguards, can provide adequate mitigation against conflicts of interest. SSA further indicated that the FAR allows the Agency to avoid, neutralize, or mitigate conflicts of interest. SSA believes it has appropriate mitigation strategies. See Appendix H of this report for SSA's full comments.

OIG RESPONSE

Appendices

Appendix A
Acronyms

Abt Abt Associates, Incorporated
BOND Benefit Offset National Demonstration
C.F.R. Code of Federal Regulations
COTR Contract Officer Technical Representative
DI Disability Insurance
EI Early Intervention
FAR Federal Acquisition Regulation
OAG Office of Acquisition and Grants
OF Office of Finance
OMB Office of Management and Budget
SGA Substantial Gainful Activity
SSA Social Security Administration
U.S.C. United States Code
WSO Work Support Organizations

Appendix B
Background on the Benefit Offset National Demonstration Design

The following background information was provided in the Benefit Offset National Demonstration (BOND) contract (Contract Number SS00-04-60110).

Purpose of BOND

In this demonstration project, the Social Security Administration (SSA) plans to test a $1 reduction in benefits for every $2 in earnings in combination with other changes to SSA's rules, with the goal of enabling more beneficiaries to return to work and maximize their employment, earnings, and economic independence. Unlike current Title II policy, the benefit offset would allow beneficiaries to face a gradual reduction in their benefits should they earn above a specified amount of monthly income. Participating beneficiaries would also maintain ongoing eligibility for health care benefits and other supports linked to Title II eligibility.

SSA anticipates that a strategy of combining various changes to SSA's rules with a benefit offset will reduce barriers to work and allow beneficiaries with disabilities to increase their employment, earnings, and independence. Participants in the benefit offset demonstration will be able to take advantage of the interventions offered, without the possibility of losing all their cash benefits if they have earnings even minimally above the Substantial Gainful Activity (SGA) level. Just as importantly, participants earning above SGA will maintain ongoing Medicare coverage eligibility.

The Ticket to Work and Work Incentives Improvement Act of 1999

The BOND project was initiated under The Ticket to Work and Work Incentives Improvement Act of 1999, Public Law No. 106-170 § 302, 42 U.S.C. § 434 note (the Act), signed into law on December 17, 1999. One of the major congressional findings cited in the Act is that, despite recent changes in public law and assistive technologies that have provided historic employment opportunities for the disabled, and the desire of many beneficiaries with disabilities to work and support themselves, fewer than one-half of 1 percent of Social Security Disability Insurance (DI) (and other beneficiaries receiving disability benefits under the Title II program) and Supplemental Security Income recipients leave the rolls due to work. In addition, the findings state that "beneficiaries cite financial disincentives to work and earn income" as a barrier to employment. For example, one financial disincentive that is often cited is the abrupt loss of all cash benefits in the Title II disability program, frequently termed the "cash cliff." Under current law, Title II disability beneficiaries are allowed a 9-month period of trial work in which they may have earnings of any amount with no effect on their benefits. After the trial work period (and a brief grace period), however, beneficiaries who are determined to be engaging in SGA lose their entire benefit and face the eventual loss of Medicare benefits as well. Current law provides for at least 7 ¾ years of Medicare coverage beyond the trial work period for beneficiaries who return to work.

Among other things, the Act requires that SSA undertake a series of demonstration projects or experiments to test alternatives to current program work incentives. Section 301 amended the Social Security Act by adding section 234. Section 234(a) directs the Commissioner of SSA to carry out experiments and demonstration projects to determine the relative advantages and disadvantages of (1) various alternate methods of treating work activity of individuals receiving Title II benefits based on disability, including such methods as a reduction in benefits based on earnings, designed to encourage these beneficiaries to return to work; (2) altering other limitations and conditions, such as lengthening the trial work period, or altering the 24-month waiting period for Medicare; and (3) implementing a sliding scale benefit offset. This section also requires that these demonstration projects be designed to show that savings will accrue to the Trust Funds or will otherwise promote or facilitate the administration of Title II. Section 234(b) also provides that these projects must be conducted in a manner that will allow SSA to evaluate the appropriateness of implementing such a program on a national scale.

Section 234(c) allows the Commissioner of SSA to waive compliance with the benefit provisions of Title II of the Social Security Act (42 United States Code [U.S.C.]
§ 401 et seq.), and the requirements of section 1148 of the Act (42 U.S.C. § 1320b-19) as they relate to the Ticket to Work program, and the Secretary of Health and Human Services to waive compliance with the benefit requirements of Title XVIII of the Social Security Act (42 U.S.C. § 1395 et seq.), insofar as is necessary for a thorough evaluation of the alternate methods under consideration.

Appendix C
Contracting Duties

The contract officer, appointed by the Office of Acquisition and Grants (OAG), is responsible for awarding and administering Social Security Administration (SSA) contracts. The contract officer requires Team Leader or Division Director approval on modifications and new awards that exceed $100,000. For the Abt Associates, Incorporated, contract, all modifications exceeding $100,000 require the approval of the Division Director. Any required contract revisions must be transmitted to the contractor over the signature of the contract officer.

The project officer, an employee in SSA's Office of Retirement and Disability Policy, is responsible for the technical administration of the contract. The project officer is responsible for determining whether supplies or services delivered by the contractor conform to the contract quality requirements.

Contract monitoring varies considerably both in intensity and in methodology, depending on the importance and size of the contract effort, as well as the type of contract. Cost reimbursement type contracts generally warrant closer monitoring because the Government's risk is higher than under a fixed price contract. When unsatisfactory contract performance is identified, the project officer must notify the contract officer promptly so that remedial steps can be taken. Silence on the part of the Government could be interpreted by the contractor as Government acceptance of performance, which may differ from that stated in the contract. Such situations could adversely affect the Government's right to withhold payments, terminate for default, or otherwise exercise certain rights under the contract.

MODIFICATIONS

A contract modification is any written change in the terms of the contract, that is, statement of work, period of performance, quantity, price or other provisions of a contract, whether accomplished in accordance with a contract provision or by mutual agreement of the parties. During the contract life, different types of modifications may

be necessary to incorporate new requirements or to handle problems that develop after the contract is awarded. Contract modifications must be made in writing by the contract officer to preclude misunderstanding between the parties concerning work to be performed.

Requests for unilateral modifications are initiated by the Government. Unilateral modifications (such as administrative changes) are within the authority of the contract officer, without agreement from the contractor. Modifications resulting from bilateral modifications can be initiated by a written request from either the Government or the contractor. For bilateral modifications, the project officer must prepare a supporting memorandum to document the need for the modification and provide other appropriate information necessary to process it including, but not limited to

the type of modification recommended;
the basis for the modification (explaining the circumstances-for example, who, what, when, where and why-that resulted in the need for the modification and the reasons why a modification should be made);
a brief description of the contractor's performance, as well as the identification of any known problem areas;
an independent Government estimate of cost for the modification;
the estimated total time necessary to accomplish the required services; and
a complete description of the work to be changed or modified.

Appendix D
Scope and Methodology
To achieve our objectives, we:

Reviewed applicable sections of the Federal Acquisition Regulation, Administrative Instructions Manual System, Social Security Administration (SSA) Acquisition Regulations, Department of Health and Human Services Project Officers' Contracting Handbook, and the Prompt Payment Act.
Reviewed the contract between SSA and Abt Associates, Incorporated, (Abt) (Contract Number SS00-04-60110) to understand the contract requirements.
Interviewed the SSA project officer in the Office of Program Development and Research, contract officer and other staff in the Office of Acquisition and Grants (OAG), and staff in SSA's Office of Finance (OF) to discuss the required contract deliverables and payment terms.
Reviewed the records used to monitor the contractor's performance and determined whether SSA and Abt performed their duties in accordance with the contract.
Reviewed documentation (vendor invoices, and task and delivery order information) supporting payments to Abt under the contract from September 2004 through December 2007.
Obtained a list of paid invoices for this contract from OF. From these invoices, we identified the total amount paid to Abt. We reviewed 37 paid invoices that totaled $4,999,827 paid from September 2004 through December 2007. We reviewed the 37 paid invoices to ensure (1) SSA paid amounts approved in the contract; (2) invoices were approved by the project officer before payment; (3) SSA paid invoices timely in accordance with the terms of the contract; and (4) invoice amounts were recorded correctly.
Interviewed representatives from the four States involved in the Four-State Pilot.

We determined that the data used for this audit were sufficiently reliable to meet our audit objectives. The entities audited were OAG and OF within the Offices of Budget, Finance and Management and Retirement and Disability Policy, respectively. We conducted our audit in Chicago, Illinois, and Baltimore, Maryland, from August 2007 through August 2008. We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix E
Benefit Offset National Demonstration (BOND) Contract Timeline

Note on Chart: The chart demonstrates some but not all of the costs and milestones related to the contract.

Note 1. This amount is reflected in the Base Contract.
Note 2. The demonstration pilots were designed to inform BOND. The States involved in the pilots were Connecticut, Utah, Vermont and Wisconsin.
Note 3. Early Intervention was added under Modification #3 (see page 1 of Appendix F).
Note 4. Health Benefits was added under Modification #5 (see page 1 of Appendix F).
Note 5. This date is reflected in the Base Contract. The costs reflect the award as of Modification #6 (see page 1 of Appendix F).
Note 6. WSO was added under Modification #7 (see page 1 of Appendix F).
Note 7. The conference with the four States was added under Modification #9 (see page 2 of Appendix F).
Note 8. Early Intervention was dropped under Modification #10 (see page 2 of Appendix F).
Note 9. BOND Stand Alone Calculator and Concurrent beneficiaries were added under Modification #13 (see page 2 of Appendix F).
Note 10. Both WSO and Health Benefits were dropped under Modification #15 (see page 2 of Appendix F).
Note 11. This estimated completion date is reflected in Modification #16 (see page 3 of Appendix F).

Appendix F

Benefit Offset National Demonstration Contract Modifications
Modification Effective Date Purpose of Modification Award 1 12/30/04 Contract officer change. 02
3/3/05 Revised the contractor's representative information. 03 9/30/05 Incorporated an early intervention model in the Benefit Offset National Demonstration (BOND) design. $270,670 4 11/22/05 Provided information in a letter regarding the Electronic Subcontracting Reporting system. 05
8/21/06 Incorporated a design for a health benefits package and extended the contract to December 2006. $404,657 6
9/14/06 Funded additional in-scope work for development of benefit offset simulation model to inform the choice of treatment parameters and to generate bounded cost estimates for the demonstration. $207,177 7 1/5/07 Funded additional in-scope work to conduct pre-implementation tasks, including developing a pre-implementation plan, workforce investment boards, One-Stop Service Centers, and Work Support Organizations; developing a plan for procurement of fiscal agent; interim tasks for developing the BOND data warehouse pages; developing the BOND benefit calculator; developing a data security plan for BOND; furthering early intervention design specification; and developing public information specifications. Also extended the contract through June 30, 2007. $2,081,570 8 12/29/06
Extended the contract at no cost to the government through January 31, 2007. 0 Modification Effective Date Purpose of Modification Award 9 1/23/07 Provided funding for conference facilitation services and reflected cost increases.b $146,791 10 6/27/07 Deleted workforce investment boards and One-Stop Service Centers and the early intervention design specification. Extended the contract period through August 31, 2007. 011 8/27/07 Extended the contract period of performance until September 30, 2007. 012 9/13/07 Added funding to the contract and extended the contract period of performance through October 31, 2007. $49,983
13 10/30/07 Added funding to the contract to develop the BOND stand-alone calculator; required new bounded cost estimates, new sample design and final design parameters; added some references to concurrent beneficiaries; and extended the contract period of performance through January 31, 2008. $1,136,196
14 1/29/08 Extended the contract period of performance until February 15, 2008. 015 2/12/08 Added funding to the contract and extended the contract period of performance through May 31, 2008; deleted the health benefits provision design; deleted plan for procurement of fiscal agent and replaced it with developing plans for earnings reporting and adjustment; deleted Work Support Organizations; added tasks to conduct validation of phase 1 software component specifications; developed baseline architecture; tested SSA data exchange; developed public information specifications; added BOND site office specifications planning and implementation; developed BOND site office procedures manual; designed intensified benefit counseling; developed detailed specifications and cost estimates; conducted research and site visits; developed specifications and procedures for second stage intake; developed consent form; developed plans for BOND pilot; and developed plans and materials for BOND training. $1,833,595 Modification Effective Date Purpose of Modification Award 16 5/27/08 Extended the contract completion date from May 31, 2008 through September 29, 2008 and included new tasks and deliverables, including progress memorandum for phase 2 testing, progress reports for implementation plans, phase I test plans of BOND Data System testing, progress memos for revisions to baseline architecture, code development and BOND Data System component testing, and development of benefit counseling recruitment materials. $2,049,557 Note a: Contract Modification #6 was to end December 31, 2006. Contract Modification #8 was initiated to extend the end date of the contract to January 31, 2007. Contract Modification #7 was initiated to fund pre-implementation tasks and extend the entire contract to June 30, 2007.

Note b: This conference related to the BOND contractor meeting with the Four-State Pilot participants.

Appendix G
Benefit Offset National Demonstration Tasks and Deliverables
Task/
Deliverable
Description

1 Project start-up activities
2 Design of the interventions
3 Develop Technical Research Design
4 Develop Evaluation Plans
5 Develop Survey Plans
6 Prepare Office of Management and Budget clearance package
7 Development of Data Collection/Information Management Plans
8 Final Report
9 Health Benefits (Deleted) Modification 15, Effective 2/12/08
Management Time spent by Abt's Project Director, Deputy Project Director, and Finance and Contract Administrator on administrative functions; coordination of the extended project team, including subcontractors and consultants; internal contract review reports and meetings; preparation of progress reports to the Social Security Administration (SSA); interaction with SSA project officer on management issues; budget preparation, maintenance, and updating; invoice checking and approval; labor planning and scheduling; and processing multiple contract modifications, including additions, removals, and modifications to the subcontract documents for numerous subcontractors and the agreements for several consultants.
10 Develop a Pre-Implementation Plan
11 Workforce Investments Boards and One-Stop Service Centers
12 Work Support Organizations (WSO)
(Deleted) Modification 15, Effective 2/12/08
13 Develop a Plan for Procurement of Fiscal Agent
(Deleted and Replaced) Modification 15, Effective 2/12/08
Develop Plans for Earnings Reporting and Adjustment
14 Interim Tasks for developing the BOND Data Warehouse
15 Develop the BOND Benefit Calculator
16 Develop a Data Security Plan for BOND
17 Further Early Intervention Design Specification
(Deleted) Modification 10, Effective 6/27/07
18 Develop Public Information Specifications
19 Develop the BOND Stand Alone Calculator
20 BOND site office specifications, planning and implementation
21 Design Intensified Benefit Counseling
22 Develop specifications and procedures for Second Stage Intake
23 Develop plans for BOND Pilot
24 Develop plans and materials for BOND training

Appendix H
Agency Comments

MEMORANDUM

Date: February 9, 2009

To: Patrick P. O'Carroll, Jr.
Inspector General

From: James A. Winn
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report, "Contract for the Benefit Offset National Demonstration Project with Abt Associates, Incorporated (A-05-08-18041)

Thank you for the opportunity to review and comment on the draft report. We appreciate OIG's efforts in conducting this review. Our response to the report findings and recommendations is attached.

Please let me know if we can be of further assistance. You may direct staff inquiries to Ms. Candace Skurnik, Director, Audit Management and Liaison Staff, at (410) 965-4636.

SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL DRAFT REPORT, "CONTRACT FOR THE BENEFIT OFFSET NATIONAL DEMONSTRATION (BOND) PROJECT WITH ABT ASSOCIATES, INCORPORATED" (A-05-08-180-18041)

In general, we agree with the report and recommendations. We have already made changes to improve the overall management of the project as we begin to move toward implementation. Our response to the specific recommendations is as follows.

Recommendation 1

Before awarding the next phase of BOND, ensure a detailed cost-benefit analysis of the BOND design phase has been conducted and documented to support an agency decision to continue with this contractor. As a part of the analysis, the agency should have: a) reviewed the current status of the project; b) assessed what value the agency received for the costs already incurred; c) determined whether the entire BOND project can be completed at a reasonable cost; and d) documented the agency's justification for the continued use of the current sole-source contractor as well as the type of contract being used.

Comment

We agree with this recommendation, with the following explanation:

1) Under current management, we have made substantial changes to the project design to reduce the costs and still meet the legislative requirements.

2) We are currently working through the details of the benefit offset business process. Once this is completed, we anticipate having more accurate cost estimates.

3) The design contract ended at the end of fiscal year 2008, and we are working now to finalize a Statement of Work for the implementation and evaluation contract. We are using information drawn from the detailed business process and detailed design documents to estimate the costs for the next award and to further assess whether we can complete the project at the current cost estimate ($100 million).

Recommendation 2

Establish a greater degree of management oversight by strengthening contract oversight roles and responsibilities and more clearly defining contract requirements.

Comment

We agree. Under current management, we have already made changes to the project consistent with the recommendation. The changes include:

1) working with other components (including the Office of Employment Support Programs, the Office of Communications, and the Office of Retirement and Survivors Insurance Systems) to carefully review the work performed by Abt;

2) establishing an inter-component workgroup to develop more clearly defined contract requirements and the BOND business process;

3) delaying the implementation contract until we have worked out the details of the business process, which will allow us to specify clearly defined contract requirements; and

4) having the implementation and evaluation contract Statement of Work reviewed by other components.

Recommendation 3

Improve financial monitoring by implementing the procedures necessary to (a) identify the work for which the contractor is requesting payment and (b) maintain documentation to prove to an outside party that value was received for the money expended.

Comment

We disagree with the implication that we did not have sufficient information to determine the deliverables and contract tasks associated with the costs. Our response is based on the wording of the recommendation, which clearly refers to our ability to monitor the work for which the contractor requested payment, and not just record-keeping. With regard to monitoring, different contractors report and bill differently. In this contract, the combination of the monthly invoices, the monthly progress reports, and the very close communication between the Contract Officer Technical Representative (COTR) and the contractor's project director provided enough information to make the determination that the contractor was billing correctly for each task.

Recommendation 4

Conduct and document interim and final evaluations of the contractor's performance in accordance with the contract and as recommended by the Federal Acquisition Regulation (FAR).

Comment

We agree. Please note that the Contracting Officer generated an interim performance report for the COTR in accordance with FAR 42.1502(a) on June 30, 2008. Subsequently, the contractor submitted comments to the Contractor Performance System (CPS) in accordance with FAR 42.1503(b) on September 8, 2008, at which time the Contracting Officer finalized the interim report. Additionally, the Contracting Officer generated a final performance report for the COTR in accordance with FAR 42.1502(a) during November 2008. Subsequently, the contractor submitted comments to the CPS on December 18, 2008. The Contracting Officer finalized the report at that time. The draft report should be corrected to reflect these facts.

Recommendation 5

Ensure timely and meaningful participation occurs with other parties associated with the demonstration project, such as the States involved in the Four-State Pilot, to enhance the total project.

Comment

We agree. While we concur with this recommendation, we believe we have been in compliance with this recommendation. Current management recognized this shortcoming and, since taking over the project, has taken steps to ensure that timely and meaningful participation occurs with other parties associated with the demonstration project. It is important to note that the primary intent of the Four-State project was to assist us in better understanding operational/implementation issues of a benefit offset. The Four-State projects have achieved this goal. We instituted monthly project calls with the four States, which allow us the opportunity to receive direct feedback from the sites, and ongoing conversations with our Office of Public Services and Operations Support on the operational issues related to the offset for participants in the Four-State sites.

The draft report states that "earlier communications with the States regarding the project's design would have enhanced its usefulness to SSA." However, recruitment for the pilot project did not end until December 2006, and we received the first interim evaluation findings in the spring of 2008. Nevertheless, in March 2007 we convened a joint conference with staff and management from the States involved in the pilot and the BOND contractor. The States shared valuable comments and recommendations during the conference. These were based on their expertise in the field and not on any findings from the pilot. This information helped us further develop the design for the project, which was finalized in September 2008.

Finally, both the BOND and the State projects had the same COTR, who was in constant contact with both the BOND contractor and the pilot States.

Recommendation 6

Ensure the evaluation phase of the BOND contract is conducted by an independent party.

Comment

We disagree. We believe any future contract, with appropriate built-in safeguards, can provide adequate mitigation against conflicts of interest.

The FAR allows the agency to avoid, neutralize, or mitigate conflicts of interest. We feel we have appropriate mitigation strategies and have done the following:

Evaluation structure. The evaluation for BOND has two major components: quantitative evaluation outcomes (impact) and a process evaluation that will assess the implementation of the project. With regard to the impact evaluation, we believe that a random assignment design will allow for a transparent evaluation, the results of which cannot be misconstrued by the evaluator. The methodologies used to conduct this evaluation will be heavily quantitative and standardized, and will involve few judgments by the contractor. We are confident that this design will prevent a conflict of interest from occurring. With regard to the process evaluation, the contractor will not implement the project alone. The contractor will conduct the recruitment and random assignment, collect data, and monitor the fidelity of treatments. We will implement the programmatic aspects of the demonstration, and we will calculate benefits and send notices to the beneficiaries. The contractor may make recommendations during project implementation based on difficulties it perceives, e.g., additional training, but this would be only for purposes of ensuring that the intervention is consistent with the design.

Social Security Staff Oversight. Our Office of Program Development and Research (OPDR) staff will continue to provide strong oversight. In addition to our carefully monitoring the contract, we would like to note that task 2 of the current draft Implementation Statement of Work allows for a project advisory board within the implementation contract. Under this task, the contractor shall formally establish an advisory group of approximately nine individuals, with experts in the fields of individuals with disabilities, employment, income support programs, benefits counseling, random assignment evaluations, and other domains that would complement contractor staff expertise in implementation and evaluation. The advisory group is intended to provide the contractor with the opportunity to consult with and engage experts who can provide advice and guidance to us and the contractor on the overall study implementation and approach, and those who can provide technical expertise on implementation of the evaluation.

Since we sent you our July 2008 comments, we have also moved to strengthen our independent research advice. We are working to establish a team of expert consultants to review and critique the contractor's work. This will also help address a recent recommendation made by the Government Accountability Office that we use an "expert panel to advise it about the projects on a regular basis."

The use of an expert panel has proven to be an effective way of obtaining independent evaluation of our other projects. For the early phases of this project (prior to 2002), we used a team of seven expert consultants to provide a critical review of our plans for the project. The team included the following experts:

Economics: Donald Parsons, Professor, George Washington University; John Rust, Professor, University of Maryland College Park.

Sociology: Nancy Tuma, Professor, Stanford University; Judy Tanur, Professor, SUNY-Stony Brook.

Statistics: Joseph Sedransk, Professor, Case Western Reserve University

Public Health: Donald Patrick, Professor, University of Washington

Vocational Rehabilitation: David Vandergoot, PhD, Essential Solutions.

These experts played an important role in the assessment of demonstration designs to measure induced entry and helped assess the sample size initially proposed for the demonstration. We have used this approach for other projects, such as the use of an expert panel from the Institute of Medicine to provide advice on the National Study of Health and Activity. We believe that this approach is the most effective way to provide independent oversight of the project.

In addition, we have recently drafted a task order to bring additional analytical support to OPDR to assist with methodological issues on our demonstrations.

Appendix I
OIG Contacts and Staff Acknowledgments
OIG Contacts

Walter Bayer, Director, Chicago Audit Division (312) 353-0331
Annette Dunn, Audit Manager, Chicago Office (312) 886-4160
Acknowledgments
In addition to those named above:
Lorrie Clement, Auditor-in-Charge
Nichole Purnell, Program Analyst
Linda Smid, Auditor
Wai Ho Yung, Auditor

For additional copies of this report, please visit our web site at www.socialsecurity.gov/oig or contact the Office of the Inspector General's Public Affairs Staff Assistant at (410) 965-4518. Refer to Common Identification Number A-05-08-18041.

Overview of the Office of the Inspector General
The Office of the Inspector General (OIG) is comprised of an Office of Audit (OA), Office of Investigations (OI), Office of the Counsel to the Inspector General (OCIG), Office of External Relations (OER), and Office of Technology and Resource Management (OTRM). To ensure compliance with policies and procedures, internal controls, and professional standards, the OIG also has a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit
OA conducts financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management reviews and program evaluations on issues of concern to SSA, Congress, and the general public.

Office of Investigations
OI conducts investigations related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as liaison to the Department of Justice on all matters relating to the investigation of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Counsel to the Inspector General
OCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Also, OCIG administers the Civil Monetary Penalty program.

Office of External Relations
OER manages OIG's external and public affairs programs, and serves as the principal advisor on news releases and in providing information to the various news reporting services. OER develops OIG's media and public information policies, directs OIG's external and public affairs programs, and serves as the primary contact for those seeking information about OIG. OER prepares OIG publications, speeches, and presentations to internal and external organizations, and responds to Congressional correspondence.

Office of Technology and Resource Management
OTRM supports OIG by providing information management and systems security. OTRM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, OTRM is the focal point for OIG's strategic planning function, and the development and monitoring of performance measures. In addition, OTRM receives and assigns for action allegations of criminal and administrative violations of Social Security laws, identifies fugitives receiving benefit payments from SSA, and provides technological assistance to investigations.