10-14 - Adequacy of Quality Assurance Oversight of the Loan Management and Accounting System Project

This report presents the results of our audit of the Small Business Administration's (SBA) Quality Assurance (QA) Oversight of the Loan Management and Accounting System (LMAS) Project. To guide QA activities, SBA developed a QA Plan identifying steps that it will take to evaluate overall performance of the LMAS project on a regular basis to ensure that it conforms to the Agency's quality standards and satisfies user needs. A major component of a QA plan is the Independent Verification and Validation (IV & V) of development products to ensure that design specifications are correctly implemented and that deliverables accomplish their intended purpose. SBA contracted with TestPros to develop and implement the QA plan, including conducting all IV & V activities.

The objectives of the audit were to determine the: (1) adequacy of SBA's QA Plan; (2) extent to which the QA Plan was being implemented; and (3) quality of IV& V activities. The audit was performed in response to language in the OIG's Fiscal Year (FY) 2010 appropriation, which directed the OIG to provide routine analysis and reporting on SBA' s modernization of its loan management and accounting systems.

The OIG has issued three reports on the LMAS project since it was initiated in 2005 and, due to umesolved project risks, designated oversight of the LMAS project as a new management challenge in FY 2010(1). Two of the most recent reports(2) found that the Agency had not established either an effective enterprisewide or project-level QA function to ensure that LMAS project deliverables meet SBA requirements and quality standards. The most recent audit found that the project-level QA process was not independent from project management staff and that the project lacked a defined process for accepting deliverables.

To assess the adequacy ofLMAS QA Plans, we compared the LMAS QA Plan to SEA's Systems Development Methodology (SDM) issued by the Office of the Chief Information Officer and to the Federal Acquisition Regulation. We also reviewed the Quality Assurance Surveillance Plans (QASP) for each project task order to determine whether components of the QA plan had been incorporated.

To determine the extent to which the QA plan had been implemented, we reviewed all activities of TestPros, the QAlIV &V contractor, that were documented in project status reports and correspondence between TestPros and the LMAS project management staff. We also interviewed selected personnel from SBA's Office of the Chief Information Officer (OCIO), LMAS project management staff, and contractor personnel to gain an understanding of the QA process, staff roles and responsibilities for implementing the QA plan, and the project's status. Finally, to determine the adequacy of IV & V activities, we reviewed the solicitation and other documents related to SBA' s contract for IV & V and compared the activities to work performed.

The OIG contracted with the Loch Harbour Group to assist the OIG in its audit of the LMAS QA process. We performed the audit work between October 2009 and April 2010 in accordance with Government Auditing Standards prescribed by the Comptroller General of the United States.

The audit disclosed that the QA plan does not address all key areas prescribed by the Agency's System Development Methodology (SDM), and that the QA contractor has not performed all of the QA activities stipulated in its contract, including the performance audits of the other LMAS contractors. Further, none of the LMAS task orders issued to date have undergone IV & V testing. These lapses in QA oversight of the project have made the Agency unduly reliant on the other LMAS contractors to ensure the quality of their products and increased the risk that the project will not perform as intended.

We recommended that SBA: revise the LMAS QA plan to incorporate all the components required by the enterprise-wide QA plan; take steps to hold TestPros accountable for performing all the activities specified in its contract; and revise TestPros' statement of work to include the IV & V responsibilities previously deleted. Further, we recommended that SBA evaluate and make necessary adjustments to the QA manager's workload to ensure that he can devote adequate time to oversee complete implementation of the enterprise QA oversight function. SBA management expressed concurrence with our recommendations.