Information Technology System Governance and Audit Application

1. Information Technology System Governance and Audit Application Presented by DOT OIG’s Office of Acquisition & Procurement Audits Federal Audit Executive Council (FAEC) Procurement Conference June 12, 2013

2. Agenda Scope of Audit • Audit of DOT and FAA • Basis of Selection Criteria & Background • Laws and Regulations • DOT Structure Findings, DOT’s Actions, & OIG Recommendations • Department Level • FAA Recent Initiatives Recent Initiatives • Proposed Legislation • OMB memos Audit “To Do” List • Audit Steps needed to complete • similar audit

3. Scope of Audit • Department Level: DOT • Objective:Assessed if DOT’s investment governance practices met Federal and statutory investment oversight requirements and best practices. • DOT’s FY2012 majorIT investment portfolio was valued just over $2.2 billion. • OMB Circular A-130, defines “major information system(s)” as those that require special management attention for reasons including importance to the agency’s mission; high development, operating, or maintenance costs; or significant role in administering agency programs. • Agency Level: FAA • Objective:Assessed if FAA and DOT provided sufficient oversight of FAA’s major IT investments. • Basis of Selection: • About 94 percent ($2.07 billion) of DOT’s portfolio is managed by the FAA, 30 percent is devoted to NextGen programs, a multibillion-dollar effort to modernize the U.S. air traffic control system. • Since 2005, FAA has experienced cost overruns, schedule delays, or both, on 7 of its 14 major air traffic control IT programs, including 1 that exceeded original cost estimates by $2 billion and was delayed by 14 years. • Numerous OIG and GAO audit reports and testimonies related to FAA’s major IT investments point to longstanding and significant concerns regarding the Agency’s management and oversight of these critical programs.

4. Criteria & Background • Clinger-Cohen Act of 1996: • Established the Chief Information Officer (CIO) position in Federal executive agencies. • Requires each agency to implement a management framework to optimize IT expenditures. • OMB’s Circular A-130, “Management of Federal Information Resources,” issued November 2000, requires the agency’s CIO to: • Monitor and evaluate IT investment performance through a capital planning and investment control process, and • Advise the agency head on budgetary implications of IT decisions. • In response, DOT chartered its Investment Review Board (IRB) in December 2009 and required each of the 12 agencies within DOT to establish their own IRBs. • According to FAA, the Clinger-Cohen Act does not apply to the Agency. • In 1996, Congress passed 49 U.S.C. § 40110(d) - 49 U.S.C. 106(f). • Allowed the FAA to no longer have to follow Federal Acquisition Regulation (FAR). • FAA granted authority to establish its own Acquisition Management System (AMS) to address the unique needs of FAA without requiring compliance with most Federal acquisition laws or regulations, including the Clinger-Cohen Act. • FAA established its Joint Resources Council (JRC), as its IRB, to help ensure FAA’s capital investments fulfill mission priorities and maximize resources.

5. Findings, DOT’s Actions, & OIG Recommendations • Findings • DOT’s investment oversight practices did not fully meet OMB requirements or DOT’s policies. • DOT’s IRB board was in “hibernation”. • Despite OMB’s requirements and itsIRB charter, DOT did not hold a meeting for nearly 2 years. • The Senior Procurement Executive (SPE) was a member but not a voting member of the board. • DOT relied solely on TechStat Accountability Sessions (TechStat). • TechStats review individual troubled programs, whereas IRB reviews provide strategic oversight of DOT’s entire IT investment portfolio. • DOT’s TechStat process was not institutionalized.

6. Findings, DOT’s Actions, & OIG Recommendations (cont’d) • DOT’s Actions • Before OIG’s final audit report was issued: • DOT held an IRB meeting in December 2012 which: • Proposed a draft revised IRB charter which made the SPE a voting member and included the TechStat process. • Introduced a new IT governance framework which was a conceptual model that: (1) illustrates the intended roles of the DOT’s and Agencies’ IRBs, (2) incorporates a new Investment Working Group, (3) and requires collaboration among DOT’s and Agencies’ budget offices. • OIG Recommendations • Finalize the draft revised IRB charter. • Develop a comprehensive implementation plan for its proposed IT governance framework. • Establish written policies and procedures for TechStats.

7. Findings, DOT’s Actions, & OIG Recommendations - FAA Five Key Steps in the FAA Lifecycle Management Process Concept & Requirements Definition Readiness Decision Investment Analysis Readiness Decision Initial Investment Decision Final Investment Decision In-Service Decision

8. Findings, DOT’s Actions, & OIG Recommendations - FAA (cont’d) • Findings • Both FAA and DOT are not providing sufficient oversight of FAA’s major IT investments. • FAA’s JRC established a comprehensive framework for IT governance, but does not always follow its process. • Decisions are made without required critical investment information, such as program requirements and cost and schedule estimates. • Example: JRC authorized $15 million to conduct final investment analysis for a NextGen Facilities program without: • Implementation Strategy and Planning Document (ISPD) - FAA describes as the most critical, relevant, and meaningful information for investment decision making. • Final Investment Analysis Plan - which contains cost and benefits estimates and final requirements for the most promising alternative. • Critical Investment documents were not always approved prior to decisions or were backdated. • Example: JRC made over $1 billion in investment decisions for critical NextGen programs without approval of all required documentation such as, the final business case, program baseline, and ISPD. • Decision meeting dates are not always met, therefore, threatening the schedule of interdependent programs. • As a result, FAA is taking undocumented risks when making investment decisions worth billions of dollars.

9. Findings, DOT’s Actions, & OIG Recommendations – FAA (cont’d) • Findings (cont’d) • DOT performed limited oversight of FAA investments even though it accounts for 94% of DOT’s investment portfolio. • FAA agreed to participate in DOT’s IRB program by signing the IRB charter and “voluntarily participating”, despite its independent acquisition authority. • DOT still views the independent acquisition authority as an impediment; therefore, it only reviewed 2 FAA programs between December 2009 and April 2012. • DOT’s Actions &OIG Recommendations • DOT concurred with the OIG’s recommendations which included: • Strengthen controls to ensure required investment decision documents are reviewed and approved prior to JRC decision meetings. • Develop procedures to (1) document JRC’s discussions with program offices when decisions are made without required investment documents, and (2) identify risks of moving forward without the documents, and (3) keep the documented discussions in the JRC repository.

10. Recent Initiatives • March 2013 - Rep. Darrel Issa, Chairman of The House Oversight and Government Reform Committee, introduced new legislation, entitled the Federal Information Technology Acquisition Reform Act, which: • Places one CIO clearly in charge at each agency, including the authority to approve the of hiring personnel who will have IT responsibilities. • Provides the CIO with budget authority over all IT spending. • Establishes an inter-agency forum led by the CIO Council. The Council shall develop portfolio management policies to allow for the development of cross-agency shared services and shared platforms. • Consolidates resources and expertise to make smarter purchases. • Develops specialized assisted acquisition centers of excellence within the Federal Government to promote the effective use of best acquisition practices, development of specialized expertise in the acquisition of IT, and Government-wide sharing of acquisition capabilities to augment any shortage in the IT acquisition workforce.

11. Recent Initiatives (cont’d) • OMB Memorandum “Implementing PortfolioStat”, dated March 30, 2012. • 5 phase plan to institute PortfolioStats - part of initiative to root out waste and duplication across the Federal IT portfolio. • (1) Baseline Data Gathering: by May 2012 – Chief Operating Officer(COO) to perform high-level survey of agency IT Portfolio status. • (2) Analysis and Proposed Action Plan: Using data from phase 1 – COO to identify wasteful or duplicative investments. In partnership with the CIO, CAO, CFO, COOO to draft proposed action plan to consolidate commodity IT spending. • (3) PortfolioStat Session: by July 31, 2012 - Federal CIO and agency CIO, CAO, CFO, COO meet for 1 hour to review portfolio data and proposed action plan. Agree on concrete next steps resulting in a final plan with corrective actions. • (4) Final Action Plan Implementation: by August 31, 2012: agencies shall submit to OMB a document outlining its plan to rationalize and consolidate its IT portfolio. • (5) Lessons Learned: by February 2013: agencies submit to OMB a document of its successes, challenges, and lessons learned. • OMB Memorandum “Fiscal Year 2013 PortfolioStat Guidance: Strengthening Federal IT Portfolio Management”, dated March 27, 2013. • As a result of the 2012 Portfolios and the above actions, OMB identified the following best practices: • Empowering Agency CIOs • Strengthening IT Portfolio Governance • Advancing Service Delivery

12. Audit “To Do” List • Research current regulations, guidance, and best practices affecting IT governance and oversight of major IT investments • Understand organization’s governance framework • Collect IRB charters, policies, and procedures and review for comprehensiveness and organizational compliance. • Identify and assess effectiveness of oversight processes. (For example, TechStats and/or PortfolioStats). • Evaluate collaboration among key stakeholders. • Interview executive officials and key stakeholders • CIO, SPE, CFO, Program Management Officials. • Evaluate IT Investment Lifecycle Management structure • Identify investment thresholds, acquisition planning documents, and approvals required at key decision points within investment review process. • Identify internal controls implemented to ensure process is followed. • Attend an IRB meeting to verify that key decision-makers are in attendance, appropriate information is presented, and risk mitigation and alternative investment options are discussed. • Identify sample of major IT investments and collect key investment decision documents to evaluate comprehensiveness of investment review process and ensure appropriate actions are taken to minimize risk and document investment decisions.