Breakout Session # 504

2. How DOE Assesses its Acquisition System under OMB Circular A-123 Using the GAO Framework Breakout Session # 504 Michael P. Fischetti, Director, Office of Procurement and Assistance Policy Department of Energy Date April 15, 2008 Time 10:45 am

3. Agenda • Introduction • Background • DOE’s Balanced Scorecard Approach • What is the GAO Framework? • The Cornerstones • The Evaluation Team • Performing the Assessment • OMB Circular A-123: DOE’s Approach and Results • An Example • Final Outcome • Summary • Compliance Model

4. Introduction • DOE is striving to be removed from the “High Risk List” for contract management • DOE addressing GAO agency criteria for removal • Demonstrate strong commitment and leadership • Demonstrate progress in implementing corrective measures • Develop a corrective action plan that identifies • Root causes • Effective solutions • Near-term plan for implementing the solutions

5. Background • DOE learned of the “Framework” in September 2003 • Obtained a draft for agency comment • Monitored development with GAO through FY2004 until formal publication in September 2005 • Developed evaluation criteria in the form of questions • Completed an internal assessment during FY2005 and FY2006 • Monitored use of “Framework” criteria as part of GAO audits of TSA, INS, and DHS procurement functions

6. DOE’s Balanced Scorecard Approach • Maintaining BSC performance measures in four strategic assessment areas: • Customer Perspective • Internal Business Perspective • Learning and Growth Perspective • Financial Perspective

7. DOE’s Balanced Scorecard Approach • DOE initiated an internal BSC project to assess its procurement system using the “Framework” developed by the GAO to assess the health of its procurement system • Identified areas requiring management attention or early mitigation • Followed up with an action plan for achievement • Incorporated into existing FY-2006 Balanced Scorecard (BSC), self-assessment procurement performance reviews, and management system

8. DOE’s Balanced Scorecard Approach • Objective • Using “Framework” developed a DOE • Risk evaluation model for internally assessing DOE’s procurement system • Approach for implementation • Approach • Review the DOE BSC Self Assessment Guide against the “Framework” • Coordinate with the BSC Program Manager and Senior Procurement Management • Assess current procurement functions • Establish permanent process for proactive, continuous evaluation of procurement areas

9. DOE’s Balanced Scorecard Approach • Documentation was: • Collected • Independent evaluation performed • Final report completed • Remediation conducted providing iterative improvement • Follow-up actions: • Internal management control assessments under OMB Circular A-123 guidelines • Development of follow-on BSC initiatives addressing lower-rated areas

10. What is the GAO Framework? • Cornerstones • Organizational Alignment and Leadership • Policies and Processes • Human Capital • Knowledge and Information Management • Elements • Critical Success Factors • Key Questions, Things to look for, Cautions

11. The Cornerstones • Organizational Alignment and Leadership • Aligning Acquisition with Agency Mission and Needs • Commitment from Leadership

12. The Cornerstones • Plans and Processes • Planning Strategically • Effectively Managing the Acquisition Process • Promoting Successful Outcomes of Major Projects

13. The Cornerstones • Human Capital • Valuing and Investing in the Acquisition Workforce • Strategic Human Capital Planning • Acquiring, Developing, and Retaining Talent • Creating Results-Oriented Organizational Cultures

14. The Cornerstones • Knowledge and Information Management • Identifying Data and Technology to Support Acquisition Decisions • Safeguarding the Integrity of Operations and Data • Not part of review

15. The Evaluation Team • Selected senior procurements analysts new to the Department • Why? • Not familiar with existing processes • Significant diversity of procurement knowledge and experience • This is not just a “file review” • Benchmarking against other organizational experience • Objectivity • Dedicated resources

16. Performing the Assessment • Understanding the “Framework” • Collecting documentation (biggest challenge) • Benchmarking other organizations • Document • Adhere to principles in OMB A-123 • Stay on schedule

17. OMB Circular A-123: DOE’s Approach and Results “Agencies should strive to integrate control related activities within the control framework outlined in A-123. In particular, management should identify opportunities to integrate and coordinate in order to leverage the internal reviews already being performed” (Implementation Guide dated July 2005) • A-123 provides a vehicle to assess and evaluate internal controls • Assurance they are well designed and operated • Appropriately updated to meet changing conditions • A-123 assesses internal controls for: • Effectiveness and efficiency of operations • Reliability of financial reporting • Compliance with laws and regulations • OFPP circulated draft Guide for using the Framework for review • DOE provided comments to the draft based on its experience

18. OMB Circular A-123: DOE’s Approach and Results • DOE developed: • Test plan • Tested pre-defined areas for impact, likelihood of occurrence, and design effectiveness • Test sub-categories included: • Management’s commitment to competence • Management of Human Capital • Proper execution of transactions and events • External relevant, reliable, and timely communications • Risk analysis and actions

19. OMB Circular A-123: DOE’s Approach and Results • The Office of Procurement and Assistance Management provided reasonable assurance that: • Internal controls for financial reporting were working effectively • There were no material weaknesses identified in the design or operation of the specific Headquarters controls over financial reporting evaluated

20. An Example • Human Capital: Having the capacity (people and resources) to resolve the problems • Framework Assessment • Demographic trends analysis • Educational Levels • Internal training database (now migrated to ACMIS) • Funding levels • Position Descriptions • Risks identified • Recruitment, retention, and training of the workforce

21. An Example • Responses to identified risks translated to BSC initiatives • “Talent Management Plan” • Human Capital Survey • Survey • Conducted demographically representative focus groups with a summary report to the Director • Leadership Interviews • Documentation review

22. An Example • Responses to identified risks translated to BSC initiatives • Department-wide Acquisition Career Development (ACD) Policy and Program Handbook (revised and issued) • Outlines certification requirements for the acquisition workforce • “Acquisition Letters” issued • Strengthened organizational relationships between Department Headquarters and field operational contracting elements, including re-emphasizing ACD policy • Performed Skill-Gap Analysis of 1102 workforce • Met with DOE IG in drafting an Advisory Memo to the Deputy Secretary * See the follow-on briefing on DOE’s ACD program

23. Final Outcome Continuous Monitoring of Procurement System Effectiveness • “Monitor and independently validate the effectiveness and sustainability of corrective measures” • BSC continuous review and identification of system improvements • Using the “Framework” tool • Conducting periodic reviews and monitor field operations self- assessments of contracting office operations • Implementing field Procurement Management Review • Review Contract Management Plans • Review contractor purchasing systems

24. Final Outcome • Support interagency working groups in procurement and acquisition regulatory development • Monitor procurement risks, issues, or vulnerabilities of other organizations • Government Accountability Office (GAO) reports • Agency Inspector General (IG) reports • Internal financial and management audits and reviews • Professional and trade association periodicals • Procurement news sources or other documents • Benchmarking/surveying other agencies • Validation under A-123 Review

25. Summary • Corrective Feedback using BSC • Use of the GAO Framework • Reporting and compliance under OMB Circular A-123, as implemented by the DOE Chief Financial Officer

26. Headquarters Strategic Direction Balanced Scorecard Procurement Management Reviews Best Practices * * * * Policy Development PMRs Contracting Activities complete Compliance Self Assessment and PMR Checklist Compliance Self Assessment and PMR Compliance Self Checklist revised as necessary Assessment team and PMR review Compliance Self Assessment and PMRs summary and trend data compiled Compliance Self Assessment and PMR - Assessment Team recommendations Compliance Model

27. OMB Circular A-123DOE Next Steps • Implementing the “Framework” tool and A-123 in the field • Establishing policy/oversight/review functions in field offices • Procurement Management Review Program