“Good Practices in Managing for Results” Workshop Santiago, Chile

1. Managing for Results At GAO “Good Practices in Managing for Results” Workshop Santiago, Chile October 27th and 28th , 2010 Benjamin Nelson Managing Director for Quality Office of Quality and Continuous Improvement

2. Outline • The GAO Context • Managing for Results Strategy • Strategic Planning • Integrated Quality Assurance Framework • Measures and Results • Monitoring

3. The GAO Context • Broad mandate • Large organization • Large volume and different product types • 80 or more senior executives can sign products • Active involvement in the legislative process • All performance audits follow the Yellow Book: • Generally Accepted Government Auditing Standards (GAGAS) • Well developed and transparent quality and risk management approach • 14 Teams carry out the work related to GAO’s strategic goals • Matrix management of engagements • Wide range of expertise among GAO staff

4. The GAO Context • Employed about 3,300 people, three-fourths in HQ, with a budget over $0.5 billion • Received over 950 Requests/Mandates • 95% of resources are spent on requests and mandates • 5% are spent on CGA work • Produced over 960 products • 192 testimonies • 600 reports and correspondence • 42 oral presentations • 127 other (guides, referrals, statements for the record, other correspondences) • Made 2,005 Recommendations • Historically about 80-85% implemented

6. Targets Set For Performance Measurement Internal Operations Results Clients People Financial and non-financial benefits Attracting and retainingstaff Getting work done Testimonies Products with Recommendations /implemented Developing, supporting, and using staff Quality of work life Timeliness Leading, recognizing, and listening to staff Note: Internal business process measures support the categories listed above.

7. Specific Categories of Work • Performance audits, which include program effectiveness and results audits, economy and efficiency audits, internal control assessments, and prospective analyses to • assess program or policy alternatives • advantages or disadvantages of proposals • best practices • Financial audits and attestation engagements • Non-audit services, which include • objective, rigorously researched information on a program’s condition, without the application of criteria • legal opinions • bid protest adjudications • technical assistance • investigations

8. One Quality Assurance System • While GAO undertakes a wide variety of engagements—using many different approaches and methodologies and a wide range of specialized skills—we have one quality assurance system. • The quality assurance system • Is comprised of a comprehensive set of clearly defined, well-documented, and transparent standards and control procedures, and • Assures compliance with Government Auditing Standards

9. Quality Assurance System • GAO’s Quality Assurance System provides reasonable assurance that GAO’s • Work is professional, independent, and objectively designed • Evidence is complete and reliable • Conclusions and products are fair and balanced • Recommendations are sound

10. Overview of Quality Assurance Framework • Our approach to quality is comprehensive and transparent • Designed around people, processes, and technology • Broader than the management of individual engagements • Core controls focus on engagement performance • Value- and risk-based and includes an appropriate degree of transparency about key policies, procedures, criteria, and access to the results of our work

11. Key Components of GAO’s Quality Assurance Framework • Quality Assurance Framework encompasses • GAO’s Core Values of Accountability, Integrity, and Reliability • Government Auditing Standards • Policy Manual • Professional Practices Advisory Committee • EAGLE: Electronic Assistance Guide for Leading Engagements

12. Quality Assurance Framework Structured summary of the key elements of GAO’s quality control system. • Helps the staff understand what they are to do and why

13. GAO’s Quality Assurance Framework for Assuring Compliance With Government Auditing Standards GAO’s Intranet QA Framework Leadership Mission, Standardsand Core Values Tone at the Top Independence Strategic Planning Portfolio & Risk Management Congressional Protocols Agency Protocols Human Capital Recruiting/ Hiring Assigning Staff Professional Development PerformanceManagement Advancement Engagement Performance Engagement Planning Consultations with Experts and Specialists Stakeholder Involvement Supervisionand Review Evidence Message Agreement Referencing Partner Concurrence Agency Comments Audit Documentation Public Reporting Monitoring/Policy Review Annual Inspections External Audit/ Audit Committee Peer Review Internal Audit Quality and ContinuousImprovement Assessments Professional Practices Advisory Committee

14. EAM Authority Independence Risk Level Cost Complexity Controversy Priority Internal Controls Stakeholders ERM Design Commitments toClients Draft Reports and Agency/3rd Party Views (all risk levels) Other issues (i.e., access, independence) HIGH MEDIUM Weekly Meeting Bi-weekly Meeting Risk Based Engagement Management Process Requests Mandates CGA Starts

15. GAO’s Quality Assurance Framework for Assuring Compliance With Government Auditing Standards Engagement Performance Engagement Planning Consultations with Experts and Specialists Stakeholder Involvement Supervisionand Review Evidence Message Agreement Referencing Partner Concurrence Agency Comments and Third-Party Views Engagement Documentation Public Reporting

16. Roles and Responsibilities Analysts Analysts in Charge Assistant Directors First Partners Stakeholders engagements Experts and Specialists Second Partners

17. Electronic Assistance Guide for Leading Engagements (EAGLE) • Standardizes engagement management process • Assures compliance with GAGAS, GAO Policy Manual, and Quality Assurance Framework • Provides “one-stop shopping” for auditors to access guidance, forms, templates, and points of contact for conducting GAO work

18. Electronic Assistance Guide for Leading Engagements (EAGLE)

19. Measures and Results – Performance and Accountability Report • 15 measures • Recommendation follow-up • Accomplishment Reporting

20. Annual Performance Plan

22. Results: Financial Benefits

23. Results: Nonfinancial Benefits

24. Results: New Products with Recommendations

25. Results: Past Recommendations Implemented

26. Clients: Testimonies

27. Clients: Timeliness 1 1 Since fiscal year 2004 we have collected data from our client feedback survey on the quality and timeliness of our products, and in fiscal year 2006 we began to use the independent feedback from this survey as a basis for determining our timeliness.

28. Monitoring and Review • Annual internal inspection • Peer review every 3 years • Annual audit by external auditors • Audit Advisory Committee • GAO’s Inspector General reviews • Annual internal assessments ofinternal controls

29. Annual Internal Inspection • Reports whether GAO’s quality control system is suitably designed and operating effectively • Conducted for products issued in a given calendar year • Deals with substance as well as process • Recognizes good practices • Based on criteria (GAGAS or policy)

30. Continuous Assessment Loop