Performance Budgeting and Performance Management in the U.S. Government: Lessons from the PART Initiative

1. Performance Budgeting and Performance Management in the U.S. Government: Lessons from the PART Initiative John Pfeiffer U.S. Office of Management and Budget jpfeiffer@omb.eop.gov Presentation at a World Bank Brown-Bag Seminar June 6, 2007

2. Overview • Background • PART Process • PART Questions • Challenges • PARTWeb and ExpectMore.gov • How OMB Manages the PART Initiative

3. What is the PART? • The Program Assessment Rating Tool is a diagnostic tool used to assess program performance and to drive improvements. • The PART is designed to provide a consistent approach to assessing and rating programs across the Federal government. • PART assessments review overall program effectiveness, from design through implementation and results. • Once completed, PART reviews help inform budget decisions and identify actions to improve results. • Agencies are held accountable for implementing PART follow-up actions, i.e., improvement plans, for each program.

4. When We Began in 2002 • Many systems in place to collect and report data • Unclear relationship between strategic and annual goals • Tendency to measure what we could instead of what we should • Uneven attention to performance measurement • Lots of measures, but priorities not transparent • Performance data used more for reporting than decision-making

5. Where We Are TodayDistribution of Cumulative Ratings 2002 - 2006

6. PART Process • The PART questionnaire is divided into four sections: program purpose/design, planning, management, and results/accountability. • Answers must be clearly explained and cite relevant supporting evidence, such as agency performance information, independent evaluations, and financial information. • Answers translate into section scores weighted to generate an overall score and rating: Effective, Moderately Effective, Adequate, Ineffective. • Programs without performance measures or data are rated “Results Not Demonstrated.” • Additional questions are asked about particular types of programs: Block/Formula Grant, Capital Assets and Service Acquisition, Competitive Grant, Credit, Direct Federal, Regulatory-based , Research & Development.

7. 2007 PART Schedule (p. vii) • 2007 PARTs identified - January 19. • PARTWeb available for data entry - January 22. • Questionnaire guidance available - January 29. • PART training for OMB and agencies - mid February. • PART drafts due - March 30. • Consistency check & performance measures review - May 3-10. • OMB revises PARTs & passes back results to agencies - May 18 • Agencies submit appeals - May 25. • Summaries & improvement plans ready for ExpectMore.gov - July 9. • Data entry locked - August 3. • PARTs published on ExpectMore.gov - mid August.

8. PART Questions • Four sections • I. Program Purpose and Design (20%) • II. Strategic Planning (10%) • III. Program Management (20%) • IV. Results (50%)

9. Section I: Program Purpose and Design(pp. 16-22) • 20% weight of total score • Clarity and relevance of program purpose • Soundness of program design • Addresses program’s structural issues • Clear design and purpose an essential for identifying performance measures

10. Section II: Strategic Planning(pp. 23-37) • 10% of total score, with links to Section IV questions • Addresses program plans and approach to long-term goals • Programs must have long-term and annual performance measures and ambitious targets • Emphasizes independent, quality performance evaluations, plus budget transparency and budget-performance integration

11. Section III: Program Management(pp. 38-54) • 20% of total score • Addresses: • Accountability of managers, performance of partners • Coordination with related programs • Financial management and efficiency improvements • Correction of deficiencies • Do programs have procedures in place to measure and achieve efficiencies and cost effectiveness?

12. Section IV:Program Results/Accountability(pp. 55-61) • 50% of total score • Assesses achievement of long-term and annual performance and efficiency goals • Compares actual performance to targets (identified in Sections II and III) • Effectiveness in achieving goals based on independent evaluations • Compares with performance of similar programs

13. Performance Measures are Central to the PART • Allows tailoring to the specific program • Results are the most valuable information “product” of the PART • Takes most time in completing the PART • Biggest determinant of overall score and rating • Updated regularly to help keep PART information current

14. How PART improves performance measurement • Outcome-oriented long-term measures reflecting program purpose • Outcome-oriented annual measures that directly support long-term goals • If goals are outputs, must explain how they reflect progress toward desired outcomes • Challenging but realistic quantifiable targets and timeframes • Clear baseline from which to measure changes in performance • Credit in results section tied to measures in strategic planning section • Performance measures used to manage • Accountability for achieving performance goals

15. Where performance measures are today Of programs assessed • 86% have long term measures • 72% have ambitious targets for long-term measures • 87% have annual measures • 72% have ambitious targets for annual measures

16. Examples of Improved Measures Coast Guard Aids to Navigation • Old focus: Percentage of time radio navigational systems available • Current focus: Five year average of number of collisions, allisions, and groundings National Bone Marrow Donor Registry • Old focus: Number of donors in registry • Current focus: Number of transplants facilitated and post-transplant survival rate Leaking Underground Storage Tanks • Old focus: Number of clean-ups completed • Current focus: Number of clean-ups that exceed state risk-based standards for human exposure and ground water migration Community Health Centers • Old focus: Numbers and characteristics of persons served and services provided • Current focus: Heath outcomes such as low birth weight babies Small Business Development Centers • Old focus: Number of small businesses counseled or trained • Current focus: Number of jobs created (new businesses v. old businesses)

17. Performance Measures • Outcome: Events or conditions external to the program and of direct importance to the public, beneficiaries and/or customers. They relate to theprogram’s mission, purpose, and strategic goals. • Output: Internal program activities – products and services delivered to the public, beneficiaries. • Efficiency: Reflect economical and effective acquisition, use, and management of resources to achieve program outcomes or produce program outputs. • Outcome efficiency • Output efficiency • Input productivity

18. Performance Goals • Targets – Improved levels of performance needed to achieve stated goals. • Programs must have ambitious but realistic, achievable targets and timeframes for performance measures. • Together, measures, targets, and timeframes establish the program’s performance goals.

19. Program Evaluations • Scope- Examine underlying cause and effect relationship between program and achievement of performance targets. • Independence- Performed by non-biased parties with no conflict of interest. • Quality • Applicability – All programs expected to undergo some type of evaluation. • Impact – Prefer effectiveness evaluations (outcome, e.g., whether Federal intervention makes a difference). • Rigor– The most rigorous evidence that is appropriate and feasible for that program.

20. Does It Ever End? • Steps after PARTs are completed • Draft summaries for ExpectMore.gov • Spring Updates in PARTWeb • Complete Improvement Plans • All programs must have, regardless of PART rating • Focus on findings in the PART assessment • Implement plans and report on progress • ExpectMore.gov release mid-August

21. Challenges: Lessons to Learn Quickly • Share drafts, communicate frequently. • Use clear, direct language. • Stick to deadlines. • Don’t take the PART personally. • Rely on evidence, not anecdotes.

22. Challenges: Measurement • Uneven quality of performance measures in PARTs • Several areas difficult to measure • Increasing the timeliness of performance reporting • Consistency: agencies and OMB answer some questions differently

23. Challenges: Program Evaluation • Want to promote evaluation to measure and improve program design, implementation, and effectiveness, including cost-effectiveness.. • Evaluations are not used enough to assess impact and improve performance • Decision makers do not appreciate and, consequently, do not routinely invest in evaluations. • Technical complexity can make them hard to understand and thus undermine confidence in results.

24. Challenges: Improvement Plans • Aggressiveness varies • Unclear how they impact program results (versus PART score) • Uneven attention to plans across agencies and OMB

25. Challenges: Improving Performance • Improving PART score versus improving performance • Ensuring that program managers are empowered and accountable • Assessing improvement plans fairly • Sharing good approaches and models

26. Challenges: Impact • Executive Branch • Management, funding, or authorization decisions are not regularly based on the PART • President’s Management Agenda Budget and Performance Integration initiative is being used to leverage greater use of PART results • Congress • Rare, diverse references to PART • Not the basis for legislative action • Few oppose vigorously • Crosscutting • Opportunity for collaboration among like programs

28. ExpectMore.gov Summary

29. PART Resources Online • www.omb.gov/part • Information on process and schedule • Guidance for completing PART • PARTWeb link, user’s manual • Supporting materials • www.ExpectMore.gov

30. How OMB Manages the PART Initiative

31. President’s Management Agenda A strategy for improving Federal management and performance with five government-wide and nine agency-specific goals. Strategic Management of Human Capital Competitive Sourcing Improved Financial Performance Expanded Electronic Government Budget and Performance Integration The President directed agency heads to designate a “Chief Operating Officer“ for day-to-day operations. The President designated the President’s Management Council (PMC) as an integrating mechanism for policy implementation across government, headed by OMB’s Deputy Director for Management and comprised of the COOs.

33. BUDGET AND PERFORMANCE INTEGRATIONCriteria for Achieving GREEN Senior managers meet at least quarterly to examine reports integrating financial and performance information for all major Department responsibilities. Agency works to improve program performance and efficiency each year; Strategic plans contain a limited number of outcome-oriented goals and objectives. Annual budget and performance documents incorporate measures identified in the PART and focus on the information in the senior management report; Reports the full cost of achieving performance goals accurately in budget and performance documents and can accurately estimate the marginal cost of changing performance goals; Has at least one efficiency measure for all PARTed programs; Uses PART evaluations to direct program improvements and hold managers accountable for them, and uses PART findings and performance information to justify funding requests, management actions, and legislative proposals; and Less than 10% of agency programs receive a Results Not Demonstrated rating for two years in a row. To maintain green status, agency: Improves program performance and efficiency each year; and Uses marginal cost analysis to inform resource allocations, as appropriate.