1 / 15

Transitioning to Outcome-Focused Management

Transitioning to Outcome-Focused Management. Findings from a Cross-Agency Study. prepared for: National Environmental Partnership Summit 2006 Measurement and Targeting Session, Wednesday May 10, 2006. prepared by: Tracy Dyke Redmond Senior Associate Industrial Economics (IEc)

ray
Download Presentation

Transitioning to Outcome-Focused Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Transitioning to Outcome-Focused Management Findings from a Cross-Agency Study prepared for: National Environmental Partnership Summit 2006 Measurement and Targeting Session, Wednesday May 10, 2006 prepared by: Tracy Dyke Redmond Senior Associate Industrial Economics (IEc) tdr@indecon.com 617-354-0074 x 193

  2. Topics to Address • Study objectives and methodology • Background • Definitions • Case study programs • Findings • Reflections on themes from panel

  3. Study Objectives and Methodology • EPA: Benchmarking study to find best Fed. Agency practices for measuring results of: • stewardship programs • innovation programs • Data Source for Identifying Cases: OMB Program Assessment Rating Tool (PART) reviews • Phase 1: paper review of PART assessments • Phase 2: follow-up on three strong programs and comparison of EPA and other agency PART reviews

  4. Background: What is the PART? • Framework OMB uses to assess Fed agency programs • PART Questions look at: • Program purpose and design, • Strategic planning, • Program management, and • Program results. • PART supports Government Performance and Results Act (GPRA), which requires every federal agency to: • Develop strategic and annual plans, • Measure progress annual, and • Report annually.

  5. Definitions: What are “stewardship” and “innovation”? For the purpose of this review… • Stewardship programs: • cultivate a sense of responsibility for the natural environment, or • lead to actions that protect specific natural resources such air, water, and land. • Innovative programssearch for, test, develop, and/or promote adoption of practices that are more effective or cost effective in achieving agency objectives.

  6. Three Case Studies Model Outcome-Focused Programs • Coast Guard Marine Environmental Protection prevents and responds to marine environmental risks. • Department of State Global Educational and Cultural Exchanges strengthens international relations through cross cultural exchange. • Department of Transportation National Highway Traffic Safety Administration (NHTSA) saves lives, prevents injuries, and reduces traffic-related health care and other costs.

  7. Overarching Findings on Stewardship and Innovation Programs • Well-rated programs set stewardship goals and measure progress in terms of: • Outcomes (Coast Guard and NHTSA) • Attitudes (NHTSA and State) • No well-rated programs found that set or measure innovation goals, but … • Innovation readily occurs in programs that set outcome-focused goals and use outcome-focused measurements (all three cases)

  8. Finding 1: Performance Measurement Regularly Used for Program Improvement Performance measurement not just a reporting requirement, but integrated into program management. • Local managers use routinely collected data to prioritize risks and target problems. • NHTSA gathers pre-crash, crash, and post-crash data • HQ managers use data to confirm assumptions and test program impacts. • NHTSA re-evaluates rule effectiveness after 7 yrs.

  9. Finding 2: Program Goals Should be Clear, Outcome-Focused, Realistic, and Ambitious • Successful programs clearly define their goals and have at least one outcome measure tied to each goal. • Coast Guard – oil and chemical spills, nuisance species • NHTSA – traffic fatalities, seat belt use • State – survey of practices, such as following US news • Programs must balance having realistic and ambitious goals. • NHTSA struggled with overly ambitious goals • Shift from process to outcomes is hard, but worth it.

  10. Finding 3: Outcome Goals Foster Innovation • Establishing clear outcome goals with measures shifts staff perspectives about the best ways to do their jobs. • State – focus on participants’ entire experience • State – creation of alumni group • When staff were allowed flexibility to make program changes reflecting their new perspectives, it produced positive performance results. • Coast Guard – shift focus from compliance to oil spills

  11. Finding 4: Federal Programs Build Partnerships by Aiding and Informing States/Locals • Federal programs often must rely on state/local authorities to gather data and implement programs. • Feds can add value by: • Analyzing cross-state data and sharing it with states and local authorities (e.g., NHTSA’s motorcycle helmet analysis) • Testing programs to find which programs are most effective, and packaging results to promote those that work best (e.g., NHTSA’s Click-it or Ticket campaign)

  12. Finding 5: Program Evaluation Builds Stronger Programs • In-house evaluation builds stronger programs. • State’s evaluation office is in high demand • Successful programs set aside funds for evaluation. • State program offices voluntarily contribute to evaluation office’s budget • Evaluation is new and increasingly valued • Program evaluation can build on routine/daily performance measures. • NHTSA uses routine measures for look-back evaluations

  13. Finding 6: Innovative Programs Continuously Scan for Effective/ Cost-effective Approaches • Innovative programs look for evidence of effective practices from states, field offices, other countries, the literature, and their own experiments. • NHTSA gathers data about programs believed to reduce crashes and injuries • In looking for best practices, innovative programs focus on outcomes, and measure progress on this basis. • Coast Guard charts identify biggest contributors to risk, tactics change based on the data

  14. Finding 7: Stewardship Programs Measure Attitude Changes through Surveys • Longitudinal studies track broad attitude changes over time among key target audiences • State conducts regular surveys of program participants • Comparison groups are important to be able to demonstrate causality. • State compares attitude changes of participants to those who did not participate • Broad scale surveys for domestic programs may face ICR restrictions, but problems not found in cases.

  15. Reflections on Themes from Panel • How to use existing data to measure program performance • EPA can add value by compiling and analyzing data, highlighting trends, and identifying effective approaches • Approaches for gathering new data to measure program performance • Surveys can be helpful, especially when conducted regularly to establish baseline and trend data • Attribution - can programs "take credit" for observed changes in performance? • Control groups and before/after comparisons are useful in teasing out a program’s impacts

More Related