The Big Picture Role of Evaluation in Public Health Practice

1. The Big PictureRole of Evaluation in Public Health Practice Luann D’Ambrosio, MEd

2. Activity • Think of your favorite quote or sage advice that helps guide you through your professional work.

3. I hear and I forget. I see and I believe. I do and I understand. —Confucius (551–479 BC) Wise Advice

4. CDC Evaluation Framework Standards Utility Accuracy Propriety Feasibility

5. Objectives • Describe the relationship between performance management, program planning and evaluation • Describe the steps of program planning and evaluation process • Identify key stakeholders in an evaluation • Develop the components and elements of a program logic model • Create a basic evaluation plan including data collection and analyzing methods • List three ways to use evaluation data to draw conclusions about a program • Define ROI, cost benefit ratios and social return on investment.

6. Accreditation is a major accomplishment for a health department. It means that it is addressing key community health problems. Just as the public expects hospitals, law enforcement agencies, and schools to be accredited, so should they come to expect public health departments. Thomas Frieden, MD, MPH CDC Director Accreditation

7. CDC Grants CDC awards grants for public health performance management systems • $42.5 million allocated for 94 projects in state, local, tribal, and territories health departments • Distributed in 49 states, 16 tribes and territories and nine local jurisdictions • Five-year cooperative agreement awards are being funded by the health care reform law

8. Benefits of Accreditation Accountability & credibility Tool for improvement Highlights HD strengths Greater collaboration Accreditation+ Team building Recognition & validation Better understanding of public health

9. Strategic Planning Phase 4 Evaluation Monitoring Implementation Budgets Action Plans Strategies Phase 3 Strategic Issues Goals & Objectives Phase 2 Vision Mission Values SWOC Analysis & Environmental Scan Plan the Plan Readiness Assessment Stakeholder Assessment Mandate Analysis Phase 1

10. Performance Management (PM) A systematic process aimed at helping achieve an organization’s mission and strategic goalsby improving effectiveness, empowering employees, and streamlining the decision-making process.  Performance Behavior Results = +

11. They Are Not the Same

12. They Are Not the Same

13. Value of Performance Management • Disciplined approach • Systemic • Systematic • Engaged customers and staff • Results focused • Proven method

14. Culture of QI

15. Performance Management Frameworks • Baldrige • Six Sigma • Lean

16. PDSA Cycle What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? Act Study Plan Do

17. Memory Jogger

18. Tools Logic models and work flow charts Data and analysis tools • Customer-supplier relationships • Client flow, information flow • Root cause tools: fishbone diagram, Pareto chart • Force field analysis • Interrelationship digraph

19. PERFORMANCE STANDARDS • Identify relevant standards • Select indicators • Set goals & targets • Communicate expectations • PERFORMANCE MEASUREMENT • Refine indicators & define measures • Develop data systems • Collect data PERFORMANCE MANAGEMENT SYSTEM • QI PROCESS • to improve policies, programs, & outcomes • Manage changes • Create learning organization • REPORTING OF PROGRESS • Analyze data • Feed data back to managers, staff, policy makers, constituents • Develop a regular reporting cycle • QI PROCESS • Use data for decisions Turning Point Performance Management Collaborative, 2003

20. Questions?