1 / 38

Performance Measurement: The nuts and bolts of establishing effective measures

Performance Measurement: The nuts and bolts of establishing effective measures. Mary Davis, DrPH , MSPH; NC Institute for Public Health Kristin Adams , Ph.D., CHES; Indiana State Department of Health. Performance Measurement Definitions.

lita
Download Presentation

Performance Measurement: The nuts and bolts of establishing effective measures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Performance Measurement: The nuts and bolts of establishing effective measures Mary Davis,DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health

  2. Performance Measurement Definitions Performance measurement is the “regular collection and reporting of data to track work produced and results achieved” Performance measure is “the specific quantitative representation of capacity, process, or outcome deemed relevant to the assessment of performance” Performance measurementis NOT punishment

  3. Organizational Performance Monitoring Indicators and Outcomes Monitoring the results of performance review against PHAB Standards and Program Evaluation

  4. Mary Davis, UNC Dr. Mary Davis is a Senior Investigator at the NC Institute for Public Health and Adjunct Assistant Professor in the Department of Health Behavior, both part of the UNC, Gillings School of Global Public Health. Since Dr. Davis conducts public health systems and services research and program evaluations using participatory approaches; specialty areas are accreditation, quality improvement, and public health partnerships. Her experiences in these areas have been shaped by participating in the Multi-State Learning Collaborative, evaluation of the NC Local Health Department Accreditation program, and working with the NNPHI and CDC to evaluation the National Public Health Improvement Initiative. Dr. Davis earned her doctorate from the Johns Hopkins University School of Public Health and her masters in science in public health from the University of Alabama at Birmingham.

  5. Kristin Adams, ISDH Dr. Kristin Adamsis the Director of the Office of Public Health Performance Management at the Indiana State Department of Health (ISDH).  She oversees agency performance management, quality improvement, and future public health accreditation efforts.  She was the Principal Investigator in Indiana for the Multi-State Learning Collaborative grant. She is currently the Principal Investigator for the National Public Health Improvement Initiative cooperative agreement from CDC.  Prior to joining ISDH, Dr. Adams worked for a national non-profit agency focusing on program evaluation and training for health programs targeted toward girls and young women. Dr. Adams has taught numerous health education courses and has presented at national and international conferences.  Dr. Adams holds a BS and MA in School Health Education from Indiana State University, a Ph.D. in Health Education from Southern Illinois University Carbondale, and is a Certified Health Education Specialist.

  6. Performance Measurement: Nuts and Bolts Basics Mary V. Davis, DrPH, MSPH Open Forum for Quality Improvement November 20, 2013

  7. Outline • What is a performance measure? • Where are performance measures in a performance management system? • What should be in place before establishing measures? • What’s a system of work? • What are the characteristics of a useful process measure? • How do you go about establishing useful process measures?

  8. What is a Performance Measure? • Performance Measurement is regular collection and reporting of data to track work produced and results achieved. • Performance Measure is quantitative representation of • Capacity • Process • Outcome

  9. Where are Performance Measures in a Performance Management System? YOU ARE HERE!

  10. Standards and Measures: PHAB Example • Standard 9.1: Use a performance management system to monitor achievement of organizational objectives. • Measure 9.1.2 A Implement a performance management system. • Use a process to determine and report on achievement of goals, objectives, and measures set by the performance management system.

  11. What Should be in Place Before Measures?

  12. Results help to inform the planning process Performance Monitoring and Analysis Results and Customer, Stakeholder, Partner Feedback Vision Mission and Values Goals and Objectives Strategies Strategic Planning Performance Measures and Operational Indicators Operational/ Business Plans Improvement Opportunities Quality Improvement Projects and Process Improvement Opportunities Improvement Opportunities Performance Monitoring and Analysis Washington State Department of Health Example:

  13. Line of Sight Framework Reduction in food-borne illnesses and disease outbreaks Public uses restaurants with acceptable sanitation grades restaurants achieving acceptable sanitation grades can post grades Long term outcome Restaurant inspections conducted and reports produced Short term Outcome Process Outcome Work Process Measure

  14. What is a System of Work? Factory: How? Widget: What? Customers: Who? Outcomes: Why?

  15. Focus on the Widgets • What’s a Widget?

  16. Widgets • Are Things • Are Deliverables • Can be Counted • Are Specific

  17. Common Public Health Agency Widgets • Environmental Permits, Notices of Violations • Rules and regulations • Vaccinations • Epidemiology Reports • Infrastructure • Filled Vacancies • Contracts

  18. What Widgets Do You Produce? • Take a minute: write down 5 widgets you produce • What examples do you have to share?

  19. Measures: General Considerations • Measures reveal our values • Measures drive behavior • Measures can inspire us • Measures can help us learn

  20. Useful Measures: Specifics • Meaningful • Focused on customer needs • Accurate • Valid • Reliable • Responsive • Functional • Credible • Simple enough to be understood • Available • Abuse-Proof • Cost effective to collect and report • Comparable data over time

  21. SMART Measures • Specific • Measurable • Aggressive • Results-Oriented • Time-bound

  22. Developing Measures • Talk with customers and those involved in the process • Identify which process measures • Are of value to customers and producers • Will drive behavior • Will inspire • Will promote learning • Determine how process measures fit into other measurement processes in the agency (performance standards, strategic plan)

  23. What to Include in a Measure • Statement of the Measure • Target (population, process) • Numerator and denominator • Target goal • Who will collect the information

  24. What to Include in a Measure • Form or tool to collect information • How often data will be collected • Who will conduct data analysis • How often will the data be reported

  25. MarMason Consulting Data Description Form

  26. Resources • “We Don’t Make Widgets” Ken Miller • Turning Point Resources: Guidebook for Performance Measurement • Marni Mason tools and presentations • Washington State Department of Health QI Plan

  27. Thank You!Mary V. Davis, DrPH, MSPHmary_davis@unc.edu

  28. Getting to the 500 foot level of Performance Measures Kristin A. Adams, Ph.D., CHES

  29. Setting Goals/Objectives • Ask a series of questions: • Who is your customer • What do they care about • What is the purpose of your funding/mission/vision of the program area • Does this goal and objective tie to the strategic plan • Does your program have a role in the key priorities of the agency • Can you measure it • Is it meaningful???

  30. What should we measure? • Several areas of resources to determine measures: • State law/code • Program requirements • Healthy People 2020 • Historical data of the program • Benchmarks by national groups (i.e., March of Dimes)

  31. How Do We Get There • Develop the key outcome measure • Decrease Smoking Rates • Decrease Obesity Rates • Increase Immunization Rates • Decrease the late payment fees • Identify key activities to get you to making a change in the measure • Increase number of people receiving coaching from quitline • Increase the complete streets communities throughout the state • Reminder recalls--% of those who become compliant due to the reminder notice • Decrease invoice wait time; increase invoice processing time

  32. Development of a Dashboard Sometimes it is all about Process

  33. Lessons Learned • Ensure your epidemiologists are involved • Ensure the experts to the program are involved • If you are doing a “data pull” ensure the IT/database manager is involved • Involve Executive Leadership • Do a “mass” education campaign

More Related