MLC Open Forum Washington, DC September 16 th , 2010

1. Building a Performance Improvement System in a Large Urban Public Health Department: Linkages and Learning Collaboratives MLC Open Forum Washington, DC September 16th, 2010 Dawn Marie Jacobson, MD, MPH Director, Performance Improvement Los Angeles County Department of Public Health

2. Presentation Objectives • Provide an overview of performance improvement efforts at Los Angeles County Department of Public Health • Discuss linkages required for efficient analysis and reporting • Data • Standards/Benchmarks • Reports • Describe the DPH Performance Improvement Learning Collaborative (PILC)

3. Quality Improvement—LAC DPH Quality Improvement Division Office of the Medical Director Organizational Development and Training Nursing Administration Health Education Administration Public Health Investigation Physician Administration Oral Health Quality Improvement Functions • Performance Improvement • Professional Practice • Science Review • Service Quality Gunzenhauser JD, Eggena ZP, Fielding JE, Smith KN, Jacobson DM, Bazini-Barakat N. The Quality Improvement Experience in a High-Performing Local Health Department, Los Angeles County. Journal of Public Health Management and Practice, 2010, 16 (1): 39-48

4. Strategic Planning determine priorities and goals PerformanceMeasurement data management reporting Performance Improvement Projects Modified IHI Method for Improvement Other tools (RCA, Fishbone diagrams, etc) PerformanceImprovement—LACDPH Key Elements

5. Strategic Planning: Determine Priorities and Goals • What are the priority public health issues in Los Angeles County? • What are the behaviors and outcomes related to these issues that we want for people who live in LA County? • How can we measure these conditions?

6. LAC DPH—Strategic Planning • Quality Improvement Division • Public Health Measures required Mission and Vision Statements, Goal Setting, and Evidence-Based Strategies (2004-2007) • Office of Planning • Department-Level Strategic Plan (2008-2011) • Division and Program Level Strategic Plans • Office of Planning may provide technical assistance

7. Performance Measurement: Public Health Measures • The LAC DPH approach based on Mark Friedman’s “Results Accountability” • 32 operational units identified population health indicators linked to program performance measures to follow over time • Healthy People 2010 objectives often identified and used as the “Standard” to achieve over time • Organized by Essential Services of Public Health/NACCHO Standards/Accreditation Domains

8. Public Health Measures PERFORMANCE MEASURES (measures of program effort and output) POPULATION INDICATORS (measures of population-level health outcomes and behaviors) AND Public Health Measures

9. Selecting Indicators and Measures Population Health Program Performance Population Goals Goal 1 Population Indicators Indicator Indicator Effective Strategies Strategy 1 Strategy 2 Performance Goals Goal 1 Goal 2 Performance Measures Measure 1 Measure 2 NACCHO Standards Federal, State, or Local Guidelines Strategic Plan Healthy People 2010/2020 Community Guide Clinical Guide Other Sources

10. Decision-Making in Public Health: Evidence Review Tier1 Evidence

11. Decision-Making in Public Health: Other Rationale Tier 2 Other Rationale

12. Public Health Measures:Population Indicators • Longer life span • Increased quality of life • Increased health equity • Less disease • Less premature death • Healthier choices • Safer environment • Healthier homes POPULATION-LEVEL HEALTH OUTCOMES & BEHAVIORS

13. Public Health Measures: Performance Measures • Who are our clients? • Which services do we provide to our clients? • What evidence-based strategies will lead to positive change in our clients? • How can we measure if our clients are better off? • How can we measure if we are delivering services well?

14. Performance Measures • Policies Created • People Informed • Partners Engaged • Surveillance Performed • Investigations Completed • Increased Access to Services • Client satisfaction MEASURES OF PROGRAM EFFORT & OUTPUT

15. Example: Immunization Program • Population Goal To reduce morbidity and mortality from vaccine-preventable diseases by improving immunization levels • Population Indicator • Percentage of children, ages 19-35 months, who are fully immunized with one • of the series of the Advisory Committee on Immunization Practices (ACIP) • recommended vaccines • Effective, Evidence-Based Strategies (selected subset) • Change provider behavior through systems change—Provider recall/reminder systems in clinics • Change provider behavior through education—multi-component interventions with education • Increase demand and access to immunizations—reduce out-of-pocket costs Performance Goal (NACCHO Standard 9) Performance Measure Percent of Immunization Program public and nonprofit clinic partners who routinely meet the Standards for Pediatric Immunization Practices for provider and client recall/reminder systems

16. Public Health Measures:Data Management • Standardized spreadsheet for reporting data • Labeling System • Population Indicators= letters • organized by population goals • Program Performance Measures= numbers • organized by Accreditation domains • Data Documentation • Standard Documentation

17. Public Health Measures: Data Measurement Worksheet • Type of measure (PI or PM) • Measure name and description • How calculated • Target • Data source (Name, govt level, dept, program) • Data collection instrument • Data collection plan • NACCHO Standard (if applicable)

18. Population Indicators Examples: Los Angeles County Health Survey LA FANS Disease specific surveillance systems Vital Records CA Health Interview Survey OSHPD (Healthcare Utilization data) BRFSS YRBS National Immunization Survey Program Performance Measures Examples: Casewatch (STD, AIDS) RASSCLE (lead surveillance) EHMIS TRIMS (TB control) vCMR (outbreak reporting and investigation) Syndromic surveillance Clinic utilization data Contracts and grants management Project-specific databases Common Data Sources

19. Population Indicators Examples: Healthy People State of CA plans County of LA plans (e.g. Commission on HIV) Internal DPH Program Performance Measures Examples: Healthy People CDC guidelines State of CA guidelines or mandates Grant-specific guidelines Professional associations Internal DPH Common Standard Sources

20. Example: Immunization Program

21. Example: Immunization Program

22. Public Health Measures: Data Management • Data collected two times per year • Data analyzed and reported one time per year • Option to update content of Public Health Measures one time per year • Public Health Measures database in development

23. National Efforts CDC Guidelines or Performance Measures State Efforts State Performance Measures Mandates and Regulations County Efforts Performance Counts! County Progress Report Department Efforts Annual Performance Report PI Project Reports Public Health Measures: Reporting

24. LAC DPH Annual Performance Report • Internal report of a subset of Population Indicators and Performance Measures • Includes: • Department-Level Report Card • Program-Level Performance Snapshots • In-Person Progress Review with Director and Health Officer

25. Public Health Report Card

26. Public Health Report Card

28. Public Health Measures: Reporting • Future • Linked to strategic plan objectives • More frequent reports using an automated database • Portfolio of services by NACCHO Standards • Accreditation Preparation

29. Since repeated measurement by itself is not enough to improve public health practice.... Performance Improvement Projects What are common processes in our Department? How can we share best practices in common processes? What support do staff need to use PI methods (e.g. rapid cycle tests) in practice? How do we spread a successful PI approach throughout the Department?

30. Performance Improvement Projects A Learning Collaborative Approach* • Create an internal performance improvement learning collaborative (PI LC) of a diverse group of DPH units • Teams represent 8 of 32 department Divisions/Programs • Teams learn and work together for a 10 month period • Teams apply common PI methods to improve a priority area selected by their respective units *This project is part of the “Building the Evidence for Quality Improvement in Public Health ” grant program funded by the Robert Wood Johnson Foundation. The RAND Corporation is providing training and evaluation support.

31. Set the Aim Population health improvement Customer or service improvement Measure Performance Population Indicators Program Performance Measures Map the Process Make Changes for Improvement Evidence Review and Best Practices PDSA cycles Apply other Tools (RCA, Fishbone diagrams, etc.) PIMethod and Tools:The IHI Model for Improvement... Plus

32. Set the Aim: PI LC Team Aims • Improve Provider/Contractor Performance • Office of AIDS Policy and Programs • Children’s Health and Disability Prevention Program • Tobacco Control and Prevention Program • Substance Abuse Prevention and Control • Emergency Preparedness and Response • Inform and Engage Community Stakeholders • Office of Senior Health • Acute Communicable Disease Control • Office of Women’s Health

33. Measure Performance: PI LC Project Metrics • Population Indicators Examples: • % of children who qualify for the CHDP program who receive needed follow- up care • Community incident rates of reptile associated salmonella • Community rates of CVD among women • Program Measures Examples: • % of CHDP forms with a condition needing referral that have a referral identified • % of Early Childhood Education providers receiving the photonovela intervention • # of new callers to hotline per week

34. Percent of current adult smokers and current youth smokers in Los Angeles County (2001-2006) Data not collected where missing Data Sources: 1) Los Angeles County Health Survey (LACHS), LAC DPH, OHAE and 2) Youth Risk Behavior Survey (YRBS), CDC, NCCDPHP

35. Number of jurisdictions adopting a legislative-based policy that prohibits smoking in outdoor areas (00-01 to 09-10)

36. Map the Process: Examples

37. Make Changes for Improvement:PDSA and Other Tools PDSA cycles to improve subcontractor satisfaction with Task Force meetings (complete)

38. Make Changes for Improvement:PDSA and Other Tools PDSA cycles to improve subcontractor training for community campaigns to pass tobacco policies and ordinances (in progress)

39. Pareto Chart:Adult smoking rate by SPA

41. PI LC Evaluation Metrics • Improve project team metrics • Increase staff knowledge and use of QI methods • % of senior managers reporting they are aware of the 4-step Model for Improvement • % of senior managers who say they are proficient in selected quality improvement methods and tools • Disseminate QI methods across the Department • % of DPH Division/Program Directors who report using rapid-cycle PDSA to improve performance in a priority area each year • % of staff who report they are encouraged to take risks when implementing QI projects Based on: 1) Senior Manager Survey of QI culture, QI knowledge and readiness for change; 2) Key Informant Interviews of DPH Executives; 3) Monthly reports from the 8 PI LC project teams; 4) DPH Annual QI Report Card

42. PI LC Early Lessons Learned • Successes • Team engagement with learning sessions • Improved understanding of internal processes and links to key measures with process mapping • Individual team coaching • Challenges • Competing priorities (e.g., H1N1 response) • Doing rapid small scale cycles is a very new concept • Lack of readily available, validated measures and best practice tools for team aim areas

43. Performance Improvement Projects: Future Plans • Have all operational divisions and units working on at least one PI project each year • PI to track PI projects • Inclusion of key PI project measures in the Public Health Measures • Reporting PI project results • Monthly for selected programs • Yearly summary for all others

44. Annual Timeline Performance Improvement Projects Progress Reviews with Health Officer Performance Improvement Training

45. Summary • A large health department needs to link many sources of data, standards, and reporting processes to build an efficient performance improvement system • This takes time to do properly • Best with department-wide participation • Need to communicate effectively across levels of government and understand a wide variety of unit demands

46. Summary • A learning collaborative approach is essential to explore common processes and small tests of change • Brings PI champions together which generates enthusiasm • Maximizes learning and sharing • Promotes a culture of openness and transparency • Creates a “centralized” opportunity for technical assistance and coaching

47. Questions and Discussion