Performance Measurement in Pediatric Emergency Care

1. Performance Measurement in Pediatric Emergency Care Evie Alessandrini, MD, MSCE Center for Health Care Quality Division of Emergency Medicine Cincinnati Children’s Hospital Medical Center

2. Disclosure I have no relevant financial relationships with the manufacturers of any commercial products and/or provider of commercial services discussed in this CME activity.I do not intend to discuss an unapproved or investigative use of a commercial product/device in my presentation.

3. Performance Measurement Learning Objectives • Delineate the different purposes of performance measurement • Become familiar with existing endorsed performance measures of emergency care • Develop a framework for organizing pediatric emergency care performance measures for your ED

4. Measurement Motivators • Health Care System • Growing complexity and costs • Transparency initiative, external mandates • Institute of Medicine Reports • To Err is Human, 1999 • Crossing the Quality Chasm, 2001 • Performance Measurement: Accelerating Improvement, 2005 • The Future of Emergency Care, 2006 • Develop national standards for emergency care performance measurement

5. Outcomes Quality

6. Why Measure Performance? Stakeholder Survey Analysis

7. Why Measure Performance? Stakeholder Survey Analysis • Improve • Health and Healthcare • For patients and populations • Within one ED or with one practitioner • Within networks of EDs or health systems • Inform • Transparency, consumer decision-making • Regionalization of care • Incentivize • Pay for performance • National rankings

8. Performance Measurement:Accelerating Improvement2005 IOM Report • The ultimate purpose of performance measurement is to improve the health of everyone in the United States • Performance measures are yardsticks by which all health care providers and organizations can determine how successful they are in delivering recommended care and improving patient outcomes • Public reporting of performance data holds health providers accountable to both consumers and purchasers of care; transparency builds trust • Patients can also learn what the expected professional standards of care are and where they can go to receive it

9. Performance Feedback Feedback has proven most effective when • rates of adherence to practice guidelines are low • the information is directly useful for care • practitioners are motivated to change IOM. Rewarding Provider Performance: Aligning Incentives in Medicine. Washington DC: American Academy Press; 2006.

10. Feedback Report: Appropriate Use of Amoxicillin for Newly Diagnosed Acute Otitis Media (AOM) Improving Otitis Media Care with Clinical Decision Support and Feedback; AHRQ R18 HS017042

11. Feedback Report: Appropriate Use of Amoxicillin for Newly Diagnosed Acute Otitis Media (AOM) Improving Otitis Media Care with Clinical Decision Support and Feedback; AHRQ R18 HS017042

12. Transparency “Transparency aims to reduce specific risks or performance problems through selective disclosure by organizations. The ingeniousness of target transparency lies in its mobilization of individual choice, market forces, and participatory democracy.” Wikipedia Levels of transparency • Within your department • Within your hospital • Outside your hospital Examples • http://my.clevelandclinic.org/Documents/Medicine/PEDS283076A_LR.pdf • http://www.dartmouthatlas.org

13. Pay for Performance “Pay for performance systems link compensation to measures of work quality or goals” Perfect Asthma Care • The pay-for-performance program rewarded practices for • participating in an improvement collaborative • achieving network- and practice-level performance thresholds • building improvement capability • The percentage of the network asthma population receiving "perfect care" increased from 4% to 88%. Mandel, KE; Archives of Ped and Adol Med: 161(7): 650-5, 2007 July

14. Leadership & Performance Measurement “Turning Doctors into Leaders” • Organize doctors into teams • Measure performance by patient outcomes, not resource use • Apply financial and behavioral incentives • Dismantle dysfunctional cultures Harvard Business Review, April 2010; hbr.org

15. Performance Measurement Learning Objectives • Delineate the different purposes of performance measurement • Become familiar with existing endorsed performance measures of emergency care • Develop a framework for organizing pediatric emergency care performance measures for your ED

16. National Quality Forum2008 Measures Hospital-based Emergency Care • Median time from ED arrival to ED departure for admitted patients* • Median time from ED arrival to ED departure for discharged patients* • Admit Decision Time to ED Departure Time for Admitted Patients* • Door to provider • Left without being seen * Measures stratified by • Psychiatric diagnoses, observation patients, transfers, all others www.qualityforum.org

17. National Quality Forum2008 Measures Hospital-based Emergency Care (cont.) • Severe Sepsis and Septic Shock: Management Bundle • Confirmation of endotracheal tube placement • Percentage of patients with Chest Pain Symptoms in ED receiving Early Therapy including IV, Oxygen, Nitroglycerin, Morphine and Chewable Aspirin on Arrival • Pregnancy test for female abdominal pain patients • Anticoagulation for Acute Pulmonary Embolus Patients • Pediatric Weight documented in kilograms • AAP is the measure sponsor www.qualityforum.org

18. Children’s Health Corporation of AmericaExisting Measures Whole System Measures • ED Left Without Being Seen • ED Length of Stay www.chca.com

19. Urgent MattersRobert Wood Johnson Foundations Urgent Matters Initiative Goals • Rigorously evaluate the implementation of strategies for improving patient flow and reducing ED crowding • Advance the development of standard performance measurement in the ED • Promote spread of promising practices to a wider audience / variety of hospitals www.urgentmatters.org

20. Urgent MattersRobert Wood Johnson Foundations Performance Measures (NQF and CMS) • Throughput for admitted patients • Throughput for discharged patients • Time to pain management for long bone fractures • Admitted patients • Discharged patients • Median time to chest x-ray • Admitted patients • Discharged patients • Admit decision time to ED departure time www.urgentmatters.org

21. Other Existing Measures and Measurement Organizations • Joint Commission • ORYX performance measures • Children’s Asthma Care measures (inpatient) • http://www.jointcommission.org/PerformanceMeasurement • AHRQ Pediatric Quality Indicators (PDIs) • 18 risk-adjusted measures • Obtained from inpatient administrative data • www.qualityindicators.ahrq.gov/pdi_overview.htm • Alliance for Pediatric Quality • AAP, American Board of Pediatrics, CHCA, NACHRI • www.kidsquality.org

22. Other Existing Measures and Measurement Organizations ACEP National Report Card on the State of Emergency Medicine • Access to Emergency Care • Quality and Patient Safety Environment • Medical Liability Environment • Public Health and Injury Prevention • Disaster Preparedness http://www.emreportcard.org

23. Performance Measurement Learning Objectives • Delineate the different purposes of performance measurement • Become familiar with existing endorsed performance measures of emergency care • Develop a framework for organizing pediatric emergency care performance measures for your ED

24. Rationale for Framework Limitations of prior work • Single centers or geographic locales • Focus on condition-specific indicators • Preponderance of process-oriented measures • Benchmarks very focused on • Timeliness (through put) • Satisfaction (ceiling effect) • Lack of comprehensiveness regarding spectrum of ED care • Lindsay et. al., AEM, 2002 • Guttmann et. al., Pediatrics, 2006

25. Performance Measure Framework Quality indicator set development process • Adapted from AHRQ • “Defining Quality Performance Measures for Pediatric Emergency Care” • Funded by HRSA/EMSC Targeted Issues Grant H34MC08512

26. Performance Measure Development SOURCES Candidate Measures N = 60 Research Literature Measure by Measure Evaluation for Selection Actual Use Concept Selection Criteria Measure Set Evaluation for Application Adapted from AHRQ PDI development process

27. Performance Measure Framework Measure development dimensions • IOM Quality Domains • Donabedian’s framework for quality • PEM disease frequency and severity Measure evaluation dimensions • National Quality Forum criteria

28. Institute of Medicine Quality Domains Built around the core need for health care to be • Safe • Effective • Efficient • Timely • Patient-centered • Equitable

29. Institute of Medicine Quality Domains Safe • Health care avoids injuries to patients from the care that is intended to help them Effective • Health care provides services based on scientific knowledge to all who could benefit, and refrains from providing services to those not likely to benefit

30. Institute of Medicine Quality Domains Efficient • Health care avoids waste, including waste of equipment, supplies, ideas and energy Timely • Health care reduces waits and sometimes harmful delays for both those who receive and those who give care

31. Institute of Medicine Quality Domains Patient - centered • Health care provides care that is respectful of and responsive to individual patient preferences, need and values, and ensures that patient values guide all clinical decisions Equitable • Health care provides care that does not vary because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status

32. Donabedian’s Framework Structure • Indirect quality-of-care measures related to a physical setting and resources: Staff, space, supplies, equipment and financial resources Process • Measures evaluate the method or process by which care is delivered, including both technical and interpersonal components Outcome • Outcome elements describe valued results related to lengthening life, relieving pain, reducing disabilities and satisfying the consumer

33. PEM Disease Frequency & Severity • Condition-specific • Proportion of patients with croup receiving corticosteroids • General • Proportion of patients returning to the ED within 72 hours of an initial ED visit • Cross-cutting • Proportion of patients with pain who receive an analgesic

34. Choosing Condition-Specific Measures www.pecarn.org/tools Alessandrini et.al., Academic Emerg Med; February 2010

35. Measure Evaluation Criteria Importance • The measure reflects a priority or high impact aspect of healthcare • The measure addresses outcomes or is strongly linked to improving outcomes • The measure addresses an area of considerable variation or poor performance across providers or population groups

36. Measure Evaluation Criteria Scientific Acceptability • There is strong evidence for the specific measure focus, such as evidence based guidelines • The measure is reliable, reproducible and accurately represents quality of care

37. Measure Evaluation Criteria Usability • The measure provides information that is actionable and can be used to make decisions that improve the quality of care • The measure is meaningful and understandable

38. Measure Evaluation Criteria Feasibility • Data for the measure is generated during care delivery and is available in the EHR or other electronic sources • Data collection for the measure can be implemented • The information provided outweighs the costs/burdens of collecting the data

39. Results: Performance Measure Distribution by IOM Quality Domain Equitable – measures stratified by gender, age, race, ethnicity and payor

40. Results: Performance Measure Distribution by Donabedian Framework

41. Results: Performance Measure Distribution by Diagnosis Type Cross-cutting measures include pain/sedation, severe illness, diagnostic testing and medication management

42. PEM Balanced Report Card

43. Further Considerations • Measures valuable to patients • Not minimum level of competency • Composite measures • Conceptual and analytic issues • Unit of analysis • Measure target • Locus of control • Balancing measures • Are we improving parts of our system at the expense of others?

44. Measurement & Quality • Quality is central to achieving affordable care that knows patient needs and keeps them healthy  • It’s a three step process • The first step toward achieving quality is convening expert members across the healthcare industry, including patients to define quality with uniform standards and measures that apply to the many facets of care patients receive.  • Second, information gleaned from measuring performance is reported and analyzed to pinpoint where patient care falls short.  • Third, caregivers examine information about the care they are providing and use it to improve. Measure. Report. Improve.

45. References • AAP Policy Statement: Principles for the Development and Use of Quality Measures • Pediatrics 121 (2), February 2008, pp 411-418 • Pediatric Clinics of North America “Pediatric Quality”: Quality Measures in Pediatrics • Volume 56 (4), August 2009, pp 816-829

46. References • Institute of Medicine Report: Performance Measurement, Accelerating Improvement • December 2005 • www.iom.edu/Reports/2005/Performance-Measurement-Accelerating-Improvement.aspx • Joint Policy Statement—Guidelines for Care of Children in the Emergency Department • Pediatrics 2009;124:1233–1243

47. Steps in Measure Specification • Numerator statement • Denominator statement • Denominator exclusions • Data source and collection methods • Sampling • Risk adjustment • Stratification to detect disparities • Level of measurement / analysis

48. Steps in Measure Specification Risk Adjustment • Accounts for patient-associated factors before comparing outcomes across settings • “Levels the playing field” • Would be unnecessary if patients were randomly assigned to treatments, settings etc.