Ambulatory Safety & Quality Initiative: Enabling Quality Measurement

1. Ambulatory Safety & Quality Initiative: Enabling Quality Measurement Rebecca Roper Session 91 September 10, 2012

2. Agency for Healthcare Research and Quality • Long-term and system-wide improvement of health care quality • Only federal agency with a focus on health services research • With an expanding focus on implementation and system change • Not a policy-making or regulatory agency

3. Ambulatory, Safety, and Quality (ASQ) Initiative Scope of ambulatory care, increasing, as volume and complexity of care are expanding Institute of MedicineReport, Patient Safety: Achieving a New Standard for Care (IOM, 2004). Priority Areas Cornerstone of ASQ is to explore and demonstrate how health information technology (health IT) can improve quality of care provided in ambulatory care setting and for transitions in care settings See: http://healthit.ahrq.gov/ASQ

4. ASQ Grant Initiatives Includes four health IT-focused request for applications (RFAs): Enabling Quality Measurement through Health IT (EQM), HS-07-002 Improving Quality through Clinician Use of Health IT (IQHIT), HS-07-006 Enabling Patient-Centered Care through Health IT (PCC), HS-07-007 Improving Management of Individuals with Complex healthcare Needs through Health IT (MCP), HS-08-002

5. RFA-Specific Summary Products RFA-specific Report Links to final grant reports Cross link to stories and webinar RFA-specific Exemplary stories Written exemplary stories Video exemplary stories RFA-Specific National Webinar

6. Goals of RFA-Specific Report Summarizes the extent to which these projects addressed the research foci ofRFA Identifies practical insight Presents illustrative initial findings to: Inform research discussion Provide guidance to other entities implementing health IT systems for quality measurement and improvement

7. Health IT Ambulatory, Safety& Quality: Enabling Quality Measurement (EQM) Key findings and lessons from the 17 grants of the EQM grant initiative Helps researchers understand the realities and complexities in quality measurement through health IT http://healthit.ahrq.gov/ASQEQMRPT2012.pdf

8. EQM Investigators and Projects

9. EQM Investigators and Projects

10. Enabling Quality Measurement (EQM) Initiative Strategies for the development, deployment and export of quality measures from electronic health record systems. Development of retooled quality measures via health IT Development of de novo quality measures via health IT Issues addressed include: Measure development across episodes of care Clinical data needs for quality measurement export and reporting Reporting of quality data for improvement http://grants.nih.gov/grants/guide/rfa-files/RFA-HS-07-002.html

11. EQM Foci and Associated Projects Total of 17 Projects with a variety of foci * Some projects had multiple focus areas Two EQM Foci were not explicitly addressed:

12. Counts of Type of Health IT for EQM Grantees

13. Counts of EQM Grantees by IOM Priority Area

14. Counts of EQM Grantees by Type of Ambulatory Care Setting *FQHC (Federally Qualified Health Center) CHC (Community Health Clinic)

15. Developing New Measures Projects focusing on developing new measures Berner Kaushal Thomas Turchin Vogt and Williams For a summary of findings from all projects that addressed “Developing New Measures”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative” http://healthit.ahrq.gov/EQMReport2012.pdf

16. Developing New Measures, Selected Findings Berner: new measures were calculated through patient feedback collected via telephone and interactive voice response. The topics of measures included patient-reported problem resolution, medication adherence, and followup activity Kaushal: expert panel reviewed measures related to the quality of ambulatory care. 18 existing measures were prioritized to be generated by EHRs, and 14 new measures were identified in underrepresented measurement areas. Vogt and Williams:developed EHR-based indices for the quality of cardiovascular disease management services in primary care.

17. Accuracy of Measurement Projects focusing on accuracy of measurement • Bailey and Kilbridge • Hazlehurst • Kaushal • Kmetik • Lehmann • McColm • Thomas • Turchin • Weiner • Wu and Mostashari • For a summary of findings from all projects that addressed “Accuracy of Measurement”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative” http://healthit.ahrq.gov/EQMReport2012.pdf

18. Accuracy of Measurement, Selected Findings Kmetik tested the accuracy of patient, medical, and system-related reasons for excluding patients from measure denominators. Frequency of exclusions was relatively low and the health IT systems identified them accurately compared to manual chart review. Bailey and Kilbridgeused NLP to search clinical, demographic, encounter, laboratory, and pharma­cy data to identify ADEs in children with cystic fibrosis, sickle cell disease, and cancer. The system did not perform as well as chart review. The system, however, identified 4 to 20 times more ADEs than the typical voluntary reporting system.

19. Accuracy of Measurement, Selected Findings Hazlehurst tested an NLP approach to the measurement of 18 measures related to the quality of outpatient asthma care. Most health IT-enabled measures gave results comparable to manual chart review Sensitivity rates above 60 percent for 16 of the 18 measures Kaushaltested the reliability of electronic generation of 11 established measures at a local FQHC. Sensitivity of 88 percent and specificity of 89 percent compared to manual chart review. Reliability varied con­siderably across measures, with measures relying on data from both structured fields and unstructured notes tending to be less reliable.

20. Accuracy of Measurement, Selected Findings Lehmann implemented flags in the EHR to identify patients in need of medication monitoring according to measures developed by the National Committee for Quality Assurance. This was significantly more accurate than manual chart review with higher PPV, sensitivity, and specificity. McColmcompared manual coder performance with electronic extraction and coding of data from the EHR. Electronic extraction and coding was highly accurate for case identification for blood pressure, hemoglobin A1c, and low-density lipoprotein data elements.

21. Capturing and Integrating Data Projects focusing on accuracy of measurement • Bailey and Kilbridge • Davidson • Hazlehurst • Lazarus • Lehmann • Logan • McColm • Schneider • Turchin • Vogt and Williams • Weiner • Wu and Mostashari • For a summary of findings from all projects that addressed “Capturing and Integrating Data”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative” http://healthit.ahrq.gov/EQMReport2012.pdf

22. Capturing and Integrating Data, Selected Findings Logan implemented and evaluated a set of 15 measures of the quality of colonoscopy procedures. This confirmed the feasibility of the generating measures using data captured at the point of care through custom data entry screens in an EMR. Turchin researchers developed NLP software to extract information on insulin dosing to identify patients for whom medication therapy was intensified. Davidson worked with nine local CHCs to collaboratively define requirements for a shared quality information sys­tem Team developed business requirements for templates for capturing data re­lated to diabetes and smoking cessation.

23. Capturing and Integrating Data, Selected Findings Vogt and Williamsdeveloped EHR-based quality indices for 11 cardiovascular primary care services. Even though the indices were imple­mented in Kaiser Permanente sites that had substantial experience using the same EHR, investigators had to create an extensive process for extracting, cleaning, and coding the data. Weiner integrated EHR data from two institu­tions that cared for some of the same patients to test a method of risk adjusting physician-level diabetes quality of care rankings. Using a linkage between database tables of demographics and patient identifiers from the two systems, the researchers were able to find patients with visit activity in both locations and conduct a descrip­tive analysis of their patterns of care.

24. Capturing and Integrating Data, Selected Findings Wu and Mostasharicreated health IT tools to assist primary care physicians in small practices in measuring the quality of care. The software “hard-coded” 34 existing measures into the EHR, making them easily accessible to the provider. Schneider attempted to develop a measurement approach that integrated data from primary care practices participating in three community-wide, multi-payer HIE efforts. Several barriers prevented a successful evaluation of the adequacy of these data sources for performance measure­ment

25. Feedback to Clinicians Projects focusing on feedback to clinicians Berner Davidson Lehmann Logan Selby Wu and Mostashari For a summary of findings from all projects that addressed ‘Feedback to Clinicians”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative” http://healthit.ahrq.gov/EQMReport2012.pdf

26. Feedback to Clinicians, Selected Findings Selby offered feedback to staff responsible for population management on the need to intensify medication treatment for those patients with out-of-range values. Modest impact on treatment intensification rates for patients with elevated systolic blood pressure and low-density lipoprotein levels. No observed impact on proportions of patients with levels in the target range Logan posted monthly, physician-specific performance reports Based on feedback from participating physicians, the researchers stream­lined the reports and integrated them into the EHR (analysis of the impact of the reports is in progress).

27. Feedback to Clinicians, Selected Findings Wuand Mostashari provided patient-specific clini­cian reminders and decision support at the point of care and real-time reports on a provider’s overall performance on the quality measures. Provider performance on nearly all measures exhibited statis­tically significant improvements, ranging from 5 to 20 percentage points per measure. Davidson found that use of built-in templates to support clinicians providing and documenting care led to improvements in measures related to smoking cessation at some sites. Lehmann gave primary care providers an EHR-generated paper bulletin listing patients due for therapeutic monitoring tests related to one or more medications. Patients appearing on the bulletins were somewhat more likely to receive monitoring within 2 months.

28. Efficiency of Measurement Projects focusing on efficiency of measurement Bailey and Kilbridge Lazarus Thomas For a summary of findings from all projects that addressed ‘Feedback to Clinicians”, see “Findings and Lessons from the Enabling Quality Measurement Through Health IT Grant Initiative” http://healthit.ahrq.gov/EQMReport2012.pdf

29. Efficiency of Measurement, Selected Findings Baileyand Kilbridge used NLP to search clinical, demographic, encounter, laboratory, and pharma­cy data to identify pediatric ADEs. Time-savings en­abled the researchers to identify a greater number of serious ADEs. Lazarus created a method for prospectively inte­grating multiple types of EHR data with the goal of identifying potential adverse events related to vaccinations. Data showed that 2.6 percent of vaccinations resulted in possible reactions. Thomas applied electronic triggers that might be associated with a diagnostic error This methodology was more efficient than conducting random record reviews and identified errors that were more consequential than many routine errors.

30. Using Electronic Health Records To Measure and Improve Quality for Colonoscopy Procedures Judith Logan, OHSU Effectiveness of colonoscopy screening procedures, typically done in an ambulatory setting, depends on providing high quality examinations that result in accurate diagnoses and few complications. • Story and video highlight how the investigators were able to use data from electronic medical records for quality measurement for colonoscopy procedures. Investigators discuss the lessons learned as they formulated, implemented, presented measures the clinicians. http://healthit.ahrq.gov/EQMStoryLogan2012.pdf http://healthit.ahrq.gov/EQMLoganVideo

31. LOGAN (continued) Logan and her team created and evaluated an electronic quality measurement and feedback program—known as excellence report-- for colonoscopies. • Findings • Physicians receptive to feedback as a way to improve effectiveness and safety of their procedures. • Point-of-care data entry was not seen as overly burdensome. • Physicians wished to have feedback shared broadly. • Continued Use • Excellence report is now delivered in the EHR. http://healthit.ahrq.gov/EQMStoryLogan2012.pdf http://healthit.ahrq.gov/EQMLoganVideo

32. Developing and Testing Quality Measures for Interoperable Electronic Health Records Rainu Kaushal, Weil Cornell Demonstrate effective use of both electronic health records (EHRs) and health information exchange (HIE) to electronically measure quality of care delivered in ambulatory settings. • Story and video highlight the methods used to identify quality measures that could be supported and impacted by EHRs and HIE; results of reliability testing of the quality measures; and subsequent the impact of the investigators’ work on national health IT policy and “Meaningful Use”. http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf http://healthit.ahrq.gov/EQMKaushalVideo

33. Kaushal (continued) • Dr. Rainu Kaushal and team pursued the identification, prioritization, development, and reliability testing of quality measures using an interoperable EHR in a primary care setting. • Process: • With the assistance of an expert panel, they applied a four-part conceptual framework to identify 18 prioritized measures of chronic disease management and preventive services. • Findings: • Electronic reporting correctly identified 88 percent of the patients who received recommended care and 89 percent of the patients who did not receive recommended care compared to manual chart review. • Sustainability: • Fourteen new HIE-enabled measures were developed in five important categories: test ordering, medication management, referrals, followup after discharge, and revisits. http://healthit.ahrq.gov/ASQStoryKaushal2012.pdf http://healthit.ahrq.gov/EQMKaushalVideo

34. Standardization and Automatic Extraction of Quality Measures in an Ambulatory EHR, McColm Exemplary Story - Written The lack of standards for clinical documentation in an EHR is a major barrier to automated quality measurement DenniMcColm’s team established standards for clinical documentation and demonstrated the efficiency and accuracy of using data extraction and reporting to perform quality measurement in the ambulatory care setting. Story highlights the methods used to establish standards and findings from the implementation of an automated system for data extraction of quality measures in the ambulatory setting, including valid, reliable reports that provide actionable insight for the measurement and analysis of care. http://healthit.ahrq.gov/EQMStoryMcColm2009.pdf

35. Use of Natural Language Processing to Improve Quality Measurement, a National Web Conference Purpose To address the existing gap between a health care and a public health practitioner's competencies as it relates to the health IT environment. This specific webinar illustrated how new methods of analyzing free text data stored in electronic health records can impact quality measurement. Learning Objectives Discuss the principles of NLP design and implementation. Describe how NLP is used to operationalize the assessment of quality measurement in asthma care. Explain how NLP is used in monitoring intensification of treatment for patients with diabetes. http://healthit.ahrq.gov/nlp-eqmwebinar

36. Additional EQM Products and Links ASQ Web site: http://healthit.ahrq.gov/ASQ EQM Report: http://healthit.ahrq.gov/EQMReport2012.pdf NLP Webinar: http://healthit.ahrq.gov/nlp-eqmwebinar Kaushal Video: http://healthit.ahrq.gov/EQMKaushalVideo Kaushal Story: http://healthit.ahrq.gov/EQMStoryKaushal2012.pdf Logan Video: http://healthit.ahrq.gov/EQMLoganVideo Logan Story: http://healthit.ahrq.gov/EQMStoryLogan2012.pdf McColmStory: http://healthit.ahrq.gov/EQMStoryMcColm2009.pdf