Differences in adverse events detected using different methods of identification?

1. Differences in adverse events detected using different methods of identification? James M Naessens; Claudia R Campbell; Bjorn Berg; John J Lefante; Arthur R Williams; Richard A Culbertson Division of Health Care Policy & Research, Mayo Clinic, Rochester MN School of Public Health and Tropical Medicine, Tulane University, New Orleans LA

2. Reporting of Patient Safety Measures • Hospital acquired conditions, patient safety indicators and “never” events are gaining more attention in public venues. • Current estimates of medical errors are believed to be substantially undercounted. • Standard methods and definitions have not always been the norm in reports on patient safety.

3. Study Objective • Determine the degree of congruence between several measures of identifying adverse events among hospitalized patients.

4. Definitions • Medical error – mistake or failure of the care process • Adverse event – result of an act of commission or omission with unintended harm to the patient • Walshe, QSHC (2000) • Negativity • Patient impact • Causation

5. Measures of Error & Adverse Events • Morbidity and mortality conferences • Malpractice claims • Error reporting • Administrative data • Medical chart review • Electronic medical record • Direct observation • Clinical surveillance Thomas and Petersen, J Gen Int Med, 2003

6. Measures of Error & Adverse Events • Morbidity and mortality conferences • Malpractice claims • Error reporting –provider-reported adverse events • Administrative data –AHRQ PSIs • Medical chart review –trigger tool • Electronic medical record • Direct observation • Clinical surveillance –bacteremias Thomas and Petersen, J Gen Int Med, 2003

7. Methodology Data Sources – Adverse Events • All hospital inpatients at Mayo Clinic Rochester hospitals discharged in 2005 with research access authorization (N= 60,599). • AHRQ PSI – administrative data • Reported event – concurrent staff pager system • Bacteremia – active surveillance • Trigger tool – retrospective review of 10 records every two weeks (N=235)

8. AHRQ Patient Safety Indicators (PSI) • Based on computer algorithms applied to secondary diagnosis and procedure codes • Mixture of hospital complications and preexisting comorbidities • Incorporated present on admission (POA) indicator • Small difference in definition from presently mandated code since 10/1/2007

9. Reported Adverse Events • Reporting system – centralized “event pager” carried by RN • Events categorized into medication, equipment, falls, skin events and miscellaneous • Harm measure recorded at time of capture on all but skin events • Reported “near miss” and events without harm were excluded

10. IHI Trigger Tool Adverse Events • Nurse review of medical record looking for ~55 “triggers” (e.g., INR > 6) • If trigger discovered, more intense case review to determine if adverse event related to trigger • 2 independent nurses reviewed every case and came to consensus • Cases presented to physician for determination of adverse event and resulting level of harm

11. ResultsOccurrence of Adverse Events

12. Occurrence of Adverse Events **Diagram is not proportionately correct**

13. Reported Events Skin N=399* Medication N=207 Falls N=190 Miscellaneous N=342 * - All considered, no harm scale PSI Puncture / laceration N= 761 Postop DVT/PE N= 196 Postop Hemorrhage N= 124 Postop Resp Arrest N= 91 Most Frequent Types

14. Skin Events, 2005

15. Decubitus Ulcers2008 • Minnesota Reportable Events • 3rd or 4th degree (and unstageable) skin ulcers : 16 cases • Only 4 have secondary diagnosis code reflecting decubitus ulcer • PSI #3 • 153 cases with secondary diagnosis • 57 not present on admission, 1 unk.

16. Infections 14 patients with bacteremia had other PSI 39% of all bacteremias had any PSI

17. Trigger tool cases • 65 cases found with adverse event, most frequent cause UTI, majority with temporary harm. • 12 of the 14 reviewed cases with an adverse event found through another method had a trigger-identified event. One PSI and one reported skin event were not detected through trigger tool.

18. Limitations • Only one location • Inpatient care only • Homogeneous delivery care system with limited number of discharge diagnoses collected (15) • Limited medical records review due to budget constraints

19. Summary • Use of patient safety measures in pay for performance, public reporting, other applications need to be aware of limitations of their data collection method. • Adverse events based on diagnosis codes from administrative data differ from cases identified from provider reported systems, even for similar events (infections, falls, decubitus ulcers)

20. Summary • 4% of cases with an adverse event identified through both PSI and reported events (higher when including trigger tool) • Multiple methods recommended to identify adverse events for internal improvement