Classification for reporting and learning

1. Classification for reporting and learning Joanne Cunningham Trinity College Dublin snichuin@tcd.ie

2. Outline • What & Why • Classifications • International Classification for Patient Safety WHO • RT examples

3. Definition • “Taxonomy is simply a classification or ordering into groups or categories. The key in the definition is ordering or having an organisation behind the categories, rather than simply a listing.” Thomadsen, B, Lin, S-W. Taxonomic Guidance for Remedial Actions http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=aps.section.1796

4. International Classification of Diseases • Well established Classification System • Late 19th Century • Revised approx every 10 years According to the ICD, in 1913 one cause of death was being “Worn Out” Another Cause of Death = “non-existent disease” BUT still a valuable Epidemiological Tool

5. A classification or taxonomy is ... • A tool for analysing and learning from incidents • particularly to identify similarities between incidents not otherwise considered comparable; aggregate data • A means to better understand incident occurrence, prevention, and recovery • reliable and valid • diligently applied WHO 2005

6. Conditions of Work (current) Latent Failures Active Failures Management Decision Organisational Process • Background • conditions: • Workload • Supervision • Communication • Training/ • knowledge/ • ability • Equipment • Unsafe Acts: • Omissions • Action slips • / failures • Cognitive • failures • (mistakes • and memory • lapses) • Violations Reason’s Model of Organisational Accidents Multilayered Defences

7. Principles and criteria a classification system must meet World Health Organisation. Project to Develop the International Patient Safety Event Taxonomy: Updated Review of the Literature 2003-2005. Prepared by Heather Sherman PhD and Jerod Loeb PhD, Joint Commission for the Accrediation of Healthcare Organisations. Geneva: WHO. 2005.

8. Key elements to be considered in the design are: • The purpose of the system • The types of data that are available • The resources that are available to maintain the system • Facilitate analysis for learning • Straightforward (e.g. hazard identification, and summaries and descriptions), or • More analytic (e.g. trends and cluster analysis, correlations, risk analysis, causal analysis, and systems analysis)

9. International Classification for Patient Safety • The ICPS is conceptual based on the • National Reporting and Learning System (United Kingdom) • Advanced Incident Management System (Australia) • Eindhoven/PRISMA-Medical Classification Model (The Netherlands • Patient Safety Event Taxonomy (United States) • Purpose: facilitate the description, comparison, measurement, monitoring, analysis and interpretation of information to improve patient care

10. ICPS conceptual framework • Consisting of 10 high level classes: 1. Incident Type 2. Patient Outcomes 3. Patient Characteristics 4. Incident Characteristics 5. Contributing Factors/Hazards 6. Organizational Outcomes 7. Detection 8. Mitigating Factors 9. Ameliorating Actions 10. Actions Taken to Reduce Risk

11. Classification CERRO

12. Accident, Incident, Adverse event, Near-miss • A patient safety incident is an event or circumstance which could have resulted, or did result, in unnecessary harm to a patient. • An adverse event is an incident which results in harm to a patient. • A near miss is an incident that did not cause harm WHO 2009 • ROSIS: • an incident is defined as the incorrect delivery of radiation • a near-incident / near miss is considered to be any event, which might have resulted in an incident, but for some reason there was no incorrect irradiation.

13. IAEA Definitions • Radiation accident as “an unintended event (operating error, equipment failure or other mishaps) that has or may have consequences.” • Incident as “Any unintended event, including operating errors, equipment failures, initiating events, accident precursors, near misses or other mishaps, or unauthorized act, malicious or non-malicious, the consequences or potential consequences of which are not negligible from the point of view of protection or safety.” • Near miss as: “A potential significant event that could have occurred as the consequence of a sequence of actual occurrences but did not occur owing to the plant conditions prevailing at the time.” IAEA safety glossary: Vienna. 2007.

14. Tripartite Agreement defn of Incident

15. Radiation Oncology Practice Standards (Tripartite Agreement)

16. Radiation Oncology Practice Standards (Tripartite Agreement)

17. ROSIS Classification (1) Three main requirements: • Effective tool for analysis and learning – RO specific, I/N-I, detailed • Flexible • Applied to different departments and processes • Translated into different languages • Incorporated into the reporting system – classified prospectively

18. Key Features of the ROSIS Classification • Radiation Oncology Specific • Method • Literature review • RT incident-types from ROSIS database • Purpose • Organise reports • Facilitate analysis • Improve safety • Scope • All incidents and near-incidents relevant to an RO dept • Preventative & corrective factors • Intent • Maximise learning - Collect detailed information • Feasibility • Incorporated into online Reporting System • To be evaluated: • Analysis • Sensitivity • Reliability and Validity

19. ROSIS Classification (2) OVERVIEW OF CLASSIFICATION

20. ROSIS Classification (3) Severity Potential Outcome yes Causes Event Detection Actual Outcome Process

21. ROSIS Process Classification“Level 1” 52 134 39 141 6 Where in process did it originate? What element was affected? 4 “levels” 46 82

22. LEVELS 2 AND 3 0-4 5-9 10-19 20-29 30+ “Level 2 & 3” 56 37 8 141 5 35

23. Analysis of Process Classification • Retrospective Analysis of Process Classification • 3 persons • Each classified 1st 200 ROSIS reports • MS Access Database • Excluded ( n=21): • Non-process reports (n=16) • Non-RT specific reports (n=2) • Not completed at Level 1 (n=3) • 179 reports for comparison • Frequency of use of categories • Agreement between persons

24. Frequency of Categories – Level 1 All Categories Excl. Tx Preparation Activity Activity Pearson Chi-Square 8.134 p=.616 Pearson Chi-Square 21.494 p<0.05

25. ROSIS Classification Severity Potential Outcome yes Causes Event Detection Actual Outcome Process LESSONS TO LEARN

26. Dept A Dept C Dept E Dept J Comparison between 4 Departments with >50 reports Simulation Planning Prescription Dose Calculation Tx Preparation Tx Delivery

27. Summary of Classification • Useful tool in collating, analysing and learning from incidents • Role for disciplinary-specific classifications and reporting systems • Compatibility between systems • Not a perfect science

28. “Table of casualties” England in the 17th Century Some categories: Burnt & Scalded Wolf Cut of the Stone Execution Fainted in a bath Falling sickness Kings Evil Lunatick Suddenly Found dead in the streets Cancer, Gangrene & Fistula Killed by several accidents Stopping of the Stomach

29. References... • World Health Organisation Patient Safety: Reduction of Adverse Events Through Common Understanding and Common Reporting Tools. Towards an International Patient Safety Taxonomy: A Review of the Literature on Existing Classification Schemes for Adverse Events and Near Misses, A Draft Framework to Analyze Patient Safety Classifications, and a Draft Comparative Glossary of Patient Safety Terms. Prepared by Jerod M Loeb PhD and Andrew Chang JD MPH Joint Commission on Accrediation of Healthcare Organisations. Geneva: WHO. 2003. • World Alliance for Patient Safety. WHO Draft Guidelines for Adverse Event Reporting and Learning Systems: From Information to Action. Geneva: WHO. 2005. • World Health Organisation. Project to Develop the International Patient Safety Event Taxonomy: Updated Review of the Literature 2003-2005. Prepared by Heather Sherman PhD and Jerod Loeb PhD, Joint Commission for the Accrediation of Healthcare Organisations. Geneva: WHO. 2005. • Thomadsen, B, Lin, S-W. Taxonomic Guidance for Remedial Actions In: Henriksen K, Battles J, Marks E, Lewin D, editors. Advances in patient safety: from research to implementation. Vol. 2, Concepts and methodology. AHRQ Publication No. 05-0021-2., Rockville, MD: Agency for Healthcare Research and Quality. 2005. Available from http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=aps.section.1796. Accessed 12th July 2006;75-86. • An Organisation with a Memory: Report of an expert group on learning from adverse events in the NHS. An Organisation with a Memory: Report of an Expert Group on Learning from Adverse Events in the NHS 2000. • World Health Organisation (WHO). The Conceptual Framework for the International Classification for Patient Safety (ICPS). Version 1.1. Geneva: WHO. 2009. • International Atomic Energy Agency. IAEA safety glossary: Terminology used in nuclear safety and radiation protection. Vienna. 2007. • Runciman, W, Hibbert, P, Thomson, R, Van DerSchaaf, T, Sherman, H, Lewalle, P. Towards an International Classification for Patient Safety: key concepts and terms. Int J Qual Health Care 2009;21:18-26. • W. van Vuuren / Safety Science 33 (1999) 13±29

30. A framework for analysing classification methods WHO 2003 • “Is the purpose of the classification fully explained and is it appropriate for its intended use? Preferably, the classification should have been tested on the types of incidents and adverse events to which it will be applied. • Is the classification broad enough for the application, neither capturing too many nor too few data elements? Is it capable of identifying preventative and corrective strategies where this is relevant? • What is the conceptual approach to the classification framework? In other words, which theory in the science of human factors and error and systems failure does it reflect, if any, and is this approach consonant with the orientation of the purpose? Is the theory well established (e.g. Reason’s human error) or is it an idiosyncratic notion that may not correspond to a broader body of knowledge?

31. A framework for analysing classification methods WHO 2003 • How feasible is the classification to implement? Can it be implemented as a paper-based and electronic on-line incident monitoring system or mapped to data collected from existing reporting systems? Is professional expertise required to apply or interpret the classification instrument? Does it use readily available data (e.g. information already contained in medical records, medicolegal files, complaints, morbidity and mortality data) and will it be readily acceptable to patient safety stakeholders? What useful purposes have been achieved using the classification? Is the classification instrument readily available and is there a cost involved? Above all, are there clear instructions that specify how the data elements are codified?

32. A framework for analysing classification methods WHO 2003 • Is it clear how data derived from the classification are analyzed? • Is it sufficiently sensitive to differentiate similar adverse events with different contributing factors, and is this adequate for the purpose? Is it suitable for recording and tracking errors only, or can it provide detailed information to inform the development of preventative and corrective strategies? • How strong is the available evidence for reliability and validity of the classification instrument? Has it been field tested in the “real world?” How many different incident reporting systems has it been compared with? How many different users have tested the classification instrument, and did they obtain similar results?”