Daniel M. Harris, PhD, The CNA Corporation, Wilson Pace, MD & Doug Fernald, MA, Univ of CO DFM,

1. A Comparison of Medical Error Reports Submitted to a Voluntary Patient Safety Reporting System by Different Types of Reporters: A report from the ASIPS Collaborative Daniel M. Harris, PhD,The CNA Corporation, Wilson Pace, MD & Doug Fernald, MA, Univ of CO DFM, Linda Marr, MS, The CNA Corporation AcademyHealth Annual Research Meeting June 6, 2004 N Working Together to Improve Health ASIPS Applied Strategies for Improving Patient Safety

2. Presentation outline • Background • Research objective • Study design • Findings • Conclusions & implications

3. Background • Applied Strategies for Improving Patient Safety (ASIPS) is a 3-yr AHRQ-funded demonstration project based at the Univ of CO Dept of Fam Med • Developed patient safety reporting system (PSRS) • Collect voluntary narrative reports of medical errors from providers, other clinical staff, and non-clinical staff at primary care practices in two CO PBRN • Anonymous or confidential-to-anonymous reports • Instructed to report “any event you don’t wish to have happen again that might represent a threat to patient safety” • Overall purpose of project is to test ability of the PSRS to collect incident reports and to use them to design interventions to improve patient safety

4. Research objective • Medical error reporting systems are advocated as a strategy for improving patient safety • Learning from errors requires receiving reports on repre-sentative range of errors • Limited evidence exists re: nature of events report-ed to such systems by different reporter types, especially in ambulatory primary care settings • Do different types tend to report different types of events? • Our objective is to (1) analyze reports submitted to ASIPS PSRS to ascertain “who reports what,” (2) identify similarities and differences in reports, and (3) characterize differentiating event attributes by reporter type

5. Study design • Error incident reports are received from 34 primary care practices throughout Colorado • Reports are consensually coded into multi-axial taxonomy of 400+ attributes by teams of coders • Attributes characterize events by type of participants & their contribution, complexity, setting, clinical intent, activities, under-lying causes, patient outcomes, interventions, and “discoverer” • Attributes are coded “1” if present in report; “0” if not present • Reporters self-classified as provider, other clinical staff, or non-clinical staff • 522 (85.8%) of 608 reports received thru Aug 2003 identified a reporter type • Report content compared by reporter type using cross tabs, ANOVA, and discriminant analysis

6. Findings • Analysis of report • Reports by reporter type • Nature of report • Event “discoverer” • Analysis of event reported • Participants • Patient harm • Communication errors • Other event characteristics • Discriminant analysis

7. Reports by reporter type • Most reports (68.6%) from providers • Unchanged from yr1 to yr2 • About 25% from other clinical staff • Increased from yr1 to yr2 • Fewest reports (6.9%) from non-clinical staff • Deceased from yr1 to yr2 despite project effort to increase participation

8. No difference by reporter type: Patient gender Perceived prevent-ability Perceived pt knows Mean taxonomy, event activity, and “don’t know” codes Differences by type: Patient age Mean lowest for non-clin; highest for provider At least 1 “insufficient information” code Non-clin most likely Noted in Med Rec’d Non-clin least likely Perceived pt harm Oth clin least likely Nature of report

9. Event “discoverer” • Providers & other clinical staff likely to report events discovered by caregivers like themselves • Non-clinical staff likely to report events discovered by office staff like themselves • Providers somewhat less likely to report events discovered by patients or patient’s families

10. Event participants • Each reporter type is more likely to report an event with a participant of the same type • All relationships statistically significant by 2 • No significant association by reporter type for other types of participants: • Patient/family or 3rd party • Number of participants or number of participant types (by ANOVA)

11. Patient harm • Overall, 26.4% of report-ed events coded w/some form of patient harm • Additional 7.7% of reports, coded w/patient may have been harmed, but too early to tell • Significant differences in type of harm (by 2): • Provider reports most likely coded w/clinical harm • Non-clinical staff reports most likely coded with non-clinical harm • Other clinical staff reports least likely coded with any harm

12. Communication • Over half (57%) of reports involve a communication error between parties • Each reporter type likely to report communication involving their own type • Oth clin staff less likely to report within office communication errors • Overall, non-clin staff less likely to report communication errors • Providers more likely to report communication involving patients

13. No difference by reporter type: Missing information General proc issue Medication error Supervision error Intervention following error detection Judgement error Clinical knowledge or skill error Resource function or availability error Differences by type: Documentation Oth clin most likely Specific proc issues Delay in Dx and Tx – prov most likely Delay in testing – oth clin staff most likely Dx testing error – non-clin least likely Pt mgmt error Non-clin most likely Distraction/inattention Prov least likely System issue Prov least likely Other event characteristics

14. Discriminant Analysis (1) • Identify event attributes that differentiate be-tween reports submitted by each reporter type • Stepwise method: attributes enter that maximize distance between 2 closest groups (16 of 32 entered) • Resulting 2 discriminant functions differentiate between the 3 reporter types • Group centroids are significantly different • Provider reports are most different from other 2 types • Canonical correlations (~ANOVA Eta) for the functions are .455 & .346; jointly account for 30% of variance in group scores • Discriminant scores correctly classify 63% of reports • Classify unknown reports in similar proportion

15. Discriminant Analysis (2) • The analysis identified attributes that best discriminate between events reported by each reporter group • Provider reports most differentiated by: • provider participant, communication involving a provider, disclosure to a patient, delay in diagnosis, diagnostic testing error, and problem with resource (availability or function) • Other clinical staff reports most differentiated by: • Non-physician provider participant, third party participant, communication involving a provider, communication within the office, delay in performing a test, diagnostic testing error, and misuse of a system • Non-clinical staff reports most differentiated by: • Non-clinical staff participant, non-clinical harm, error in patient management, distraction/inattention error, misuse of a sys-tem, and malfunction of a system

16. Conclusions • Different types of reporters tend to submit reports of different kinds of medical errors to a voluntary PSRS • Each type of reporter tends to report errors involving the kinds of activities and participants they can be most expected to observe and believe should be reported • For example: • Providers tend to report clinical harm events while non-clinical staff tend to report non-clinical harm events • Providers tend to report events involving clinical proce-dures while non-clinical staff tend to report patient management and system issue events

17. Implications • To be effective in identifying a full and repre-sentative range of errors and threats to patient safety, a voluntary PSRS needs to assure that it receives incident reports from all types of staff (and patients?) who are in a position to observe errors • Provider exclusive or dominated reporting systems will restrict this range • Our experience demonstrates the difficulty of obtaining full participation of non-providers • More effort necessary to achieve their participation

18. Working Together to Improve Health ASIPS Applied Strategies for Improving Patient Safety The CNA Corporation Questions?