The State of the Evidence in Patient Safety

1. The State of the Evidence in Patient Safety Kaveh G. Shojania, MD Canada Research Chair in Patient Safety and Quality Improvement Department of Medicine Sunnybrook Hospital University of Toronto

2. Concise, evidence-based reviews of over 75 specific patient safety practices • Over 140,000 copies obtained since publication in 2001

3. Peri-operative beta-blockers to reduce cardiac complications of non-cardiac surgery • 5 randomized trials at time of AHRQ report (2001) • Total patients ~ 600 across all 5 trials • Substantial benefit: 1 major event averted for every 4-8 patients treated  Received 2nd highest evidence rating in AHRQ report • Meta-analysisof 21 trialssubsequently showed questionable benefit and increases in harm (2005) • Recent mega-trial showed increased total mortality (Devereaux et al. Lancet 2008) Shojania et al. Making Healthcare Safer. 2001.

4. Other Examples of Major Changes or Starkly Conflicting Evidence • Universal MRSA screening • Hip protectors to prevent fall-related injuries • Supplemental oxygen to decrease postoperative infections • Acetylcysteine to prevent contrast-nephropathy • Rapid response teams

5. ‘Survival analysis’ of 100 meta-analyses reviewed in ACP J Club • Major qualitative or quantitative changes in evidence occurred for 23% within 2 years and for 15% within 1 year • 7% were already out of date at time of publication

6. Multifaceted interventions (13 studies) showed a borderline significant reduction in falls but not fractures • No other strategy (hip protectors, removal of physical restraints, fall alarm devices, changes in the physical environment, medication review in hospital) showed consistent, significant effects on falls, fallers, or fractures

7. “Safety of Patients Isolated for Infection Control” (Stelfox et al. Jama 2003) Isolated patients twice as likely to experience adverse events (31 vs 15 adverse events per 1000 days; P<.001). Included significant difference in preventable events (20 vs 3 adverse events per 1000 days; P<.001) • Isolated patients also more likely to • have no vital signs recorded as ordered (51% vs 31%; P<.001) • have days with no physician progress note (26% vs 13%; P<.001)

9. Catheter-related bloodstream infections showed a significant decrease from 0.62 (95% CI: 0.47 to 0.81) at baseline to 0.34 (95% CI, 0.23 to 0.50) at 18 months. • missing data for approx 40% of ICU months • CRBSIs definition open to bias • no corroborating blood culture results

10. Main Arguments Against Needing Evidence in Patient Safety • Too challenging: interventions are too complex to study using conventional EBM paradigm • Some interventions are self-evident

11. Villages…were pair-matched and randomly allocated to receive the intervention at study onset… or 3 years later… Loans were provided to poor women who enrolled in the intervention group. A participatory learning and action curriculum was integrated into loan meetings, which took place every 2 weeks…

12. Cluster RCTs in Patient Safety/QI • Rapid response teams • Hillman et al. Lancet 2005 • Teamwork training • Nielsen et al. Obstet Gynecol. 2007 • Computerized decision support • Eccles et al. BMJ. 2002 • Feedback of acute MI performance data to hospitals • Beck et al. Jama 2005 • PDSA and Chronic Care Model • O’Connor et al. Diabetes Care. 2005 Without these trials, we would waste huge amounts of money on ineffective (versions of these) interventions

13. “Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials.”

14. A simple, obviously beneficial patient safety intervention Concentrated KCl resembles other iv solutions Lethal errors involving injections of concentrated KCl Remove concentrated potassium from clinical areas “forcing function” that prevents errors from happening Sobering example of hospital where delays in receiving KCl from Pharmacy resulted in surreptitious hording of KCl on wards  Increase, rather than decrease in hazard

16. State of the Evidence in Patient Safety • Superficial knowledge about epidemiology of many important safety problems and their causes • Few established, highly effective interventions • most things either don’t work or we don’t know if they work • Those interventions that do work are often costly, complex, and may even create new problems

17. Robust evidence directly informs a minority of clinical situations.  often treat patients on basis of anecdotal experience But we don’t turn these practices into major recommendations In patient safety, robust evidence lacking for almost everything individual hospitals proceed with promising strategies But widespread dissemination will require rigorous evaluation to confirm benefit in wide range of settings Evidence vs. Action Auerbach et al. NEJM. 2007

18. Suggested Framework for Evaluation KCl removal Benefit self-evident? Y N RRTs, Teamwork RCT feasible? Monitor introduction High volume providers, Staffing ratios Y N P-4-P Work hour reductions (cluster) RCT Prospective evaluation Y N Controlled Before-After Study or Interrupted Time Series Multivariate modelling Modified from R.J. Lilford

19. Conclusions • Evidence in patient safety still fairly sparse • but about what one would expect for a field this young • Scant attention to implementation issues, unintended consequences, and costs • Need to distinguish levels of evidence to proceed locally with a given intervention and that required for dissemination Precisely because of complexity and contextual factors, we need large-scale evaluations to determine what intervention components required to achieve benefit in which settings