Is excessive standardized hospital mortality (HSMR) an indicator of unintended medical injury?

1. Is excessive standardized hospital mortality (HSMR) an indicator of unintended medical injury? Martine de Bruijne May 5, 2008 Invitational Conference May 5, 2008

2. Introduction • Variation in hospital mortality = variation in case-mix + variation in quality of care + residual variation Invitational Conference May 5, 2008

3. HSMR • Expected mortality mortality / observed mortality * 100 • HSMR per hospital is corrected for case-mix, to facilitate comparison between hospitals. • Based on LMR data 2003-2005 Invitational Conference May 5, 2008

4. Dutch adverse events study Invitational Conference May 5, 2008

5. Dutch adverse events study 100 hospitals 21 participating in study 4 academic 6 large teaching 11 general Invitational Conference May 5, 2008

6. Sample of patient files • Random sample from hospital information system • 7926 patient files • 3983 files of hospitals deaths • 3943 files of discharged patients Invitational Conference May 5, 2008

7. Assessment of adverse events in patient files - + Unintended harm? Disability for the patient? Caused by health care rather than underlying morbidity? Preventable? Invitational Conference May 5, 2008

8. Research questions • Is “excess mortality” assessed by HSMR related to unintended medical injury assessed by patient record review? • National level • Hospital level • Patient level Invitational Conference May 5, 2008

9. National level • Excess mortality based on HSMR 2003 to 2005: 7524 (in 3 years) • Record review study 2004: 1735 (95%CI 1482-2032) Invitational Conference May 5, 2008

10. Hospital level Invitational Conference May 5, 2008

11. Hospital level Invitational Conference May 5, 2008

12. Hospital Within sample of 7889 patient files: 50% hospitals deaths instead of 3% in NL Average HSMR 556 (rang 439-629) On average, the HSMR underestimates the risk to die in high risk patients more than it overestimates the risk in low risk patients Invitational Conference May 5, 2008

13. Hospital level Invitational Conference May 5, 2008

14. HSMR in sample 7889 Difference with hospital wide HSMRs due to: • Effect-modification, which is not taken into account and becomes more pronounced in the stratified sample • Underestimation of risk to die in high risk patients • Rather small sample for heterogeneous population Invitational Conference May 5, 2008

15. HSMR & unintended injury • Dif-score=risk to die – observed deaths (1=yes, 0=no) • Correlation of hospital rankings • 0.49 dif-score with observed death • 0.42 dif-score with potentially preventable death • 3% of variation between hospitals in preventable deaths is explained by dif-score. Invitational Conference May 5, 2008

16. Patient level Application of the HSMR model within the sample of 7889 patients Low risk to die in patients who do die -> higher risk for unintended medical injury? Invitational Conference May 5, 2008

17. Patient level Study population • 3961 patient files of hospital deaths • Per hospital: 37% to 94% of all hospital deaths were studied. • 28% no HSMR value On average, hospital deaths have a low risk to die Invitational Conference May 5, 2008

18. Distribution of observed mortality in deciles of expected mortality Invitational Conference May 5, 2008

19. Patient level Invitational Conference May 5, 2008

20. Patient level • HSMR model has C-statistic around 0.85 • Mainly due to good prediction of persons who survive • However, the risk to die in underestimated in patients at high risk to die • For good prediction of infrequent hospital deaths a very high C-statistic is needed (0.95 to 0.99) Invitational Conference May 5, 2008

21. Patient level Invitational Conference May 5, 2008

22. Limitations Record review study • Limited power at hospital level • Poor inter-observer agreement HSMR • Data quality varies between hospitals • Exclusion of patients at low risk to die Invitational Conference May 5, 2008

23. Conclusions • HSMR model seems valid to predict hospital deaths at national level • At the patient level, the difference between observed and expected death is associated with increased risk of preventable adverse events and potentially preventable hospital death. The association is statistically significant, but hardly clinically relevant. Invitational Conference May 5, 2008

24. Conclusions • A new, more detailed model may improve case-mix correction and decrease the number of excluded patients • Very high C-statistic (0.95-0.99) is needed • Future evaluation of HSMR should focus on prediction of deaths. This has implications for design of evaluation study (artificially increase number of high risk patients) Invitational Conference May 5, 2008

25. Conclusions MONITORING? • Hospital mortality at national level: YES • National incidence of potentially preventable deaths: NO • External monitoring of patient safety at hospital level: NO • Internal monitoring of patient safety at hospital level: NOT YET • Real time monitoring YES (this will help to improve patient safety and the HSMR) Invitational Conference May 5, 2008