1 / 47

Measuring performance of ICUs Does it help to improve?

Measuring performance of ICUs Does it help to improve?. Bertrand Guidet Hôpital St Antoine Paris, France Réanimation Médicale & INSERM U707. Performance indicators. Mortality Activity Efficiency/cost Structure/processes. Mortality. Interpretation of mortality data.

mercia
Download Presentation

Measuring performance of ICUs Does it help to improve?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring performance of ICUsDoes it help to improve? • Bertrand Guidet • Hôpital St Antoine Paris, France • Réanimation Médicale & INSERM U707

  2. Performance indicators • Mortality • Activity • Efficiency/cost • Structure/processes

  3. Mortality

  4. Interpretation of mortality data • Standardized mortality ratio : SMR • = Observed mortality/predicted mortality • In hospital mortality is estimated with severity scores: • SAPS 2 or SAPS 3 • APACHE II or PACHE III • MPM • If SMR < 1 : « good performance » The observed mortality is lower than the expected mortality

  5. Observed/predicted mortality (SMR) according to the origin of the patientsCUB-REA data base : year 2005 (29 ICUs)

  6. Mortality and SMRaccording to diagnosisCUB-REA 2005

  7. Ranking of ICUs with adjusted SMRAegerter P ,… Guidet BSAPS 2 revisited (ICM2005, 31 : 416-423)

  8. Ranking change after adjustment Model B : Age Mode of entry Comorbidities

  9. Standardized mortalityin ICU and in hospital

  10. Standardized hospital mortality for specific diagnosis SMR 3.8 3.3 2.8 2.3 1.8 1.3 0.8 0.3 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 ICUs Diagnosis ARF/COPD ARDS Septic Shock

  11. Activity

  12. Main data of theCUB-Rea annual report (37 ICUs, Paris area) • For the whole database • Per ICU with identification of each ICU • Mean (median, sd, range) • Comparison with previous years

  13. Global characteristics of the patients Description Npatients Age LOS Mean Median Mean Median 56.7y 540 per ICU 56.0 y 7.5 days 4.0 days Severity & mortality SAPS 2 ICU mortality Hospital mortality SMR mean median % of patients % of patients 38.4 34.0 18.0% 23.1% 0.79

  14. Workload and organ support

  15. Rationale for reporting volume of activity

  16. VOLUME and PERFORMANCE • Halm et al, Ann Int Med, 2002. Is volume related to outcome in health care ? A systematic review and methodological critique of the literature • Review of 135 studies with 27 procedures • There is a significant statistical association between volume and outcomein • 71 % studies on hospital volume • 69 % studies on physician activities

  17. Hospital volume and surgical mortality in the United States Birkmeyer et al, N Engl J Med, 2002. • Mortality decreased as volume increased for all 14 typesof procedures • pancreatic resection :12%(16.3%vs. 3.8%) • carotid endarterectomy : 1.6% (1.7% vs 1.5%).

  18. Surgeon Volume and Operative Mortality in the United StatesJD. Birkmeyer, NEJM 2003 , 349 :2117 • For many procedures, the observed associations betweenhospital volume and operative mortality are largely mediatedby surgeon volume. • Patients can often improve their chancesof survival substantially, even at high-volume hospitals, byselecting surgeons who perform the operations frequently

  19. Hospital Volume and the outcomes of mechanical ventilationKahn , NEJM 2006, 355:41-50 20 241 non surgical patients 37 hospitals From 2002 through 2003

  20. Risk-adjusted mortalityKahn , NEJM 2006, 355:41-50

  21. 44,436 patients receiving 44,436 patients receiving mechanical ventilation on mechanical ventilation on admission to 38 ICUs admission to 38 ICUs 894 patients(2%) cared for at 5 894 patients (2%) cared for at 5 ICUs ICUs with missing data with missing data 160 patients (0.3 %) with hemato 160 patients (0.3 %) with hemato - - oncologic disease oncologic disease 1635 patients (3,7%) with drug 1635 patients (3,7%) with drug - - induced coma induced coma 41,747 patients receiving 41,747 patients receiving mechanical ventilation on mechanical ventilation on admission to 33 ICUs admission to 33 ICUs CUB-REA data base 8 years

  22. Characteristics of patients

  23. Mortality

  24. Case-Volume and Mortality in Hematological Patients with Acute Respiratory Failure. Eur Respir J. 2008; 32 (in press) A case volume increase of one admission per year led to a significant mortality reduction with an odd ratio of 0.98 (95% CI : 0.97 – 0.99)

  25. Economic performance

  26. Yearly total Cost per ICU Cost/patient /day Cost/stay Cost/ bed/day Mean 3 665 885 € 985 € 5 990 € 702 € Std deviation 1 109 126 € 257 € 2 740 € 179 € Lowest 1 912 812 € 670 € 3 627 € 481 € Highest 6 425 362 € 1 537 € 12 599 € 1 114 € Crude economic performanceDirect medical costsStudy on 21 French ICUs • There is a need for adjustment to take into account the case-mix

  27. How to detect «a dangerous ICU» ? The case of Dr Shipman • Dr Shipman was a general practioner who murdered some of his patients from 1977 to 1997 : • 180 women and 55 men aged 65 years or over. • He was arrested in 1998 • Could this 20-year delay been reduced with an alarm - alert system ?

  28. ALARM - ALERT • Tekkis et al, BMJ, 2003, Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. • A two level hierarchical logistic regression modelwas used to adjust each unit’s operative mortality 1: case-mix : patient associated factor; 2: hospital associated factor

  29. ALARM - ALERT • Spiegelhalter et al, Int J Qual Health Care, 2003, Risk-adjusted sequential probability ratio tests : applications to Bristol, Shipman, and adult cardiac surgery. • Cumulative excess mortality in Bristol for cardiac surgery HES : hospital episode statistics CSR : cardiac surgery register

  30. Global quality improvement process • Benchmarking is the first step of the quality improvement process. • Adjustment technique : two level hierarchical logistic regression model to take into account patients variables (case-mix) and hospital/unit characteristics. • Once discrepancies between an ICU and the comparator are identified, objectives for improvement should be set.

  31. Reference Identification of the indicator Who is in charge ? Who controls ? • Data collection (who, how, …) • Data analysis and presentation • Goal (level, time) Time scaled graph including an acceptable goal as a reference Goal 2 1 Identify the corrective actions Observations : 1- …………… 2- ……………

  32. Measuring performanceDoes it help to improve?

  33. Impact of public release of performance dataBaker Med Care 2002 • Analysis of mortality trends during a period (1991–1997) when the Cleveland Health Quality Choice program was operational. • Medicare patients hospitalized with 6 medical situations: • acute myocardial infarction (AMI; n = 10,439), • congestive heart failure (CHF; n = 23,505), • gastrointestinal hemorrhage (GIH; n = 11,088), • chronic obstructive pulmonary disease (COPD; n = 8495), • pneumonia (n = 23,719), • stroke (n = 14,293). • Measures. • Risk-adjusted in-hospital mortality, • early postdischarge mortality (between discharge and 30 days after admission), • 30-day mortality.

  34. Results

  35. Discussion « our findings show that there is still much to learn about what public policies and private initiatives will accelerate improvements in care for medical conditions. »

  36. The effect of publicly reporting hospital performance on market share and risk-adjusted mortality at high-mortality hospitalsBaker et al, 2003, Med Care. • Despite CHQC's strengths, identifying hospitals with higher than expected mortality did not adversely affect their market share or, with one exception, lead to improved outcomes. • This failure may have resulted from • consumer disinterest • difficulty interpreting CHQC reports, • unwillingness of businesses to create incentives targeted to hospitals' performance, • hospitals' inability to develop effective quality improvement programs.

  37. Conclusion • Performance indicators should be collected • Several indicators should be looked at • Measuring performance is a managerial tool

  38. Assessing Performance of ICUA directional distance function approach at the patient levelB Dervaux, V Valdmanis, B Guidet • What is the benchmark ? • What is the productivity of each ICU ? • How to deal with variation in case mix ? • How to integrate outliers ? • Data envelopment Analysis method : measure of economic efficiency

  39. Method • Estimation of an efficient frontier that measure technical inefficiency of each patient by the use of relevant directional distance function. • An ICU is technically inefficient in treating a patient if it does not minimize its inputs given its outputs. • The measure of an ICU’s performance is the sum of its’ patient’s inefficiencies. • Chart presenting Econometric performance together with SMR

  40. Estimation of a nonparametric production frontier

  41. Results • Mean inefficiency varies from 19% to 36% • The economic inefficiency is concentrated on few patients : • 80 % of resources are concentrated in 30% of patients • 80 % of inefficiencies are concentrated on less than 20 % of patients. • Diagnosis that account for inefficiency : • ARDS, COPD, AIDS, acute renal insufficiency

  42. Ressources savings versus SMR Economic inefficiency SMR

More Related