1 / 38

PART 07

PART 07 Evaluating Hospital Performance PERFORMANCE MEASURES Patient outcomes Mortality, morbidity, satisfaction with care 30-day mortality among heart attack patients ( Normand et al JAMA 1996, JASA 1997) Process Medication & test administration, costs

niveditha
Download Presentation

PART 07

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PART 07 Evaluating Hospital Performance BIO656--Multilevel Models

  2. PERFORMANCE MEASURES Patient outcomes • Mortality, morbidity, satisfaction with care • 30-day mortality among heart attack patients (Normand et al JAMA 1996, JASA 1997) Process • Medication & test administration, costs • Laboratory costs for diabetic patients • Number of physician visits • Hofer et al JAMA, 1999 • Palmer et al. (1996) BIO656--Multilevel Models

  3. DATA STRUCTURE Multi-level • Patients nested in physicians, hospitals, HMOs, ... • Providers clustered by health care systems, market areas, geographic areas • Covariates at different levels of aggregation: • patient, physician, hospital, ... Variation in variability • Statistical stability varies over physicians, hospitals, .. BIO656--Multilevel Models

  4. MLMs are Effective Correlation at many levels • Hospital practices may induce a strong correlation among patient outcomes within hospitals even after accounting for patient characteristics Structuring estimation • Stabilizing noisy estimates • Balancing SEs • Estimating ranks and other non-standard summaries BIO656--Multilevel Models

  5. The Cooperative Cardiovascular Project (CCP) • Abstracted medical records for patients discharged from hospitals located in Alabama, Connecticut, Iowa, and Wisconsin (June 1992May 1993) • 3,269 patients hospitalized in 122 hospitals in four US States for Acute Myocardial Infarction BIO656--Multilevel Models

  6. GOALS • Identify “aberrant” hospitals with respect to several performance measures • Report the statistical uncertainty associated with ranking of the “worst hospitals” • Investigate if hospital characteristics explain variation in hospital-specific mortality rates BIO656--Multilevel Models

  7. DATA Outcome • Mortality within 30-days of hospital admission Patient characteristics • Admission severity index constructed on the basis of 34 patient attributes Hospital characteristics • Urban/Rural • (Non academic)/(versus academic) • Number of beds BIO656--Multilevel Models

  8. Why adjust for case mix?(patient characteristics) • Irrespective of quality of care, older/sicker patients with multiple diseases have increased need of health care services and poorer health outcomes • Without adjustment, physicians/hospitals who treat relatively more of these patients will appear to provide more expensive and lower quality care than those who see relatively younger/healthier patients • If there is inadequate case mix adjustment, evaluations will be unfair • But, need to avoid over adjusting BIO656--Multilevel Models

  9. Case-mix Adjustment Compute hospital-specific, expected mortality by: • estimating a patient-level mortality model using all hospitals 2. averaging the model-produced probabilities for all patients within a hospital • Hospitals with “higher-than-expected” mortality rates can be flagged as institutions with potential quality problems, but need to account for uncertainty • Need to be careful, if also adjusting for hospital characteristics • May adjust away the important signal BIO656--Multilevel Models

  10. Wrong SEs • Test-based (as we know, very poor approach) BIO656--Multilevel Models

  11. Hospital Profiling of Mortality Rates Acute Myocardial Infarction Patients(Normand et al. JAMA 1996, JASA 1997) BIO656--Multilevel Models

  12. Hierarchical logistic regression I: Patient within-provider • Patient-level logistic regression model with random intercept & slope II: Between-provider • Hospital-specific random effects are regressed on hospital-specific characteristics • Explicit regression BIO656--Multilevel Models

  13. Admission severity index(Normand et al. 1997 JASA) BIO656--Multilevel Models

  14. sevbar 0 + 1(sevij – sevbar) 0 1 BIO656--Multilevel Models

  15. we use b0i + b1i(...) BIO656--Multilevel Models

  16. b0i = *00 + N(..), etc. Interpretation of parameters is different for the two levels BIO656--Multilevel Models

  17. RESULTS • Estimates of regression coefficients under three models: • Random intercept only • Random intercept and random slope • Random intercept, random slope, and hospital covariates • Hospital performance measures BIO656--Multilevel Models

  18. Normand et al. JASA 1997 BIO656--Multilevel Models

  19. 30-DAY MORTALITY2.5th and 97.5th percentiles for a patient of average admission severity Exchangeable model • Random intercept and slope, no hospital covariates log(odds): (-1.87,-1.56) probability,scale: (0.13, 0.17) Covariate (non-exchangeable) model • Random intercept and slope, with hospital covariates • Patient treated in a large, urban academic hospital log(odds): (-2.15,-1.45) probability scale: (0.10,0.19) BIO656--Multilevel Models

  20. Effect of hospital characteristics on baseline log-odds of 30-day mortality • For an average patient, rural hospitals have a higher odds ratio than urban hospitals • Indicates between-hospital differences in the baseline mortality rates • Case-mix adjustment may be able to remove some of this difference BIO656--Multilevel Models

  21. Estimates of Stage-II regression coefficientsIntercepts BIO656--Multilevel Models

  22. Effect of hospital characteristics on association between severity and mortality(slopes) • The association between severity and mortality is modified by hospital size • Medium-sized hospitals have smaller severity/mortality associations than large hospitals • Indicates that the effect of clinical burden (patient severity) on mortality differs across hospitals BIO656--Multilevel Models

  23. Estimates of Stage IIregression coefficientsSlopes BIO656--Multilevel Models

  24. Homework is on front table BIO656--Multilevel Models

  25. Observed and risk-adjusted hospital mortality rates Urban Hospitals Histogram displays (observed – adjusted) BIO656--Multilevel Models

  26. Observed and risk-adjusted hospital mortality rates Rural Hospitals Histogram displays (observed – adjusted) Substantial adjustment for severity BIO656--Multilevel Models

  27. FINDINGS • There is substantial adjustment for admission severity • Generally, urban hospitals are adjusted less than rural • There is less variability in observed or adjusted estimated rates for urban hospitals than for rural hospitals Can you explain why? BIO656--Multilevel Models

  28. Normand et al. JASA 1997 BIO656--Multilevel Models

  29. Average the probabilities Don’t average the covariates BIO656--Multilevel Models

  30. k denotes a draw from the posterior BIO656--Multilevel Models

  31. Plug in the average covariate Keep the hospital variation BIO656--Multilevel Models

  32. BIO656--Multilevel Models

  33. Comparing measures of hospital performance Three measures of hospital performance • Probability of a large difference between adjusted and standardized mortality rates • Probability of excess mortality for the average patient • Z-score BIO656--Multilevel Models

  34. Hospital Rankings: Normand et al 1997 JASA BIO656--Multilevel Models

  35. Hospital Ranks • There was moderate disagreement among the criteria for classifying hospitals as “aberrant” • Nevertheless, hospital 1 is ranked worst • It is rural, medium sized non-academic with an observed mortality rate of 35%, and adjusted rate of 28% BIO656--Multilevel Models

  36. Adjusting for hospital-level charateristics Changes the comparison group in “as compared to what?” • All hospitals (unadjusted at hospital level) • Hospitals of a similar size, urbanicity, ... • Percent of physicians who are board certified • Hospitals with a similar death rate  Variance reduction and goodness of fit should not be the primary considerations • “As compared to what?” must dominate BIO656--Multilevel Models

  37. Discussion • Profiling medical providers is multi-faced and data intensive process with substantial implications for health care practice, management, and policy • Major issues include data quality and availability, choice of performance measures, formulation of statistical models (including adjustments), reporting results • The ranking approaches and summaries used by Normand and colleagues are very good, but some improvement is possible BIO656--Multilevel Models

  38. Multi-level models address key technical & conceptual profiling issues, including • Adjusting for patient severity • Accounting for within-provider correlations • Accounting for differential sample sizes at all levels • Stabilize estimates • Structure ranking and other, derived comparisons BIO656--Multilevel Models

More Related