Data for Outcomes Research

1. Data for Outcomes Research Andy Bindman MD Department of Medicine, Epidemiology and Biostatistics

2. What is Outcomes Research • Studies of the quality of care as judged by patients’ outcomes • IOM domains of quality • Effectiveness • Safety • Timeliness • Equity • Efficiency • Patient-Centered

3. Donabedian Model of Quality Structure Process Outcome

4. Donabedian Model of Quality Structure Process Outcome Number of nurses per hospital bed Physicians per capita

5. Donabedian Model of Quality Structure Process Outcome Beta blocker following MI Immunizations

6. Donabedian Model of Quality Structure Process Outcome Survival Functional status Satisfaction

7. Which is Best to Monitor Quality? • Structure - necessary but not sufficient • Process - many things we do/recommend don’t have proven health benefit • Outcomes - our ultimate responsibility but related to more than just the care we provide

8. Predictors of Outcomes • Outcomes = intrinsic patient risk factors treatment effectiveness quality of care random chance

9. Goals of Risk-Adjustment • Account for intrinsic patient risk factors before making inferences about effectiveness, efficiency, or quality of care • Minimize confounding bias due to nonrandom assignment of patients to different providers or systems of care

10. How is Risk Adjustment Done • On large datasets • Uses measured differences in compared groups • Model impact of measured differences between groups on variables shown, known, or thought to be predictive of outcome so as to isolate effect of predictor variable of interest

11. When Risk-Adjustment May Be Inappropriate • Processes of care which virtually every patient should receive (e.g., immunizations, discharge instructions) • Adverse outcomes which virtually no patient should experience (e.g., incorrect amputation) • Nearly certain outcomes (e.g., death in a patient with prolonged CPR in the field) • Too few adverse outcomes per provider

12. When Risk-Adjustment May Be Unnecessary • If inclusion and exclusion criteria can adequately adjust for differences • If assignment of patients is random or quasi-random

13. When Risk-Adjustment May Be Impossible • If selection bias is an overwhelming problem • If outcomes are missing or unknown for a large proportion of the sample • If risk factor data (predictors) are extremely unreliable, invalid, or incomplete

14. Data Sources for Risk-Adjustment • Administrative data are collected primarily for a different purpose (billing), but are commonly used for risk-adjustment • Disease registries

15. Sources of Administrative Data • Federal Government • Medicare • VA • State Government • Medicaid (Medi-Cal) • Hospital Discharge Data • Private Insurance

16. Advantages of Administrative Data • Computerized, inexpensive to obtain and use • Uniform definitions • Ongoing data monitoring and evaluation • Diagnostic coding (ICD-9-CM) guidelines • Opportunities for linkage (vital stat, cancer)

17. Administrative Hospital Discharge Data • Admission Date • Race • Discharge Date • Sex • Type of Admission • Date of Birth • Source of Admission • Zip Code • Principal Diagnosis • Patient SSN • Other Diagnoses • Total Charges • Principal Procedure and Date • Expected Source of Payment • Other Procedures and Dates • Disposition of Patient • External Cause of Injury • Pre-hospital Care and Resuscitation (DNR)

18. Disadvantages of Administrative Data • No control over data collection process • Missing key information about physiologic and functional status • Quality of diagnostic coding can vary across sites • Non capture of out of plan/out of hospital/out of state events

19. Linking Administrative Data • Strategy for enhancing number of predictor or outcomes variables • Linkage dependent on reliable shared identifiers such as social security numbers in both datasets • Probabilistic matching of less specific variables (age, sex, race, date of birth, etc)

20. 30-day Mortality: Hospital Discharge Data and Vital Statistics • CAP • > 60,000 admissions per year • 30 day mortality - 12.2% • AMI • >130,000 admissions per year • 30 day mortality - 13.0%

21. Disease Registries • Attempt to capture all or large sample of the cases of a specified condition • Often include more clinical information than administrative datasets • Many of these can support assessments of survival beyond acute period • May require permission/approved protocol to access all or some of the data

22. Example Registries • UNOS end stage renal disease • CABG surgery (OSHPD) • SEER Cancer Registry • SEER Cancer Registry linked with Medicare

23. Doing Your Own Risk-Adjustment vs. Using an Existing Product • Is an existing product available or affordable? • Would an existing product meet my needs? - Developed on similar patient population - Applied previously to the same condition or procedure - Data requirements match availability - Conceptual framework is plausible and appropriate - Known validity

24. Conditions Favoring Use of an Existing Product • Need to study multiple diverse conditions or procedures • Limited analytic resources • Need to benchmark performance using an external norm • Need to compare performance with other providers using the same product • Focus on resource utilization, possibly mortality

25. A Quick Survey of Existing ProductsHospital/General Inpatient • APR-DRGs (3M) • Disease Staging (SysteMetrics/MEDSTAT) • Patient Management Categories (PRI) • RAMI/RACI/RARI (HCIA) • Atlas/MedisGroups (MediQual) • Cleveland Health Quality Choice • Public domain (MMPS, CHOP, CSRS, etc.)

26. A Quick Survey of Existing ProductsIntensive Care • APACHE • MPM • SAPS • PRISM

27. A Quick Survey of Existing ProductsOutpatient Care • Resource-Based Relative Value Scale (RBRVS) • Ambulatory Patient Groups (APGs) • Physician Care Groups (PCGs) • Ambulatory Care Groups (ACGs)

28. How Do Commercial Risk-Adjustment Tools Perform • Better than age/sex to predict health care use/death • Better retrospectively (~30-50% of variation) than prospectively (~10-20% of variation) • Lack of agreement among measures • More than 20% of in-patients assigned very different severity scores depending on which tool was used (Iezzoni, Ann Intern Med, 1995)

29. Summary • Risk adjustment is a multivariate modeling technique designed to control for patient characteristics so that judgments can be made about the quality of care • Risk adjustment requires large datasets such as administrative datasets or disease registries • Commercial risk adjustment products exist for patients in different health care settings • There are many reasons why one might choose to develop a risk adjustment model - we will talk about how to do this next week!