Lectures

1. Health and Cost Data InputsAdvanced Training in Clinical Research DCEA Lecture 4February 12, 2009UCSF Department of Epidemiology and BiostatisticsJames G. Kahn, MD, MPHjgkahn@ucsf.eduBrian Harris, BA, MPP Candidatebrianharris1113@hotmail.com

2. Lectures 1 - Introduction to Decision Analysis 2 - Consideration of Utility and QALYs 3 - CEA Overview/Developing an Analysis/Costs

3. Now: Data Inputs • What type of data do we use to determine • The health inputs that are changing? • The costs that are changing? • Where do we find these data?

4. Today’s Objectives:Data Type, Quality & Source • To Understand the General Issues in Gathering and Presenting Health and Cost Data Inputs • To Understand Data Sources and Synthesis Methods for Health and Cost Inputs • To Understand Common Criticisms Surrounding Health and Cost Data Inputs

5. Lecture Structure A. Data Type, Quality, & Source • General Approach • Hierarchies of Types of Studies • Health Inputs • Cost Inputs • Presentation of Inputs • Review/References • Data Searches 7. Resources available at UCSF 8. Using PubMed 9. Other Sources of Data … the big G

6. 1: General Approach A. Research Question & Conceptual Model B. Model Inputs: 1. Measures of Health States & Preferences 2. Measures of Resource Use C. Tradeoffs - Refine Model Given Data Availability D. Best Estimates & Plausible Ranges - Base Case & Range

7. 1-C: Tradeoffs--Refine the Model, Given Data Availability 1. CEA models usually have many inputs 2. Resources are limited a. Your time b. Your budget c. Time for decision to be made 3. Therefore, model can be modified to utilize reliable data currently available. Iterative process highlights critical vs. marginal components.

8. 1-D: Best Estimates and Plausible Ranges 1. Best Estimate = Base Case = Baseline a. The most likely value for the input: the value in the center of the best available data. b. You can intentionally err in one direction to prove the strength of your result. • 2. Plausible Range: similar to 95%, not 99.99%, C.I. • a. When using a single empirical data base, calculate a formal confidence interval (typically + or – 95%). • b. With multiple sources, informal use of best range.

9. 1-Example: Model – To Clip Or Not To Clip

10. 2: Hierarchies of Types of Studies • Hierarchy used by this Decision Cost-Effectiveness Analysis Course • Hierarchy recommended by the U. S. Preventive Services Task Force • Hierarchy used by Drummond, et al. in “Methods for the Economic Evaluation of Health Care Programmes” • Melded Hierarchy

11. 2-A: DCEA Hierarchy • Definitive trials (large randomized controlled trials) • Meta-analysis of trials • Systematic Review • Smaller Trials • Large Cohort Studies • Small Cohort Studies • Expert Opinion

12. 2-A-1: Definitive Trials • Large RCTs (Randomized Controlled Trials) • Participants are assigned randomly to treatment group or control group • Confounding variables therefore randomly distributed across groups • Problems: i. Does study population represent your modeled group? ii. Clinical trial conditions > real world conditions

13. 2-A-2: Meta Analysis • Combines several studies, diluting errors of a single study and increasing statistical power. • Problems: i. “File Drawer Syndrome” ii. Unresolvable heterogeneity iii. LeLorier suggests large RCT > Meta iv. Meta-analysis often requires $100,000 • The Cochrane Library has established useful protocols for Meta Analysis methodology.

14. 2-A-3: Systematic Review • Excellent method: Far easier than Meta-Analysis, often with similar results. • Careful, critical, structured compendium of studies, summarizing methods & findings. • Describe: subjects, study design, outcome measurement, and outcomes. • Baseline & range selected by informal weighting of study size and quality.

15. 2-A-3: Systematic Review (CONT.) • Most common method of obtaining inputs, often used for several inputs in a CEA. • When no definitive study is available (usually the case), this is far better than relying on a single non-definitive report. g. At least 95% less work than a formal Meta-Analysis.

16. 2-A-4: Smaller Trials • Unbiased if well-done, but imprecise due to small size. • Point estimate may be wrong, C.I. large. c. Often used, perhaps because of promising clinical results . . . . Reader beware: recall “File Drawer Syndrome”.

17. 2-A-5: Large Cohort Studies • Cohort = persons already exposed to risk factor, treatment, etc., vs. unexposed or untreated control group. • No random assignment, therefore selection bias and confounding variables. • In untreated populations (when no treatment was available), can be used to estimate outcomes.

18. 2-A-6: Small Cohort Studies • Same problems as large cohort studies. b. Additionally, imprecise due to small size.

19. 2-A-7: Expert Opinion • Virtues: i. Absent data, it’s all you’ve got. You need to use something; expert opinion is common for 1-2 inputs in a CEA. ii. Experts do know a lot. iii. Experts are influential & can help you. b. Problems: they’re just opinions; small sample creates bias or random error.

20. 2-B: U. S. Preventive Services Task Force Hierarchy • Randomized Controlled Trials. • Cohort Studies. 3. Case-Control Studies.

21. 2-B-3: Case-Control Studies • Study and control groups selected on the basis of having or not having the disease. • Therefore, retrospective study. • Problems: i. Confounding variables hard to identify. ii. Observer bias: health outcome known. iii. Recall bias: imperfect recall. iv. Selection problems similar to cohort studies.

22. 2-C: Drummond, et al. Hierarchy • Randomized Controlled Trials. • Cohort Studies. • Case-Control Studies. • Case Series. a. No control group. b. E.g.—Heart Transplant in UK: Untreated group considered unethical. c. Common with new procedures.

23. 2-C: Drummond, et al. (cont.) 5. RCTs may lack precision important to economic analysis but not important to clinical outcomes. 6. Precise clinical setting creates unbiased and precise data: high signal to noise (high internal validity). 7. Clinical precision unlikely to be replicated in real world, so results might not be generalizable (low external validity).

24. 2-D: Melded Hierarchy • Definitive trials (large RCTs) • Meta-analysis of trials • Systematic Review • Smaller Trials • Large Cohort Studies • Small Cohort Studies • Case-Control Studies • Case Series • Expert Opinion

25. 3: Health Inputs • Overview • Steps • How to Find Inputs

26. 3-A: Health Inputs Overview 1. Health State Outcomes a. Relevant Outcome States b. Probability Estimates 2. Health Preferences Weights a. Preference Weights for Outcomes b. Utilities, QALYs 3. Population Characteristics a. Relevant Population b. Disease Prevalence in Population

27. 3-B: Health Inputs - Steps • List Potential Health Outcome States and Relevant Population • Find Data on States & Probabilities - Start With Comprehensive Literature Search • Find Data on Utilities • Find Data on Population Characteristics

28. 3-B-1 & 2: Health States Key Questions: • What Are the Relevant Health States Over Time for the Disease Under Study? • When Do These States Occur, and How Long Do They Last? • What Are the Likely Side Effects or Other Unintended Consequences for Each Group? What Are Key Outcomes of Interest for Relevant Stakeholders, e.g. Patients, Clinicians, Payers, Employers, Policy Makers, Society as a Whole? • For Which Health States, Are There Credible Estimates? • Are These Estimates Appropriate for Your RQ (Research Question) and for Your Population?

29. 3-B-1 & 2: Example – Aneurysm Analysis For the aneurysm analysis, health outcomes were estimated from multiple sources: • Aneurysm rupture rates large cohort study • SAH case fatality meta-analysis • SAH disability medium cohort study, • meta-analysis • RR mortality with disability small cohort study • Surgical mortality, disability meta-analysis • RR rupture (= 0) expert opinion (informal)

30. 3-B-3: Preference Weights - Utilities a. Disease-specific Utilities b. Generic Utilities c. Key Questions: i. Do Credible Estimates Exist for Your RQ? ii. Are the Data Appropriate for Your RQ and Your Model? iii. Whose Perspective Are You Taking? iv. Disease Specific Ratings v. Community Ratings vs. Patient Ratings

31. 3-B-4: Population Characteristics a. Prevalence of the Disease b. Key Questions: i. What is the Relevant Disease Prevalence? ii. National, Representative Samples iii. Disease Surveillance Databases iv. Integrated Delivery System Databases v. Claims Databases c. What Competing Risks Exist (Unrelated to RQ)? d. Do Credible Estimates Exist? e. Are These Estimates Appropriate for Your RQ?

32. 3-B-4-Example: Population Characteristics - Aneurysm Analysis • Prevalence of disease - not needed, not a study about screening or an estimate of total societal costs • All-cause mortality - very important because of the low risk of aneurysm rupture and hence the high risk of dying before rupture occurs • Estimated by age and sex from a data base maintained at CDC, available on the internet.

33. 3-C: Health—How to Find Inputs • Comprehensive Literature Review • Primary Data Collection • Commonly Used Health Data Sources 4. Current Recommendations

34. 3-C-1: Comprehensive Literature Review • Employ hierarchies presented in Part 2. • Always consider relevance to your model. • Revise your model when the cost of the data your model requires is greater than the benefit from not revising the model.

35. 3-C-3: Commonly Used Health Data Sources a. Clinical Trials b. CMS/VA Databases c. Disease Registries d. Quality of Well-Being Index (QWB), Health Utilities Index (HUI) - www.healthutilities.com/overview.htm e. Disability/Distress Index, EuroQol Instrument

36. 3-C-4: Current Recommendations*:US Panel on Cost-effectiveness in Health and Medicine a. Provide a Reference Case Analysis b. Select Data Inputs From Highest Quality Sources That Are Relevant to the RQ and the Population c. Expert Opinion Is Relevant Only When No Other Adequate Data Exists d. Use QALYs to Incorporate Morbidity and Mortality Into a Single Measure e. Use Community Preferences for Health States f. Perform a Sensitivity Analysis on Inputs * See Gold MR et al. for Complete Set of Recommendations

37. 4: Cost Inputs • Direct Costs • Fixed vs. Variable Costs • Time Costs • Current Recommendations

38. 4-A: Direct Costs 1. Published Estimates 2. Resources Used x Cost per unit Used 3. Cost Data Bases

39. 4-A-1: Published Estimates a. If available, these are the easiest b. Should be well-done c. Health care scenarios should match d. Often, costs must be updated

40. 4-A-1-b: Should be well done i. Sources of data used should be documented. • The data used should be appropriate (e.g.—not unadjusted, unnegotiated billed charges) iii. Algorithms used to process that data should be transparent and credible

41. 4-A-1-c: Health-Care Scenarios Should Match i. Diagnosis & severity/complications • Age mix • Care practices • Care setting

42. 4-A-1-d: Costs must be Updated • Adjust using the medical component of the Consumer Price Index (CPI), maintained by the Bureau of Labor Statistics (BLS) • Substitute particular unit costs with updated values for the same services • Whichever method is used, all costs must be denominated in a single currency adjusted to a single year

43. 4-A-2: Resources Used x Cost per Unit Used • Resources b. Costs

44. 4-A-2-a: Resources i. Theoretical model • Empirical evidence—typically better, but more difficult • Micro vs. Macro data

45. 4-A-2-b: Unit Costs • Reimbursements • Billed Charges • Cost Accounting Systems • Price References

46. 4-A-2-b-i: Reimbursements a. Acceptable, especially if based on negotiated rates • Medicare reimbursement for inpatient care is based on prices established for DRGs (Diagnostic Related Groups) • Excellent approach: RBRVS (resource-based relative value scale) used by Medicare for outpatient services • When deductibles and copayments are charged, these should be included when calculating cost for the societal perspective • In some databases, “allowed charges” summarize total reimbursement

47. 4-A-2-b-ii: Billed Charges • Can be used • Must be adjusted with hospital department- specific cost-to-charge ratios • Even then, imperfect: a single cost-charge ratio is used for all services in a department

48. 4-A-2-b-iii: Cost-Accounting Systems • Costs calculated from time and motion studies • Most common system is TSI (Transition System, Inc.)

49. 4-A-2-b-iv: Price References • Drugs: “Pharmacy Red Book” of average wholesale prices • There are varied publicly available estimates for health worker hourly wages, diagnostic and laboratory equipment, and even for most supplies

50. 4-A-3: Cost Data Bases • Record both resources and costs for specific diseases • Examples: • California Office of Statewide Health Planning & Development (OSHPD) hospital discharge data base • Medical Expenditure Panel Survey