Previous Lectures

1. Health and Cost Data InputsAdvanced Training in Clinical Research DCEA Lecture 4February 3, 2011UCSF Department of Epidemiology and Biostatistics

2. Previous Lectures 1 - Introduction to Decision Analysis 2 - Consideration of Utility and QALYs 3 - CEA Overview/Developing an Analysis/Costs

3. Cost-Effectiveness Ratio  Costs (Numerator) _________________________________ • Health (Denominator) The Incremental Cost of Obtaining an Additional Unit of Health From One Decision Compared With Another Decision (How much additional health do we receive for each additional dollar spent?)

4. Now: Data Inputs • What type of data do we use to determine • The health inputs that are changing? • The costs that are changing? • Where do we find these data?

5. Today’s Objectives:Data Type, Quality & Source • To Understand the General Issues in Gathering and Presenting Health and Cost Data Inputs • To Understand Data Sources and Synthesis Methods for Health and Cost Inputs • To Understand Common Criticisms Surrounding Health and Cost Data Inputs

6. Lecture Structure A. Data Type, Quality, & Source • General Approach • Hierarchies of Types of Studies • Health Inputs • Cost Inputs • Presentation of Inputs • Review/References • Data Searches 7. Resources available at UCSF 8. Using PubMed 9. Other Sources of Data … the big G

7. 1: General Approach A. Research Question & Conceptual Model B. Model Inputs: 1. Measures of Health States & Preferences 2. Measures of Resource Use C. Tradeoffs - Refine Model Given Data Availability D. Best Estimates & Plausible Ranges - Base Case & Range

8. Tradeoffs--Refine the Model, Given Data Availability 1. CEA models usually have many inputs 2. Resources are limited a. Your time b. Your budget c. Time for decision to be made 3. Therefore, model can be modified to utilize reliable data currently available. Iterative process highlights critical vs. marginal components.

9. Best Estimates and Plausible Ranges 1. Best Estimate = Base Case = Baseline a. The most likely value for the input: the value in the center of the best available data. b. You can intentionally err in one direction to prove the strength of your result. • 2. Plausible Range: similar to 95%, not 99.99%, C.I. • a. When using a single empirical data base, calculate a formal confidence interval (typically + or – 95%). • b. With multiple sources, informal use of best range.

10. 1-Example: Model – To Clip Or Not To Clip

13. 2: Hierarchies of Types of Studies • Hierarchy used by this Decision Cost-Effectiveness Analysis Course • Hierarchy recommended by the U. S. Preventive Services Task Force • Hierarchy used by Drummond, et al. in “Methods for the Economic Evaluation of Health Care Programmes” • Melded Hierarchy

14. DCEA Hierarchy • Definitive trials (large randomized controlled trials) • Meta-analysis of trials • Systematic Review • Smaller Trials • Large Cohort Studies • Small Cohort Studies • Expert Opinion

15. Definitive Trials • Large RCTs (Randomized Controlled Trials) • Participants are assigned randomly to treatment group or control group • Confounding variables therefore randomly distributed across groups • Problems: i. Does study population represent your modeled group? ii. Clinical trial conditions > real world conditions

16. Meta Analysis • Combines several studies, diluting errors of a single study and increasing statistical power. • Problems: i. “File Drawer Syndrome” ii. Unresolvable heterogeneity iii. LeLorier suggests large RCT > Meta iv. Meta-analysis often requires $100,000 • The Cochrane Library has established useful protocols for Meta Analysis methodology.

17. Systematic Review • Excellent method: Far easier than Meta-Analysis, often with similar results. • Careful, critical, structured compendium of studies, summarizing methods & findings. • Describe: subjects, study design, outcome measurement, and outcomes. • Baseline & range selected by informal weighting of study size and quality.

18. Systematic Review (CONT.) • Most common method of obtaining inputs, often used for several inputs in a CEA. • When no definitive study is available (usually the case), this is far better than relying on a single non-definitive report. g. At least 95% less work than a formal Meta-Analysis.

19. 2 Smaller Trials • Unbiased if well-done, but imprecise due to small size. • Point estimate may be wrong, C.I. large. c. Often used, perhaps because of promising clinical results . . . . Reader beware: recall “File Drawer Syndrome”.

20. Large Cohort Studies • Cohort = persons already exposed to risk factor, treatment, etc., vs. unexposed or untreated control group. • No random assignment, therefore selection bias and confounding variables. • In untreated populations (when no treatment was available), can be used to estimate outcomes.

21. Small Cohort Studies • Same problems as large cohort studies. b. Additionally, imprecise due to small size.

22. Expert Opinion • Virtues: i. Absent data, it’s all you’ve got. You need to use something; expert opinion is common for 1-2 inputs in a CEA. ii. Experts do know a lot. iii. Experts are influential & can help you. b. Problems: they’re just opinions; small sample creates bias or random error.

23. U. S. Preventive Services Task Force Hierarchy • Randomized Controlled Trials. • Cohort Studies. 3. Case-Control Studies.

24. Case-Control Studies • Study and control groups selected on the basis of having or not having the disease. • Therefore, retrospective study. • Problems: i. Confounding variables hard to identify. ii. Observer bias: health outcome known. iii. Recall bias: imperfect recall. iv. Selection problems similar to cohort studies.

25. Drummond, et al. Hierarchy • Randomized Controlled Trials. • Cohort Studies. • Case-Control Studies. • Case Series. a. No control group. b. E.g.—Heart Transplant in UK: Untreated group considered unethical. c. Common with new procedures.

26. Drummond, et al. (cont.) 5. RCTs may lack precision important to economic analysis but not important to clinical outcomes. 6. Precise clinical setting creates unbiased and precise data: high signal to noise (high internal validity). 7. Clinical precision unlikely to be replicated in real world, so results might not be generalizable (low external validity).

27. 2-D: Melded Hierarchy • Definitive trials (large RCTs) • Meta-analysis of trials • Systematic Review • Smaller Trials • Large Cohort Studies • Small Cohort Studies • Case-Control Studies • Case Series • Expert Opinion

28. 3: Health Inputs • Overview • Steps • How to Find Inputs

29. Rough Outline of the Markov Model Rough Draft of Health States

30. Health Inputs Overview 1. Health State Outcomes a. Relevant Outcome States b. Probability Estimates 2. Health Preferences Weights a. Preference Weights for Outcomes b. Utilities, QALYs 3. Population Characteristics a. Relevant Population b. Disease Prevalence in Population

31. Health Inputs - Steps • List Potential Health Outcome States and Relevant Population • Find Data on States & Probabilities - Start With Comprehensive Literature Search • Find Data on Utilities • Find Data on Population Characteristics

32. Health States Key Questions: • What Are the Relevant Health States Over Time for the Disease Under Study? • When Do These States Occur, and How Long Do They Last? • What Are the Likely Side Effects or Other Unintended Consequences for Each Group? What Are Key Outcomes of Interest for Relevant Stakeholders, e.g. Patients, Clinicians, Payers, Employers, Policy Makers, Society as a Whole? • For Which Health States, Are There Credible Estimates? • Are These Estimates Appropriate for Your RQ and for Your Population?

33. Example – Aneurysm Analysis For the aneurysm analysis, health outcomes were estimated from multiple sources: • Aneurysm rupture rates large cohort study • SAH case fatality meta-analysis • SAH disability medium cohort study, • meta-analysis • RR mortality with disability small cohort study • Surgical mortality, disability meta-analysis • RR rupture (= 0) expert opinion (informal)

34. Preference Weights - Utilities a. Disease-specific Utilities b. Generic Utilities c. Key Questions: i. Do Credible Estimates Exist for Your RQ? ii. Are the Data Appropriate for Your RQ and Your Model? iii. Whose Perspective Are You Taking? iv. Disease Specific Ratings v. Community Ratings vs. Patient Ratings

35. Population Characteristics a. Prevalence of the Disease b. Key Questions: i. What is the Relevant Disease Prevalence? ii. National, Representative Samples iii. Disease Surveillance Databases iv. Integrated Delivery System Databases v. Claims Databases c. What Competing Risks Exist (Unrelated to RQ)? d. Do Credible Estimates Exist? e. Are These Estimates Appropriate for Your RQ?

36. Example: Population Characteristics - Aneurysm Analysis • Prevalence of disease - not needed, not a study about screening or an estimate of total societal costs • All-cause mortality - very important because of the low risk of aneurysm rupture and hence the high risk of dying before rupture occurs • Estimated by age and sex from a data base maintained at CDC, available on the internet.

37. Health—How to Find Inputs • Comprehensive Literature Review • Primary Data Collection • Commonly Used Health Data Sources 4. Current Recommendations

38. Comprehensive Literature Review • Employ hierarchies presented in Part 2. • Always consider relevance to your model. • Revise your model when the cost of the data your model requires is greater than the benefit from not revising the model.

39. Commonly Used Health Data Sources a. Clinical Trials b. CMS/VA Databases c. Disease Registries d. Quality of Well-Being Index (QWB), Health Utilities Index (HUI) - www.healthutilities.com/overview.htm e. Disability/Distress Index, EuroQol Instrument

40. Current Recommendations*:US Panel on Cost-effectiveness in Health and Medicine a. Provide a Reference Case Analysis b. Select Data Inputs From Highest Quality Sources That Are Relevant to the RQ and the Population c. Expert Opinion Is Relevant Only When No Other Adequate Data Exists d. Use QALYs to Incorporate Morbidity and Mortality Into a Single Measure e. Use Community Preferences for Health States f. Perform a Sensitivity Analysis on Inputs * See Gold MR et al. for Complete Set of Recommendations

41. [Perspective] • Societal • Government • Private insurance

42. Costs included in a cost-effectiveness analysis of a harm reduction behavioral program

43. 4: Cost Inputs • Direct Costs • Fixed vs. Variable Costs • Time Costs • Current Recommendations

44. Direct Costs 1. Published Estimates 2. Resources Used x Cost per unit Used 3. Cost Data Bases

45. Published Estimates a. If available, these are the easiest b. Should be well-done c. Health care scenarios should match d. Often, costs must be updated

46. Should be well done i. Sources of data used should be documented. • The data used should be appropriate (e.g.—not unadjusted, unnegotiated billed charges) iii. Algorithms used to process that data should be transparent and credible

47. Health-Care Scenarios Should Match i. Diagnosis & severity/complications • Age mix • Care practices • Care setting

48. Costs must be Updated • Adjust using the medical component of the Consumer Price Index (CPI), maintained by the Bureau of Labor Statistics (BLS) • Substitute particular unit costs with updated values for the same services • Whichever method is used, all costs must be denominated in a single currency adjusted to a single year (ref at the end)

49. Resources Used x Cost per Unit Used • Resources b. Costs

50. Resources i. Theoretical model • Empirical evidence—typically better, but more difficult • Micro vs. Macro data