jth: John starts

1. jth: John starts Health and Cost Data InputsAdvanced Training in Clinical Research DCEA Lecture 4February 17, 2005UCSF Department of Epidemiology and BiostatisticsBrian Harris, BA, MPP Candidatebrianharris1113@hotmail.com

2. Special Thanks To: John Hsu, MD, MBA, MSCE KFRI/DOR jth@dor.kaiser.org for developing this lecture.

3. Additional Thanks: • Guide to CLINICAL PREVENTIVE SERVICES: Report of the U. S. Preventive Services Task Force • Methods for the Economic Evaluation of Health Care Programmes, 2nd Edition (Drummond, O’Brien, Stoddart & Torrance)

4. Lectures Introduction to Decision Analysis Consideration of Utility and QALYs CEA Overview/Developing an Analysis

5. Cost-Effectiveness Ratio  Costs (Numerator) _________________________________ • Health (Denominator) The Incremental Cost of Obtaining an Additional Unit of Health From One Decision Compared With Another Decision (How much additional health do we receive for each additional dollar spent?)

6. Now: Data Input. Today’s Objectives • To Understand the General Issues in Gathering and Presenting Health and Cost Data Inputs • To Understand Data Sources and Synthesis Methods for Health and Cost Inputs • To Understand Common Criticisms Surrounding Health and Cost Data Inputs

7. Lecture Structure • General Approach • Hierarchies of Types of Studies • Health Inputs • Cost Inputs • Presentation of Inputs 6. Review/References

8. 1: General Approach A. Research Question & Conceptual Model B. Model Inputs: 1. Measures of Health States & Preferences 2. Measures of Resource Use C. Tradeoffs - Refine Model Given Data Availability D. Best Estimates and Plausible Ranges - Base Case and Range

9. 1-C: Tradeoffs--Refine the Model, Given Data Availability 1. CEA models usually have several inputs 2. Resources are limited a. Your time b. Your budget c. Time for decision to be made 3. Therefore, model can be modified to utilize reliable data currently available

10. 1-D: Best Estimates and Plausible Ranges 1. Best Estimate = Base Case = Baseline a. The most likely value for the input: the value in the center of the best available data. b. You can intentionally err in one direction to prove the strength of your result. • 2. Plausible Range: similar to 95%, not 99.99%, C.I. • a. When using a single empirical data base, calculate a formal confidence interval (typically + or – 95%). • b. With multiple sources, informal use of best range.

11. 1-Example: Model – To Clip Or Not To Clip

12. 1-Example: Data Inputs

13. 2: Hierarchies of Types of Studies • John Hsu Hierarchy • U. S. Preventive Services Task Force Hierarchy C. Drummond, et al. Hierarchy

14. 2-A: John Hsu Hierarchy • Definitive trials (large randomized controlled trials) • Meta-analysis of trials • Systematic Review • Smaller Trials • Large Cohort Studies • Small Cohort Studies • Expert Opinion

15. 2-A-1: Definitive Trials • Large RCTs (Randomized Controlled Trials) • Participants are assigned randomly to treatment group or control group • Confounding variables therefore randomly distributed across groups • Problems: i. Does study population represent modeled group? ii. Clinical trial conditions > real world conditions

16. 2-A-2: Meta Analysis • Combines several studies, diluting errors of a single study and increasing statistical power. • Problems: i. “File Drawer Syndrome” ii. Unresolvable heterogeneity iii. LeLorier suggests large RCT > Meta iv. Meta-analysis often requires $100,000

17. 2-A-3: Systematic Review • Excellent method: Far easier than Meta-Analysis, often with similar results. • Careful, critical, structured compendium of studies, summarizing methods & findings. • Describe: subjects, study design, outcome measurement, and outcomes. • Baseline & range selected by informal weighting of study size and quality.

18. 2-A-3: Systematic Review (CONT.) • Most common method of obtaining inputs, often used for several inputs in a CEA. • When no definitive study is available (usually the case), this is far better than relying on a single non-definitive report. g. At least 95% less work than a formal Meta-Analysis.

19. 2-A-4: Smaller Trials • Unbiased if well-done, but imprecise due to small size. • Point estimate may be wrong, C.I. large. c. Often used, perhaps because of promising clinical results . . . . Reader beware.

20. 2-A-5: Large Cohort Studies • Cohort = persons already exposed to risk factor, treatment, etc., vs. unexposed or untreated control group. • No random assignment, therefore selection bias and confounding variables. • In untreated populations (when no treatment was available), can be used to estimate outcomes.

21. 2-A-6: Small Cohort Studies • Same problems as large cohort studies. b. Additionally, imprecise due to small size.

22. 2-A-7: Expert Opinion • Virtues: i. Absent data, it’s all you’ve got. You need to use something; expert opinion is common for 1-2 inputs in a CEA. ii. Experts do know a lot. iii. Experts are influential & can help you. b. Problems: they’re just opinions; small sample creates bias or random error.

23. 2-B: U. S. Preventative Services Task Force Hierarchy • Randomized Controlled Trials. • Cohort Studies. 3. Case-Control Studies.

24. 2-B-3: Case-Control Studies • Study and control groups selected on the basis of having or not having the disease. • Therefore, retrospective study. • Problems: i. Confounding variables hard to identify. ii. Observer bias: health outcome known. iii. Recall bias: imperfect recall. iv. Selection problems similar to cohort studies.

25. 2-C: Drummond, et al. Hierarchy • Randomized Controlled Trials. • Cohort Studies. • Case-Control Studies. • Case Series. a. No control group. b. E.g.—Heart Transplant in UK: Untreated group considered unethical. c. Common with new procedures.

26. 2-C: Drummond, (cont.) • RCTs may lack precision important to economic analysis but not important to clinical outcomes. • Precise clinical setting creates unbiased and precise data: high signal to noise (high internal validity). • Clinical precision unlikely to be replicated in real world, so results might not be generalizable (low external validity).

27. 3: Health Inputs • Overview • Steps • How to Find Inputs

28. 3-A: Health Inputs Overview 1. Health State Outcomes a. Relevant Outcome States b. Probability Estimates 2. Health Preferences Weights a. Preference Weights for Outcomes b. Utilities, QALYs 3. Population Characteristics a. Relevant Population b. Disease Prevalence in Population

29. 3-B: Health Inputs - Steps • List Potential Health Outcome States and Relevant Population • Find Data on States & Probabilities - Start With Comprehensive Literature Search • Find Data on Utilities • Find Data on Population Characteristics

30. 3-B-1 & 2: Health States Key Questions: • What Are the Potential Clinical Outcome Differences Across the Study Groups? • What Are the Relevant Health States Over Time for the Disease Under Study? • When Do These States Occur, and How Long Do They Last? • What Are the Likely Side Effects or Other Unintended Consequences for Each Group? • What Are Key Outcomes of Interest for Relevant Stakeholders, e.g. Patients, Clinicians, Payers, Employers, Policy Makers, Society As a Whole? • For Which Health States, Are There Credible Estimates? • Are These Estimates Appropriate for Your RQ (Research Question) and for Your Population?

31. 3-B-1 & 2: Example – Aneurysm Analysis For the aneurysm analysis, health outcomes were estimated from multiple sources: • Aneurysm rupture rates large cohort study • SAH case fatality meta-analysis • SAH disability medium cohort study, meta-analysis • RR mortality with disability small cohort study • Surgical mortality, disability meta-analysis • RR rupture (= 0) expert opinion (informal)

32. 3-B-3: Preference Weights - Utilities a. Disease-specific Utilities b. Generic Utilities c. Key Questions: i. Do Credible Estimates Exist for Your RQ? ii. Are the Data Appropriate for Your RQ and Your Model? iii. Whose Perspective Are You Taking? iv. Disease Specific Ratings v. Community Ratings V. Patient Ratings

33. 3-B-3-Example: Utilities – Aneurysm Analysis For the aneurysm analysis, utilities were estimated by extrapolation: • Disability studies of similar types of neurological disability • Worry formula calibrated to utility for similar anxiety • Mild symptoms studies of similar type of pain

34. 3-B-4: Population Characteristics a. Prevalence of the Disease b. Competing Risks c. Key Questions: i. What is the Relevant Disease Prevalence? ii. National, Representative Samples iii. Disease Surveillance Databases iv. Integrated Delivery System Databases v. Claims Databases • What Competing Risks Exist (Unrelated to RQ)? • Do Credible Estimates Exist? f. Are These Estimates Appropriate for Your RQ?

35. 3-B-4-Example: Population Characteristics - Aneurysm Analysis • Prevalence of disease - not needed, not a study about screening or an estimate of total societal costs • All-cause mortality - very important because of the low risk of aneurysm rupture and hence the high risk of dying before rupture occurs • Estimated by age and sex from a data base maintained at CDC, available on the internet.

36. 3-C: Health—How to Find Inputs • Comprehensive Literature Review • Primary Data Collection • Commonly Used Health Data Sources 4. Current Recommendations

37. 3-C-1: Comprehensive Literature Review • Employ hierarchies presented in Part 2. • Always consider relevance to your model. • Revise your model when the cost of the data your model requires is greater than the benefit from not revising the model.

38. 3-C-2: Health - Primary Data a. Clinical Trials b. Direct Health State Utilities / Preference Weights Assessments c. Expert Opinion

39. 3-C-3: Commonly Used Health Data Sources a. Clinical Trials b. CMS/VA Databases c. IDS Databases (KP) d. Disease Registries • Quality of Well-Being Index (QWB), Health Utilities Index (HUI) - www.healthutilities.com/overview.htm f. Disability/Distress Index, EuroQol Instrument

40. 3-C-4: Current Recommendations*:US Panel on Cost-effectiveness in Health and Medicine a. Provide a Reference Case Analysis b. Select Data Inputs From Highest Quality Sources That Are Relevant to the RQ and the Population c. Expert Opinion Is Relevant Only When No Other Adequate Data Exists d. Use QALYs to Incorporate Morbidity and Mortality Into a Single Measure e. Use Community Preferences for Health States f. Perform a Sensitivity Analysis on Inputs * See Gold MR et al. for Complete Set of Recommendations

41. 4: Cost Inputs • Cost Inputs—Types • Costs—Basic Definitions • Costs—Issues • Costs—How to Find Inputs • Commonly Used Cost Data Sources • Current Recommendations • Cost-Benefit Analysis

42. 4-A: Costs Inputs--Types 1. Direct Costs 2. Fixed v. Variable Costs 3. Time Costs

43. 4-B: Costs--Basic Definitions 1. Resources 2. Opportunity Costs 3. Money

44. 4-C: Costs--Issues 1. Published Estimates 2. TC = Q * C [Resources * Unit Costs] a. Quantity Consumed b. Cost per Unit 3. Discounting a. Time Value of Money b. Time Preference 4. Perspectives

45. 4-A-1: Direct Costs “Resources that are consumed in the provision of an intervention or in dealing with the side effects or other current and future consequences linked to it.” a. Direct Health Care Costs i. Diagnostic Tests, Drugs, Durable Medical Equipment ii. Health Care Personnel Costs b. Direct Non-health Care Costs i. Transportation Costs ii. Care-taker Costs, Child Care

46. 4-A-2: Fixed vs. Variable Costs • Fixed Costs • Variable Costs

47. 4-A-3: Time Costs a. Productivity Costs b. Morbidity Costs c. Mortality Costs d. Friction / Transaction Costs

48. 4-D: Costs—How to Find Inputs 1. Comprehensive Literature Review 2. Primary Data Collection

49. 4-D-1: Comprehensive Literature Review a. Employ hierarchies presented in Part 2. b. Relevance of Data c. Comparability of Data

50. 4-D-2: Cost Inputs - Primary Data Collection a. Existing Data Sources b. Charges c. Cost-accounting Systems d. Micro-cost Estimates / Time-motion Studies