Economic evaluation of health programmes

1. Economic evaluation of health programmes Department of Epidemiology, Biostatistics and Occupational Health Class no. 19: Economic Evaluation using Patient-Level Data II Nov 12, 2008

2. Plan of class • Collecting patient-level data alongside RCTs – some issues • Analysis of uncertainty

3. Collecting economic data alongside clinical trials

4. Issues with adding economic component to an RCT • Useful complement to efficacy data – very common in UK; and, relatively cheap • However: • Is comparison condition relevant? • Non-standard, more intensive measurement of outcomes that could affect results • Use of intermediate health outcomes • Inadequate follow-up period or sample size • Need to abstract from protocol-driven costs • Artificial rules mandated by protocol (selection of subjects, keeping them in study, adherence to Tx)

5. What to do? • ‘Pragmatic’ trials: • Patients representative of typical caseload • Routine follow-up • Meaningful and wide range of outcomes • Longer follow-up if relevant • Usually larger sample size • Unblinded patients and physicians • Not often done, particularly having a sample size powered for cost data • Modeling! • With adjusted trial data

6. Data collection methods • Add sheets to case report forms in hospital for hospital-based study • More or less detailed depending on what is being evaluated • Administrative data • Questionnaires and diaries • Accuracy of patient recall is an issue • Ideally monitor every 30 days

7. Statistical analysis of economic data • Nature of economic data • Skewed distribution (see examples next slide) • Missing data • Hopefully missing at random • Censored data • Difficulties with ICERs

8. Examples of skewed cost data For discussion: Why are health care cost data typicallyskewed?

9. 4 problems with ICERs

10. 1: Opposite meanings, same ICER

11. 2: If one of 3 interventions (each compared to a fourth one) is dominant, and all 3 ICERs are negative, magnitude of ICER is meaningless • Example: • A: (1 LY, - $2,000): ICER = -2,000 $/LY • B: (2 LY, - $2,000): ICER = -1,000 $/LY • C: (2 LY, - $1,000): ICER = - 500 $/LY • B is preferred yet is intermediate in value

12. 3: X/0 = Infinity! • If difference in effects can be zero, with non-zero probability, then values will become impossible to calculate. • If confidence interval spans negative and positive difference, ICER becomes discontinuous: 0 Difference in effects

13. 4: Ratios don’t work well as dependent variables in regressions • Often interesting to try to identify factors associated with cost-effectiveness!

14. Limits of hypothesis testing • Test H0 : No difference in effects • Even if cannot reject H0, this does not prove there is no difference (power of test could be low – high risk of Type II error). • Point estimate remains best estimate of difference • Mean incremental cost of £186 (95% CI, - £26 to £375) • Mean incremental QALYs: 0.007 (-0.008 to 0.023) • Decision-maker may be willing to accept higher risk than 0.05 of incorrectly rejecting null (may prefer to go with intervention even if not in fact more effective)

15. Confidence region for costs and effects

16. Cost-effectiveness acceptability curves

17. Incremental cost-effectiveness ratio Average cost per person: Experimental Tx (E) Control group(C) C E - C C E E - E C RCEI= Average value of effectiveness measure: Experimental group Control group (C)

18. Using the bootstrap to obtain a measure of the sampling variability of the ICER • Suppose we have nEXP et nCON observations in the experimental and control groups, respectively. One way to estimate the uncertainty around an ICER is to: • Sample nCON cost-effect pairs from the control group, with replacement • Sample nEXP cost-effect pairs from the experimental group, with replacement • Calculate the ICER from those two new sets of cost-effect pairs • Repeat steps 1 to 3 many times, e.g., 1000 times. • Plot the resulting 1,000 ICER values on the Cost-effectiveness plane See Drummond & McGuire, Eds., Economic evaluation in health care, Oxford, 2001, p. 189

19. An illustration of step 1 (Note: These are made-up data)

20. Going over the next steps again… • Do exactly the same steps for data from the experimental group, independently. • Calculate the ICER from the 2 bootstrapped samples • Store this ICER in memory • Repeat the steps all over again • Of course, this is done by computer. Stata is one program that can be used to do this fairly readily.

21. Bootstrapped replications of an ICER with 95% confidence interval Note: ellipses here are derived using Van Hout’s method and are too big; the bootstrap gives better results Source: Drummond & McGuire 2001, p. 189

22. Bootstrapped replications that fall in all 4 quadrants Source: Drummond & McGuire 2001, p. 193

23. A solution: the Cost-effectiveness acceptability curve • Strategy: We recognize that the decision-maker may in fact have a ceiling ratio, or shadow price RC – a maximum amount of $ per unit benefit he or she is willing to pay • So we will estimate, based on our bootstrapped replications, the probability that the ICER is less than or equal to the ceiling ratio, as a function of the ceiling ratio • If the ceiling ratio is $0, then the probability that the ICER is less than or equal to 0 is the p-value of the statistic from testing the null hypothesis that the costs of the 2 groups are the same • Recall that the p-value is the probability of observing the difference in costs seen in the data set (or a smaller one) by chance if the true difference is in fact 0.

24. Cost-effectiveness acceptability curve (CEAC) Source: Drummond & McGuire 2001, p. 195