THE INCORPORATION OF META-ANALYSIS RESULTS INTO EVIDENCE-BASED DECISION MODELLING

1. THE INCORPORATION OF META-ANALYSIS RESULTS INTO EVIDENCE-BASED DECISION MODELLING Nicola Cooper, Alex Sutton, Keith Abrams, Paul Lambert Department of Epidemiology & Public Health, University of Leicester. PSI Meeting “Statistical Advances in Health Technology Assessment ” 10th June 2003

2. BACKGROUND • Increasingly decision models are being developed to inform complex clinical/economic decisions • Parameters can include: • clinical effectiveness, • costs, • disease progression rates, and • utilities • Evidence based - use systematic methods for evidence synthesis to estimate model parameters with appropriate levels of uncertainty

3. SOURCES OF UNCERTAINTY IN DECISION MODELS • Statistical error • Systematic error • Evidence relating to parameters indirectly • Data quality, publication bias, etc.

4. COMPREHENSIVE DECISION MODEL FRAMEWORK DATA SOURCES RCT1 RCT2 RCT3 OBS1 OBS2 ROUTINE EXPERT Opinion pooling EVIDENCE SYNTHESIS Meta-analysis Gen. synthesis Bayes theorem In combination Adverse Events Clinical Effect MODEL INPUTS Cost Utility DECISION MODEL

5. EXAMPLES • Net Clinical Benefit Approach • Warfarin use for atrial fibrillation • Simple Economic Decision Model • Prophylactic antibiotic use in caesarean section • Markov Economic Decision Model • Taxane use in advanced breast cancer

6. GENERAL APPROACH • Meta-analyse available evidence to obtain a distribution for each model parameter using random effect models • Transformationof the pooled results, if necessary, and input into the model directly as a distribution and evaluate the model • All analyses (decision model and subsidiary analyses) implemented in one cohesive statistical model/program • Implemented in a fully Bayesian way using Markov chain Monte Carlo simulation within WinBUGS software • All prior distributions intended to be ‘vague’. Where uncertainty exists in the value of parameters (i.e. most of them!) they are treated as random variables

7. EXAMPLE 1: NET CLINICAL BENEFIT Warfarin for Non-Rheumatic Atrial Fibrillation • Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin • However, there is a risk of a fatal hemorrhage as a result of taking warfarin • Do the benefits outweigh the risks?

8. ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Net Benefit BENEFITS minus HARMS = NET CLINICAL BENEFIT (if NCB >0 benefits outweigh harms) EVALUATION OF NET BENEFIT EVALUATION OF NET BENEFIT

9. EVALUATION OF NET BENEFIT EVALUATION OF NET BENEFIT Meta analysis Multivariate risk equations of RCTs ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Meta analysis of Net Benefit QoL study RCTs obs studies

10. 10 6 8 6 4 4 2 2 0 0 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -1.5 -1.0 -0.5 0.0 0.5 1.0 reduction in relative risk 300 250 0.4 200 150 0.3 100 0.2 50 0.1 0 0.002 0.004 0.006 0.008 0.010 0.012 0.014 risk of bleed per year 0.0 0 20 40 60 80 100 Outcome ratio EVALUATION OF NET BENEFIT EVALUATION OF NET BENEFIT Meta analysis Multivariate risk equations of RCTs ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Meta analysis of Net Benefit QoL study RCTs obs studies

11. 10 6 8 6 4 4 2 2 0 0 -2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65 -1.5 -1.0 -0.5 0.0 0.5 1.0 reduction in relative risk 300 250 0.4 200 150 0.3 100 0.2 50 0.1 0 0.002 0.004 0.006 0.008 0.010 0.012 0.014 risk of bleed per year 0.0 0 20 40 60 80 100 Outcome ratio Multivariate Risk Equation Data Net Benefit (measured in stroke equivalents) No. T hrombo - Clinical No. of % of embolism Mean Median Probability of risk patients cohort rate (% (s.e.) (95% Benefit > 0 Simulated PDF factors per year CrI) (95% CI)) 6 5 4 2 or 3 68 12 17.6 (10.5 - 0.0004 0.06 54.2 % 3 to 29.9) (0.15) ( - 0.29 to 2 0.20) 1 0 -0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 2 or 3 Clinical factors EVALUATION OF NET BENEFIT EVALUATION OF NET BENEFIT Meta analysis Multivariate risk equations of RCTs ´ Relative reduction in risk of stroke) (Risk of stroke ´ - (Risk of fatal bleed Outcome ratio) = Meta analysis of Net Benefit QoL study RCTs obs studies

12. “TAKE-HOME” POINTS 1 • Net-benefit provides a transparent quantitative framework to weigh up benefits and harms of an intervention • Utilises results from two meta-analyses and allows for correlation induced where studies included in both benefit and harm meta-analyses • Credible interval for net benefit can be constructed allowing for uncertainty in all model parameters

13. EXAMPLE 2: SIMPLE DECISION TREE • Use of Prophylactic Antibiotics to Prevent Wound Infection following Caesarean Section

14. METHOD OUTLINE • Cochrane review of 61 RCTs evaluating prophylactic antibiotics use for caesarean section • Event data rare: use “Exact” model for RR • Meta-regression: Does treatment effect vary with patients’ underlying risk (pc)? ln(RRadjusted ) = ln(RRaverage)+  [ln(pc) - mean(ln(pc))] • Risk of infection without treatment from ‘local’ hospitaldata (p1) • Derive relative risk of treatment effect for ‘local’ hospital (using regression equation with pc=p1) • Derive riskof infection if antibiotics introduced to ‘local’ hospital (p2) p2 = p1 * RRadjusted

15. =0.24 (-0.28 to 0.81) No treatment effect Local hospital event rate UNDERLYING BASELINE RISK

16. p2 RR 0.02 (0.02 to 0.03)  p1  cost using antibiotics -£49.53 (-£77.09 to -£26.79) 0.08 (0.06 to 0.10) p1 RESULTS

17. SENSITIVTY OF PRIOR DISTRIBUTIONS Caterpillar plot [1] [1] Gamma(0.001,0.001) on 1/variance [2] Normal(0,1.0-6) truncated at zero on 1/sd [3] Uniform(0,20) on 1/sd [2] [3] -80 -60 Cost -40 -20

18. “TAKE-HOME” POINTS 2 • Incorporates M-A into a decision model adjusting for a differential treatment effect with changes in baseline risk • Meta-regression model takes into account the fact that covariate is part of the definition of outcome • Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propogated • Sensitivity of overall results to prior distribution placed on the random effect term in a M-A

19. QR , CR QS , CS PSR PR PS PRP PSP QP , CP PP PPD Probability (P) Quality of Life (Q) Cost (C) QD = 0 EXAMPLE 3: MARKOV MODEL Taxanes - 2nd line treatment of advanced breast cancer Response Stable  Progressive Death Cycle length 3 weeks

20. MODEL PARAMETER ESTIMATION PSR, TAX – The probability of moving from stable to response in a 3 week period 1) M-A of RCTs: Annual ln(odds) of responding • 2) Pooled ln(odds) distribution Chan Nabholtz Sjostrom Bonneterre -0.3 (-0.9 to 0.3) Combined PSR .1 .25 1 5 Odds - log scale Respond Stable 3) Transformation of ln(odds) distrn to transition probability 4) Apply to model Progressive Death

21. THE REMAINING PARAMETERS • The Transition Probabilities need estimating for each intervention being compared • Costs & Utilities can be extracted from the literature and synthesised using a similar approach within the same framework

22. META-ANALYSES OF LITERATURE (where required)

23. TRANSITION PROBABILITIES FOR MODEL (Derived from M-As)

24. EVALUATION OF THE MODEL • A cohort of 1,000 persons is run through the model over 35 3-weekly cycles (until the majority of people are dead) for each treatment option • Costs and utilities are calculated at the end of each cycle and the average cost and utilities for an individual across all 35 cycles for each treatment option are calculated • This process is repeated 4,000 times (each time different values from each parameter distribution are sampled)

25. COST-EFFECTIVENESS PLANE Bayesian (MCMC) Simulations NE NW £10,000 £8,000 £6,000 Taxanemore Standard effectivebut £4,000 dominates more costly Incremental cost £2,000 £0 -0.50 -0.40 -0.30 -0.20 -0.10 0.00 0.10 0.20 0.30 0.40 0.50 -£2,000 Taxane Taxane less costly but less dominates -£4,000 effective SW SE Incremental utility

26. C-E ACCEPTABILITY CURVE

27. ELICITATION OF PRIORS e.g. Response Rate Taxane Standard

28. SENSITIVITY ANALYSIS

29. “TAKE-HOME” POINTS 3 • Synthesis of evidence, transformation of variables & evaluation of a complex Markov model carried out in one unified framework (facilitating sensitivity analysis) • Provides a framework to incorporate prior beliefs of experts

30. ADVANTAGES OF APPROACH • Synthesis of evidence, transformation of variables & evaluation of a complex decision model carried out in a unified framework • Facilitates sensitivity analysis • Provides a framework to incorporate prior beliefs of experts • Allows for correlation induced where studies included in the estimation of more than one parameter • Uncertainty in all model parameters automatically taken into account • Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propagated

31. FURTHER ISSUES • Handling indirect comparisons correctly • E.g. Want to compare A vs. C but evidence only available on A vs. B & B vs. C etc. • Avoid breaking randomisation • Necessary complexity of model? • When to use the different approaches outlined above? • Incorporation of Expected Value of (Perfect/Sample) Information • Incorporation of all uncertainties

32. REFERENCES • Cooper NJ, Abrams KR, Sutton AJ, Turner D, Lambert P. Use of Bayesian methods for Markov modelling in cost-effectiveness analysis: An application to taxane use in advanced breast cancer. Journal of the Royal Statistical Society Series A 2003; 166(3). • Cooper NJ, Sutton AJ, Abrams KR, Turner D, Wailoo A. Comprehensive decision analytical modelling in economic evaluation: A Bayesian approach. Health Economics 2003(In press) • Cooper NJ, Sutton AJ, Abrams KR. Decision analytical economic modeling within a Bayesian framework: Application to prophylactic antibiotics use for caesarean section. Statistical Methods in Medical Research 2002;11: 491-512. • Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR. Synthesising both benefit and harm: A Bayesian approach to evaluating clinical net benefit. (Submitted toJournal of Clinical Epidemiology).