1 / 40

The potential role of mixed treatment comparisons

The potential role of mixed treatment comparisons. Deborah Caldwell Tony Ades MRC HSRC University of Bristol. Outline of presentation. Indirect comparisons and mixed treatment comparisons (MTC). Potential concerns regarding use of indirect comparisons/ MTC.

sheba
Download Presentation

The potential role of mixed treatment comparisons

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The potential role of mixed treatment comparisons Deborah Caldwell Tony Ades MRC HSRC University of Bristol

  2. Outline of presentation • Indirect comparisons and mixed treatment comparisons (MTC). • Potential concerns regarding use of indirect comparisons/ MTC. • Hypothetical ‘simulation’ example of MTC evidence structure. • MTC of NICE appraisal for thrombolysis • Addressing potential concerns • Should MTC be routine and future areas of research.

  3. Background • For any given condition there are an array of possible interventions/ treatments. • Treatment recommendations & decisions should be evidence based. • Principle sources are systematic reviews of randomised controlled trials. • Systematic reviews focus on pairwise, direct comparisons of treatments.

  4. Indirect comparisons • In absence of trials comparing treatments A versus B, an indirect estimate of odds ratio dABis obtained from RCTs comparing A vs C and B vs C:dAB = dAC – dBC

  5. Indirect comparisons • In absence of trials comparing treatments A versus B, an indirect estimate of odds ratio dABis obtained from RCTs comparing A vs C and B vs C:dAB = dAC – dBC

  6. Indirect comparisons • In absence of trials comparing treatments A versus B, an indirect estimate of odds ratio dABis obtained from RCTs comparing A vs C and B vs C:dAB = dAC – dBC

  7. Mixed treatment comparisons • Where there are 3 or more treatments, compared using direct and indirect evidence from several RCTs = mixed (multiple) treatment comparisons (MTC). • MTC evidence structures are pervasive in Health Technology Assessments (HTA) – decisions between >5 treatments are commonplace. • A unified, coherent analysis of multiple treatments can only be achieved by including the entire evidence structure of relevant RCTs.

  8. Potential concerns about MTC • Indirect comparisons produce relatively imprecise estimates of treatment effect • They are not randomised comparisons • Suffer the biases of observational studies (level 3 of EBM evidence hierarchies?) • Direct and indirect evidence should be considered separately. • Direct evidence should take precedence.

  9. Contradictions in the ‘received wisdom’ • Why is lower level evidence used when level one is unavailable but it is irrelevant when it isn’t? • What do we do when direct evidence is inconclusive but in combination with indirect is conclusive? • If 5 treatments are all compared with each other does it make sense to separate the 10 direct pairwise comparisons from the 70 indirect?

  10. Hypothetical evidence structure

  11. Objectives • ‘Simulation’ exercise to explore benefit of increasing levels of complexity in MTC evidence structures. • To examine additional benefit of including evidence routinely excluded from systematic reviews. • To what extent different MTC evidence structures give increasing levels of precision. • Address some of the concerns outlined.

  12. Method • Contrast estimates of posterior precision of log odds ratios (LOR) from • A standard pairwise meta-analysis • Use of mixed treatment comparison analysis • Compare estimates of posterior precision • LOR of treatment A vs. treatment B • ‘Average’ precision - across all 15 possible treatment comparisons. • Assumptions: • Equal amounts of information on each treatment comparison.

  13. Simulation results: precision of dAB • Precision of pairwise dAB = 1 • Precision of MTC dAB = 1.01

  14. Hypothetical evidence structure

  15. Simulation results: precision of dAB • Additional data on a single indirect comparison increases precision by 0.51

  16. Simulation results: precision of dAB • Each additional indirect treatment comparison increases precision in dAB by 0.5

  17. Simulation results: precision of dAB • Is there value in ‘linking’ indirect comparisons? • ‘Linking’ comparison is treatment C vs D

  18. Simulation results: precision of dAB • Adding ‘linking’ comparisons doesn’t increase precision of dAB estimate. • Property of this particular evidence structure

  19. Summary of simulation results: precision of dAB • If all you believe is ‘direct’ data • precision dAB stays = 1 • Mixed Treatment Comparison analysis • Adding data on a single indirect comparison increases precision by 0.51 • Adding multiple indirect comparisons further increases precision • Equivalent to 2 extra trials on dAB comparison

  20. Simulation results: ‘average’ precision

  21. Simulation results: ‘average’ precision • 5 pieces of data/ 15 possible treatment comparisons.

  22. Simulation results: ‘average’ precision • Maximum ‘average’ pairwise precision = 1. • 15 pieces of data/ 15 possible treatment comparisons

  23. Simulation results: ‘average’ precision • MTC allows us to say something about all 15 pairwise comparisons.

  24. Simulation results: ‘average’ precision • Equivalent number of trials • 0.67*15 = 10 trials worth of data

  25. Early thrombolysis for acute myocardial infarction (AMI).

  26. Technology appraisal for National Institute for Clinical Excellence (Boland et al, 2003) • Affects 274,000 people each year • 50% die within 30 days of AMI. • National Service Framework for heart disease states thrombolysis should be given within 60 minutes. • Thrombolysis = pharmaceutical agents to dissolve blood clots.

  27. Thrombolytic treatments • Four treatments assessed: • Streptokinase (SK), • Tissue-plasminogen activator (t-PA), • Tenecteplase (TNK) • Reteplase (r-PA). • Distinction made between accelerated t-PA and standard t-PA. • SK + t-PA used in two trials

  28. Thrombolysis conclusions(Boland et al, 2003) • “Definitive (sic) conclusions on efficacy are that streptokinase is as effective as non-accelerated alteplase, that tenecteplase is as effective as accelerated alteplase, and that reteplase is at least as effective as streptokinase. • “Some conclusions require interpretation of data, i.e. whether streptokinase is as effective as, or inferior to accelerated alteplase; and whether reteplase is as effective as accelerated alteplase or not. • “Depending on these, two further questions on indirect comparisons arise, whether tenecteplase is superior to as streptokinase or not and whether reteplase is as effective as tenecteplase or not.”

  29. What is needed? • A single statistical analysis providing estimates for all the 15 pairwise comparisons, between 6 treatments. • Using classical or Bayesian statistical methods. • An assessment of which of these treatments is most likely to be best. • Method • Bayesian Markov chain Monte Carlo method

  30. Were all relevant treatments included? • Primary percutaneous transluminal coronary angioplasty (PCTA). • Keeley et al meta-analysis of PCTA vs thrombolysis (22 RCTs) • PCTA is better than thrombolysis (OR 0.70 [0.58 – 0.85]) • But surely the relevant comparison is the ‘best’ thrombolytic NOT the ‘average’ one? • 7 treatments • 21 possible pairwise comparisons

  31. Extended evidence structure

  32. Consistency of odds ratios and CIs: fixed effect analysis

  33. Lumping vs splitting: Fixed effect analysis of At-PA versus PCTA

  34. Breaking randomisation? • NO! • There are statistically invalid methods. • Our MTC analyses are based only on randomised comparisons. • Lack of assumptions about baseline risks across studies. • A weighted combination of valid estimates of treatment effect.

  35. Generalisability • Key assumption in MTC is that relative treatment effect of one treatment vs another is same across entire set of trials. • Irrespective of which treatments were actually evaluated in each trial • True odds ratio of A vs B trials is exactly the same as the A vs B odds ratio in the A vs C, B vs C trials. (fixed effect) • Common distribution of treatment effects is the same across all sets of trials (random effects).

  36. Generalisability • Helpful to consider which target population we are making treatment recommendation for. • The type of patients in the previous A vs B trials? OR • The kind of patients in ALL the MTC trials? • Clinical and epidemiological judgement necessary • Poor judgements may introduce heterogeneity

  37. Should MTC be routine? • A more appropriate question is can MTC analyses be avoided? • No real alternative in multi-treatment decision making. • Transparency • No need to lump treatments • No ‘under the table’ indirect comparisons • MTC same assumptions as meta-analysis

  38. Future areas of research • What is the extra literature searching burden of MTC – how far should searches go? • Should we include discontinued treatments in the evidence base? • Placebo controlled trials? • Greater awareness of MTC by commissioners of research when ‘scoping’ HTAs • NICE Obesity appraisals • Thrombolysis & PCTA

More Related