Georgia Salanti, Julian Higgins, Valeria Marinho XIII Cochrane Colloquium Melbourne 2005

1. How to determine the best treatment: a mixed-treatment-comparisons meta-analysis (MTM) of trials of topical fluoride therapies for the prevention of dental caries Georgia Salanti, Julian Higgins, Valeria Marinho XIII Cochrane Colloquium Melbourne 2005

2. Background • A series of seven Cochrane reviews examines the effect of fluoride in preventing caries Marinho, Higgins, Sheiham, Logan. CDSR 2002-2004Fluoride in • Dentifrice • Rinse • Gel • Varnish • Outcome measure: SMD compares caries increment across the two groups • (P-D > 0 favours D) • Placebo, No treatment

3. 69 No treat 13 4 31 4 9 3 The data Dentifrice 1 3 6 Placebo Gel Varnish 1 4 Rinse

4. D – G=M2 –M1 SD= 3 The idea Dentifrice P – D = M1 Placebo 3 Extend the idea to analyse jointly the 130 studies by combining direct and indirect evidence in a RE model across the network 6 1 3 P – G = M2 Gel 31 Varnish 4 1 Rinse

5. MT Meta-analysis • RE model, fitted in WinBUGS, taking into account correlation in multi-arm trials (I spare you the technical details) • Joint analysis of all trials by taking advantage of indirect evidence: we gain precision! • Should we be tempted do so? • Check the validity of multiple treatments meta-analysis

6. Results DIC= – 82.12, Heterogeneity standard deviation = 0.20 (0.16,0.24) • Dentifrice appears as the best treatment • Placebo seems to have an effect?

7. The problem! Dentifrice Incoherence = weighted difference between direct and indirect evidence P – D = 0.33 D – G = – 0.12 (SE 0.06) Placebo 6 1 3 P – G = 0.21 D – G = –0.01 (SE 0.06) Gel 31 Varnish 4 1 Rinse Extended in all closed loops

8. Incoherence • How important is the apparent conflict between direct and indirect evidence? • Can we identify sources of incoherence? • What can we do to improve the agreement?

9. Incoherence in each loop Estimates with 95% confidence intervals Indirect and direct evidence combined using the inverse variance method Closed loops NGR NRV NGV PDR PDG Head to head comparison is overestimated when going through placebo Coherence seems better when we use another treatment as the intermediate step PDV PGR PRV PGV DGR DRV DGV GRV NGRV PDGR PDRV PDGV PGRV DGRV PDGRV - 1.0 - 0.5 0.0 0.5 1.0

10. Incoherence: Different placebo effects? Dentifrice Incoherence D – G = – 0.12 (0.06) Placebo P-Dentifrice D – G = – 0.01 (0.06) P-Gel Gel I cannot learn about D versus G through placebo Is this the case in my data?

11. Compare different placebo effects • All placebos work the same • Analyse separately NT and placebo controlled trials

12. Incoherence: Confounding Example: year of randomisation Dentifrice 69 studies! Incoherence 1968 Placebo 1973 1969 Gel

13. Possible confounders Differences in year reflect differences in baseline

14. MT Meta-regression ) ( P-T P-T year + β SMD = θ i i i

16. Is this the future of meta-analysis in the Cochrane collaboration? • Need ‘umbrella reviews’ to compare multiple interventions • for the same condition • Increased precision • Comprehensive ranking • But don’t get too excited!!! • Need very careful examination of the underlying assumptions (especially absence of confounders) • We need user-friendly tools to assess incoherence • Expertise is needed (Bayesian models, multi-arms studies)