Multiple intervention reviews

1. Multiple intervention reviews Reflections from CoEds discussions this week

2. Portfolio management • initiated and managed by CRGs • splitting lumped reviews at update stage • OoRand MTM not always necessary/appropriate • requires strong rationale for review (why is this an important question, what are the important outcomes, why are we addressing it in this way?) • but it’s what our readers want and often need • overviews are about differences in effect across several interventions, rather than average effect of individual treatments • judicious approach required

3. What overview options are available and when should they be used? • Traditional overview of reviews (summary/synthesis by informal inappropriate interpretation at the level of review?) • NB Narrative summary can guide the reader against making inappropriate conclusions from indirect comparisons – there has been too much emphasis on the statistical synthesis • Multiple intervention reviews (summary/synthesis at the level of the trial?)

4. AND...... • What are the considerations when deciding to use one or the other? Eg: • author team skills • author/editorial team resources • general assumptions reasonable • exchangeability (patients) assumption often problematic eg – OTC vs prescribed acne treatments) • must be important questions where ranking treatments is the next logical step • what summary info is a available from cochrane/non-cochrane reviews • are the data up to date?

5. Other suggestions for editorial mgt • Need for ‘layered’ approach – eg-overview with MTM MA, narrative summary overview, individual reviews? • Could protocols for overviews be written to include MTM synthesis as an optional extra • Should there be a separate stage in the editorial process for these reviews once the preliminary work has been undertaken? • What process might be used for making the decision to tackle overview topics? (Decision-tree?)

6. How/when can CRGs really begin to adopt these methods? • currently no guidance • limited support from MG ( under what circumstances can CRGs expect support from CMIMG). • who has sign-off? • how important is it that Cochrane has a consistent approach to these issues? • what happens during the period where there is no clear guidance and methods/Handbook chapter are in transition phase?

7. Challenge of updating • Should overviews and MTM always be considered at updating stage (where field ripe, where lumped reviews exist, etc)? • Updating multiple individual reviews to produce data appropriate for overview • ‘Hostage to fortune’ - future challenge of updating overview and component reviews – software needs to work harder is this is to be part of our core activity

8. Software • RevMan/ CRS issues – what options for analysis? • RM Structural changes around methods • Checking data (internal and external) • Possibilities for extracting the data once only and store for later checking and reanalysis (CRS/SRDR depending on nature and status of available data?)

9. Interpretation of data • Challenge of reporting what was done, what was found, and what it means • Authors struggle to interpret data from pairwise direct head to head IRs, even more where lumped reviews include multiple interventions • Even more advanced skills required to understand what the data can and cannot tell you, and to avoid focus on the statistical aspects of the review (complexity and black box) • SoFT(Applicability and Recommendations MG)