IOM workshop: finding and assessing individual studies

1. IOM workshop: finding and assessing individual studies David ToveyEditor in Chief, The Cochrane Library

2. Agenda • General Comments • Search and identifying studies • Appraising individual studies • What challenges?

3. General comments • Fantastic piece of work • Good timing......almost • Moves systematic reviews to the next stage • Sets the bar high, but appropriate • Can’t cut corners....we agree!

4. Search and identifying studies • Very close to Cochrane model • Importance of protocol and utilising appropriate expertise; should include search strategy - and line-by-line search strings • Peer review of search • Unpublished studies and missing data • Importance of registries, conference abstracts, contacts with triallists, industry etc • No RCT filters...surprising? • Observational studies for harms (and benefits?) • Large administrative databases • Studies in LOE & updating searches prior to publication

5. Appraising individual studies • Importance of pre-specification of criteria in protocol • Dual appraisal/screening and dual data extraction • Importance of selective outcome reporting bias • Implies need to access trial protocol • Risk of bias: critical step • Per-outcome or per-study • Concept of intervention “fidelity” • What is the intervention and how reproducible is it? • How was it delivered

6. Challenges and major themes • Cost – quality – timeliness triad • Reporting bias & unpublished studies • Importance of licensing agency data • Observational studies • “Extended”, complex and different types of review • Knowledge translation and implementation • Relevance, applicability and timeliness

7. Summary • A major step forward • Some differences, but • Some minor and bridgeable • Some reflect different purpose and structures • Agreement on major challenges • Strong arguments for increasing co-operation between agencies • Reduce duplication, improve efficiency and quality