How to do Cochrane systematic reviews: Introductory workshop

1. How to do Cochrane systematic reviews: Introductory workshop Dr Jeremy Grimshaw Canadian Cochrane Network and Centre University of Ottawa

2. Introductions • Introduction to systematic reviews • Introduction to The Cochrane Collaboration • Steps in doing a Cochrane review Coffee Break • Advanced methods • Appraising the risk of bias • Locating and selecting studies for your systematic review Outline of the course

3. Name • Background • Previous experience of systematic reviews • What you want to get out of the training? Introductions

4. Introduction to systematic reviews • Introduction to The Cochrane Collaboration • Rationale for developing a protocol • Format of a protocol • Resources to support reviewers Session 1: Outline

5. Over 20,000 health journals published per year • Published research of variable quality and relevance • Research users (consumers, health care professionals and policy makers) often poorly trained in critical appraisal skills • Average time professionals have available to read = <1 hour/week Problems of information management in healthcare

6. Systematic reviews use rigorous scientific methods to identify, assess and synthesise the worldwide available evidence Systematic reviews

7. A review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Cochrane Collaboration (2005) Glossary of Terms in The Cochrane Collaboration Systematic reviews

8. The use of statistical techniques in a systematic review to integrate the results of included studies. Cochrane Collaboration (2005) Glossary of Terms in The Cochrane Collaboration Meta-analysis

9. Meta-analysis

10. Systematic reviews involve: • stating the objectives of the research • defining eligibility criteria for studies to be included • identifying (all) potentially eligible studies • applying eligibility criteria • assembling the most complete dataset feasible • analysing this dataset, using statistical synthesis and sensitivity analyses, if appropriate and possible • preparing a structured report of the research. Systematic reviews

11. Reduce bias • Reduce random error • Explore variability • Provide reliable basis for making decisions • Inform and influence future research Systematic reviews

12. Traditional lack rigor methodology not transparent different reviewers reach different conclusions become out of date Systematic scientific rigor to minimise bias explicit and reproducible methodology regularly updated (Cochrane) Traditional vs systematic reviews

13. “It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials”. Archie Cochrane, 1979 The Cochrane Collaboration

14. The Prototype - Effective Care in Pregnancy and Childbirth

15. The Cochrane Collaboration is an international organization that aims to help people make well-informed decisions about healthcare by preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of healthcare interventions. The Cochrane Collaboration

16. Structure of The Cochrane Collaboration Centres Review Groups Steering Group Consumer Network Fields Methods Groups

17. Cochrane Centres • Promote awareness and support of The Cochrane Collaboration on a geographical basis • support individual review authors • support review groups in that area • provide a unique function e.g. software development, consumer support • link to government and other agencies • not a production house for reviews

18. Canadian Cochrane Centre staff

19. Cochrane Review Groups • International and multidisciplinary • Produce reviews • Focused around health problems • 51, covering most of health care • Editorial base facilitates process

20. Worldwide there are over 11,000 review authors working with 51 registered review groups • 5 review groups have their editorial base in Canada • Back Group • Effective Practice and Organisation of Care Group (EPOC) • Hypertension Group • Inflammatory Bowel Disease and Functional Bowel Disorders Group • Musculoskeletal Group Cochrane Review Groups

21. Cochrane Fields • Represents a population, group, or type of care that overlaps multiple review group areas • E.g., primary care, health promotion and public health, health care of older people • Child Health Field and the Health Equity Field are located in Canada • Ensures that their priorities are reflected in the work of review groups

22. Cochrane Methods Groups • Develops methodology and advise the Collaboration on how the validity and precision of systematic reviews can be improved • Examples: Statistical Methods, Non-Randomized Studies, Information Retrieval • Reporting Bias Methods Group is located in Canada

23. Facilitates the dissemination of information to patients, their families, friends and advocates throughout the world • Plays an integral part to all activities within The Cochrane Collaboration • Supports the role of consumers within The Cochrane Collaboration • Liz Whamond (NB) current chair of The Cochrane Consumer Network Cochrane Consumer Network

24. Cochrane reviews are conducted using standardised rigorous approaches including detailed critical appraisal of all studies • Cochrane reviews are reported using standardised structure including abstract and consumer summary Cochrane reviews

25. STEP 1: formulate the problem and register the title with CRG STEP 2: write protocol, submit for peer review and publish STEP 3: locate and select studies STEP 4: critically appraise studies for risk of bias STEP 5: collect data STEP 6: analyse and present results STEP 7: interpret results and write review STEP 8: improve and update review Steps of a Cochrane systematic review

26. Decide which Cochrane CRG would be responsible • relevant CRG may be obvious • most groups are disease (or organ) focused • if unsure which CRG, contact CCN/C for advice • Information on CRGs • The Cochrane Library • www.cochrane.org • Contact CRG directly • Complete title registration form (from CRG) Title registration

27. Titles for Cochrane reviews have a set format • Format‘intervention’ for ‘problem’ in ‘category’ • ‘Caffeine’ for ‘daytime sluggishness’ in ‘adults’(hypothetical for today) • Keep it short Tips for titles

28. Systematic reviews are scientific research • Plan methods • Reduce bias • Access to peer review • Avoid duplication of effort • Rest of review follows • focus for thinking about review • planning and allocating tasks Why have a protocol?

29. Standard format • to help review authors be systematic • to help people reading the reviews to find information quickly Format of a Cochrane protocol

30. Title/review authors • Background • Objectives • Selection criteria • Search strategy • Methods Format of a Cochrane protocol

31. Background, objectives and selection criteria

32. Description of the condition and its significance • Description of the intervention and its role in practice • How the intervention might work • Why it is important to do the review Background (Handbook Section 3)

33. Follow naturally from the background • What are the questions? • does it work? • is the effect consistent or variable? • Questions should be clearly framed Objectives (Handbook Section 3)

34. Systematic review question can be framed broadly or narrowly: • Effects of audit and feedback to improve chronic diseases across all health care settings and professionals • Effects of audit and feedback to improve chronic diseases within primary care • Effects of audit and feedback to improve diabetes care within primary care Lumping and splitting

35. The ‘lumping’ rationale - systematic reviews aim to identify the common generalisable features within similar interventions, minor differences in trial design are not important. • ‘Lumped’ reviews allow generalisability and consistency of findings to be assessed across wide range of settings and populations – this reduces risk of bias or chance results • The ‘lumping principle’ is that the results of two interventions should be combined unless there are good grounds to believe they will have opposing effects. Lumping and splitting

36. The ‘splitting’ rationale is that it is only appropriate to combine trials which are very similar in design, patient selection, intervention characteristics and outcome recording. • Split reviews avoid combining ‘apples and oranges’ • Reviews can be split by participants, interventions or outcome • Very narrowly focused reviews are de facto subgroup analyses Lumping and splitting

37. Practical considerations • Lumped reviews • Logistically challenging (large number of included studies) • Analytically challenging • Heterogeneity expected • Interpretation maybe challenging – seeing the woods for the trees • Split reviews • Easier, quicker, cleaner Lumping and splitting

38. Effects of audit and feedback (85 studies) • Effects of audit and feedback to improve chronic diseases across all health care settings and professionals (18 studies) • Effects of audit and feedback to improve chronic diseases within primary care (14 studies) • Effects of audit and feedback to improve diabetes care within primary care (3 studies) Lumping and splitting

39. Follow naturally from the objectives • type of participants • type of interventions • type of outcomes • type of studies Selection criteria (Handbook Section 4)

40. Help you design the search • Selecting studies • Start thinking about the analysis • One way to minimise bias Purpose of selection criteria

41. Small group exercises

42. Writing the methods and completing the protocol (Handbook sections 6,7,8)

43. Plan what you will do before you start • Divide work among review authors and establish timeline • Minimise bias • Start thinking about the analysis Purpose of methods (Handbook sections 6,7,8)

44. Inclusion of trials - how? - by who? Methods (Handbook sections 6,7,8)

45. Assessment of risk of bias - type of assessment - by who? - how to incorporate in the review? Methods(Handbook sections 6,7,8)

46. Data extraction - what will be extracted? - by who? - how to deal with missing data? Methods(Handbook sections 6,7,8)

47. Data synthesis - what type of synthesis to do? - what summary measures to use? - what comparisons to make? • plans to investigate for differences in study results (heterogeneity) or potential for missing studies (publication bias) • plans for subgroup or sensitivity analyses? Methods(Handbook sections 6,7,8)

48. Make sure all sections are completed - title/review authors, background, objectives, selection criteria, search strategy, methods • Don’t forget the other bits and pieces - contact details of review authors, acknowledgements, conflict of interest, sources of support, references, additional tables Writing your protocol

49. The Cochrane Handbook • Open learning support • Revman • Support from CRGs: • Trial search co-ordinators • Review group co-ordinators • Contact editor • Support from CCN/C • Training • Whatever else we can do! Resources for Cochrane reviewers

50. Submit to CRG for editorial process • Internal and external peer review 1-2 months • May receive critical comments to respond to • Protocol published on The Cochrane Library • Reward yourself then start the review • Note - if you need to change your plan – report it and justify it (be transparent) After protocol is complete