Conducting systematic reviews of public health and health promotion interventions

1. Conducting systematic reviews of public health and health promotion interventions Nicki Jackson Senior Training and Support Officer Cochrane Health Promotion and Public Health Field

2. Overview Background to systematic reviews International systematic review initiatives Resources required Setting the scope of your review Asking an answerable question Searching for studies Data abstraction Principles of critical appraisal Synthesis of evidence Interpretation of results Writing the systematic review

3. Objectives This workshop will enable you to: • Be familiar with some of the key challenges of conducting systematic reviews of health promotion and public health interventions • Formulate an answerable question about the effectiveness of interventions • Identify primary studies, including developing strategies for searching electronic databases

4. Objectives cont. • Evaluate the quality of an individual health promotion or public health study • Synthesise the evidence from primary studies • Formulate conclusions and recommendations from the body of evidence • Evaluate the quality of a systematic review

5. Acknowledgement The Public Health Education and Research Program (PHERP) “Promoting and facilitating evidence-based policy and practice in Public Health and Health Promotion” • Sydney Health Projects Group, School of Public Health, University of Sydney • School of Public Health, La Trobe University • Cochrane Health Promotion and Public Health Field

6. Background to systematic reviews

7. Types of reviews Reviews (narrative/literature/ traditional) Systematic reviews Meta-analysis

8. Narrative reviews • Usually written by experts in the field • Use informal and subjective methods to collect and interpret information • Usually narrative summaries of the evidence Read: Klassen et al. Guides for Reading and Interpreting Systematic Reviews. Arch Pediatr Adolesc Med 1998;152:700-704.

9. What is a systematic review? • A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review* *Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

10. High quality Structured, systematic process involving several steps : • Plan the review • Formulate the question • Comprehensive search • Unbiased selection and abstraction process • Critical appraisal of data • Synthesis of data (may include meta-analysis) • Interpretation of results All steps described explicitly in the review

11. Scientific approach to a review article Criteria determined at outset Comprehensive search for relevant articles Explicit methods of appraisal and synthesis Meta-analysis may be used to combine data Depend on authors’ inclination (bias) Author gets to pick any criteria Search any databases Methods not usually specified Vote count or narrative summary Can’t replicate review Systematic vs. Narrative reviews

12. Advantages of systematic reviews • Reduce bias • Replicability • Resolve controversy between conflicting studies • Identify gaps in current research • Provide reliable basis for decision making

13. Increased interest in systematic reviews • Government interest in health costs • Variations in practice • Public want information • Facilitated by computer developments

14. Evidence Experience Opinions Competing factors and pressures Expectations Financial pressures Time pressures

15. Who benefits? Practitioners - current knowledge to assist with decision making Researchers - reduced duplication - identify research gaps Community - recipients of evidence-based interventions Funders - identify research gaps/priorities Policy makers - current knowledge to assist with policy formulation

16. Limitations • Results may still be inconclusive • There may be no trials/evidence • The trials may be of poor quality • The intervention may be too complex to be tested by a trial • Practice does not change just because you have the evidence of effect/effectiveness

17. Clinical Individuals Single interventions Outcomes only (generally) Often limited consumer input Quantitative approaches to research and evaluation Public health Populations and communities Combinations of strategies Processes as well as outcomes Involve community members in design and evaluation Qualitative and quantitative Health promotion theories and beliefs Clinical vs. public health interventions

18. International systematic review initiatives

19. Sources of systematic reviews in HP/PH • Cochrane Collaboration • Guide to Community Preventive Services (The Guide), US • The Effective Public Health Practice Project, Canada • Health Development Agency, UK • The Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre), UK • Centre for Reviews and Dissemination, UK • The Campbell Collaboration

20. Cochrane Collaboration • Named in honour of Archie Cochrane, a British researcher In 1979: “It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials”

21. Cochrane Collaboration International non-profit organisation that prepares, maintains, and disseminates systematic up-to-date reviews of health care interventions

22. The Cochrane Librarywww.thecochranelibrary.com

23. The Cochrane Library • Cochrane Systematic reviews : Cochrane reviews and protocols • Database of Reviews of Effects: Other systematic reviews appraised by the Centre for Reviews and Dissemination. • Cochrane Central Register of Controlled Trials: Bibliography of controlled trials (some not indexed in MEDLINE). • Cochrane database of Methodology Reviews: Cochrane reviews of methodological studies. • The Cochrane Methodology register: Bibliography of studies relating to methodological aspects of research synthesis • About the Cochrane Collaboration: Information about review groups, Fields, Centres, etc. Contact details provided. • Health Technology Assessment Database: HTA reports • NHS Economic evaluation database: Economic evaluations of health care interventions.

25. Organisation Review groups Steering group Fields Centres Methods groups Consumer network

26. Collaborative Review Groups • Focused around health problems (50) • Produce reviews • Editorial base facilitates review process • International and multidisciplinary eg. Airways Group Drugs and Alcohol Group Heart Group Injuries Group Skin Group Breast Cancer Group

27. Cochrane Centres • Support review groups and reviewers within area (13) • Promote Cochrane Collaboration • Link to Government and other agencies • Nota production house for reviews eg. Australasian Cochrane Centre South African Cochrane Centre Italian Cochrane Centre

28. Cochrane Health Promotion and Public Health Field • Registered in 1996. Administered from Melbourne. Funded by VicHealth (Co-directors at EPPI-Centre, UK) • 330 members on contact database across 33 countries Aims: • Promoting the conduct of reviews on HP/PH topics • Educating HP/PH practitioners about the Cochrane Collaboration, encouraging use of systematic reviews • Referring professionals to other databases

29. Reviews in HP/PH • Primary prevention of alcohol misuse in young people • Parent-training programmes for improving maternal psychosocial health • Interventions for preventing childhood obesity • Interventions for preventing eating disorders in children and adolescents • Supported housing for people with severe mental disabilities

30. For further information The Cochrane Collaboration http://www.cochrane.org The Cochrane Health Promotion and Public Health Field http://www.vichealth.vic.gov.au/cochrane/ The Australasian Cochrane Centre http://www.cochrane.org.au The Cochrane Library http://www.thecochranelibrary.com

31. Other sources The Guide to Community Preventive Services http://www.thecommunityguide.org/

32. Other sources Effective Public Health Practice Project (EPHPP) http://www.city.hamilton.on.ca/PHCS/EPHPP/default.asp

33. Other sources Health Development Agency http://www.hda-online.org.uk/

34. Other sources Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) http://eppi.ioe.ac.uk

35. Other sources Centre for Reviews and Dissemination http://www.york.ac.uk/inst/crd

36. Other sources The Campbell Collaboration http://www.campbellcollaboration.org/

37. Resources required

38. Conduct of systematic reviews • Topic of relevance or interest • Team of co-authors • Training and support • Access to/understanding of stakeholders or likely users • Funding and time (at least 6 months) • Access to databases of published and unpublished literature • Statistical software, if appropriate • Bibliographic software

39. Review manuals • Cochrane Collaboration Reviewers’ Handbook • Cochrane Collaboration Open Learning Materials • NHS Centre for Reviews and Dissemination Guidance for those Carrying Out or Commissioning Reviews • The Methods of the Community Guide • A Schema for Evaluating Evidence on Public Health Interventions • EPPI-Centre Reviewers’ Manual

40. Guidelines for HP/PH reviews Cochrane Health Promotion and Public Health Field website http://www.vichealth.vic.gov.au/cochrane/activities/guidelines.htm

41. Setting the scope of your review

42. Advisory Groups • Policy makers, funders, practitioners, recipients/consumers • Make or refine the review question • Provide background material • Help interpret the findings • Assist with dissemination • Formal (role descriptions) or informal

43. LUMP OR SPLIT?

44. Lump or split? • Users needs • Policy – broad reviews to answer questions when there is a range of options • Practitioners – more-specific interventions or approaches • Lump – inform which interventions to implement, more time-consuming • Split – Yes/No to implement, less time

45. Lumping or splitting • Interventions to modify drug-related behaviours for preventing HIV infection in drug users • Interventions to modify sexual risk behaviours for preventing HIV infection • Interventions to modify sexual risk behaviours for preventing HIV infection in men who have sex with men. • Interventions for preventing HIV infection in street youth • Interventions for preventing HIV infection in young people in developing countries • Counselling and testing for preventing HIV infection

46. Writing your protocol 1) Background • Why is it important? • How important is the problem? • Is there uncertainty? • What is the reasoning as to why the intervention(s) might work? (include theoretical frameworks) • Other similar reviews?

47. Writing your protocol 2) Objectives • What are the questions/hypotheses? 3) Selection criteria • PICO(T) • Population(s) • Intervention(s) • Comparison(s) • Outcomes (Primary / Secondary) • Types of studies

48. Writing your protocol 4) Planned search strategy • Databases and terms 5) Planned data extraction • Processes and outcomes? • More than one reviewer? • Planned quality appraisal (incl. checklists) 6) Method of synthesis • Tabulate • Narrative/qualitative synthesis or meta-analysis

49. Asking an answerable question

50. Importance A clearly framed question will guide: • the reader • in their initial assessment of relevance • the reviewer • on how to collect studies • on how to check whether studies are eligible • on how to conduct the analysis