Reviewing the Literature

1. Reviewing the Literature Prof. Jimmy Volmink PHC Directorate Faculty of Health Science, UCT

2. What is a literature review? • Account of what is written on a particular subject by scholars and researchers • Purpose: convey current knowledge and ideas as well as their strength and weaknesses • May be: • part of essay, research report or thesis • stand alone exercise

3. Annotated bibliography vs. literature review • AB: List of sources each accompanied by a brief note (annotation) discussing one or more aspects e.g. content, usefulness, quality, etc • LR: Discursive writing – presents themes and concepts; integrates ideas and concepts

4. Objectives of literature review • Address a research question or thesis • Critically appraise research for validity • Synthesize and summarise current knowledge • Identify controversies in literature • Formulate questions for future research

5. Information explosion • Journals • Textbooks • Unpublished literature • World Wide Web

6. Reviews… reduce the load!!!

7. Reviews: what do we want ? • A readable summary of ALL the evidence • Unbiased • Transparent • Up-to-date

8. But is this what we get?

9. Corticosteroids for prevention of respiratory distress syndrome • Expert review of the literature (Robertson, BMJ 1982) • “The evidence suggests that antenatal steroids are of value only in white males, and even for them the benefit is mainly in those of 30-32 weeks’ gestation…”

10. Corticosteroids for prevention of respiratory distress syndrome • Another view (Crowley, 1989) • “The benefits of antenatal corticosteroids have been established. No further trials are necessary with the exception of certain specific situations (such as pre-eclampsia) or to establish other dosages or routes of administration.”

11. Treatments for Acute MI • Existing evidence compared with recommendations in textbooks and journals • “advice on some life-saving therapies has been delayed for more than a decade, while other treatments have been recommended long after controlled research has shown them to be harmful.”Antman et al. JAMA 1992; 268: 240-8

12. Traditional review • Qualitative, narrative summaries of evidence on a given topic usually written by an expert in the field. • They typically involve informal and subjective methods to collect and interpret information

13. Problems with traditional reviews 1. Bias in selection of included studies • Personal (“file drawer”) bias • Publication bias: • Studies with statistically significant results are: • More likely to be published • More likely to be published in English • More likely to be published in journals with high citation impact factorsEasterbrooke et al, Lancet 1991

14. Problems with traditional reviews 2. Indexing bias Ongoing All RCTs Unpublished Completed Abstracts, theses, etc Published Non-MEDLINE journals (c. 12000) 1966+ Journals RCTs notidentified byMEDLINE MEDLINE journals (c. 4600) 1966+ RCTsidentified byMEDLINE

15. Problems with traditional reviews 3. Variable quality of published studies • Fourth Congress on Peer Review in Biomedical Publication : “Medical journals are full of serious methodological errors “ “Journal editors are giving no time, energy and thought to their craft” “Studies are published that reach false conclusions” BMJ, Sept. 2001:323

16. Problems with traditional reviews 4. Effect ofStudy Design PURPOSE: To assess the evidence of the effectiveness of vitamin supplementation, specifically vitamins A, C, and E; beta-carotene; folic acid; antioxidant combinations; and multivitamin supplements, in preventing cardiovascular disease. STUDY SELECTION: The researchers selected English-language reports of randomized trials and cohort studies that assessed vitamin supplementation in western populations and reported incidence of or death from cardiovascular events. CONCLUSIONS: Some good-quality cohort studies have reported an association between the use of vitamin supplements and lower risk for cardiovascular disease. Randomized, controlled trials of specific supplements, however, have failed to demonstrate a consistent or significant effect of any single vitamin or combination of vitamins on incidence of or death from cardiovascular disease. Ann Intern Med. 2003;139:56-70.

17. Problems with traditional reviews 5. Effect ofStudy Size “ … we still have no clear evidence that beta-blockers improve long-term survival after infarction despite almost 20 years of clinical trials.” JRA Mitchell. BMJ 1981;282:1565-70

18. The Science of Research Synthesis “If science consisted of nothing but the laborious accumulation of facts, it would soon come to a standstill...… Two processes are at work side by side, the reception of new material and the digestion and assimilation of the old; and both are essential…” Lord Rayleigh 1884

19. A Systematic ReviewHow is it different? “A review in which bias has been reduced by the: • systematic identification, • appraisal, • synthesis • statistical aggregation (if relevant) of all relevant studies on a specific topic according to a predetermined and explicit method.” Moher et al. Lancet; 354: 1896-900

20. Steps in conducting a systematic review • Formulate topic/question • Decide upon selection criteria for studies • Determine search strategy • Critically appraise research • Extract data • Synthesise and summarise data

21. 1. The review question • Do betablockers compared to placebo decrease morbidity and mortality in people with hypertension? • Does the administration of Bacillus Calmette-Guerin (BCG) influence the risk of tuberculosis and all-cause mortality in populations exposed to infection? • PICO Method

22. 2. Selection criteria for studies • Types of participants • People receiving BCG vaccination, irrespective of age • Types of interventions • BCG vaccination given by any route. • Types of outcome measures • Death • Tuberculosis • Types of studies • RCT

23. 3. Search strategy • Scope of search • Published vs. unpublished vs. grey literature • Peer-reviewed vs. non peer-reviewed • Electronic vs. paper based sources • Search should be as comprehensive as possible

24. Ingredients of a comprehensive literature search • Electronic databases (highly sensitive strategy) • MEDLINE/PUBMED • EMBASE • Cochrane Databases – CENTRAL • Other e.g. CINAHL, AIDSLINE, LILACS, etc. • Handsearching of journals • Reference lists • Conference proceedings • Personal communication

25. 4. Critical appraisal of research • Different types of clinical research • Need to understand strengths and limitations of various study designs • Know what aspects of a study to assess to determine likelihood of bias

26. 5. Extract data • Pre-specify items of information to be collected • Data extraction form • Independent data extraction

27. 6. Synthesise and summarise data • Qualitative (narrative) • Quantitative synthesis (uses statistics) • Some statistical knowledge will be needed

28. Meta-analysisPooling the results • A statistical procedure that pools the results of several independent studies considered sufficiently similar (homogeneous) • Not appropriate if studies are too dissimilar (heterogeneous) • Provides a quantitative summary of the overall treatment effect • Visual display called a “forest plot”

29. Meta-analysis and the Forest Plot • Estimate of effect • Confidence intervals • Weighting • Overall estimate of effect

30. Cohort studies Male health workers USA Social insurance, men Finland Finland Social insurance, women Male chemical workers Switzerland Hyperlipidaemic men USA USA Nursing home residents Cohorts combined Trials Male smokers Finland Skin cancer patients USA (Ex)-smokers, asbestos workers USA Male physicians USA Trials combined 0.1 0.5 0.75 1 1.25 1.5 1.75 Relative risk (95% CI) Meta-analysisBeta-carotene and cardiovascular mortality

31. Acknowledging your sources • Why bother: • Honesty and transparency • Show research done to reach conclusions • Allow readers to identify and retrieve references • Plagiarism – “taking, using and passing off as your own work the ideas or words of another” • See UCT policy

32. Types of Citation Two main types: • Harvard: author-date style • …as one author has put it “the darkest days were still ahead” (Weston 1988, p.45) • Vancouver: footnote/endnote style set by the International Committee of Medical Editors • …as one author has put it “the darkest days were still ahead”[1] • [ ] , ( ) or supercript …ahead”1 all acceptable

33. In summary… Traditional vs. systematic review

34. Traditional review • Qualitative, narrative summaries of evidence on a given topic usually written by an expert in the field. Typically, involve informal and subjective methods to collect and interpret information

35. Systematic review “A review in which bias has been reduced by the systematic identification, appraisal, synthesis, and, if relevant statistical aggregation of all relevant studies on a specific topic according to a predetermined and explicit method.” Moher et al. Lancet 1999; 354: 1896-900

36. Cochrane review • A systematic review produced by the Cochrane Collaboration (www.cochranelibrary.com) • Standardised format • Extensive peer review • Published electronically on the Cochrane Library (and indexed in Medline) • Invites comments and criticism • Kept up-to-date • Quality and reporting on average better than other systematic reviews ….but not without flaws!

37. Meta-analyses All reviews Systematic reviews Summary: Types of reviews