How to Read Systematic Review : An Approach for the Clinicians (part- 1 )

1. How to Read Systematic Review : An Approach for the Clinicians(part-1) Akbar Soltani. MD, Tehran University of Medical Sciences (TUMS) Shariati Hospital www.soltaniebm.com

2. Systematic Reviews:Objectives: Definition of systematic review and meta-analysis? Principles of methodology of systematic review Discuss differences between systematic review and narrative review Consider clinical implications of a Systematic review

3. Scenario • At the end of a long week in the office, you sink back into your chair, reflecting on some of the more memorable patients you cared for and counseled. • Through gentle history taking, you discovered that urinary incontinence is the underlying cause of an elderly patient's increasing social isolation.

4. Scenario(cont’d) • You discontinued procainamide therapy in a 72-year-old man who had asymptomatic PVC after MI. • To prevent bleeding from esophageal varices, you started ß-blocker therapy in a woman with long-standing cryptogenic cirrhosis and portal hypertension. • You presented the risk factors for major and minor bleeding to a 39-year-old woman who was considering warfarin therapy because of recently diagnosed AF and valvular heart disease.

5. Scenario(cont’d) • You stumbled while debating the pros and cons of breast cancer screening with a healthy 48-year-old woman . • You questioned the merits of a personalized walking program suggested to you by a motivated 66-year-old man with severe claudication. • Explaining that you wanted to review the best current evidence on these issues, you resolved to address your uncertainties before these patients made their next office visits, in a week's time.

6. Scenario(cont’d) • Sighing deeply, you acknowledge that you have little time to read. • You subscribe to three journals, which you browse months after they arrived either when your journal pile becomes severely high or when your guilt is sufficiently motivational. • You sometimes find the conclusions of individual articles conflicting or confusing.

7. Scenario(cont’d) • You know that some of the decisions and suggestions you made this week, specifically your decisions about stopping procainamide therapy and starting ß-blocker therapy and your advice about bleeding risks from anticoagulant therapy, were based on the best current research evidence .

8. Scenario(cont’d) • On the other hand, your patients' inquiries about breast cancer screening and exercise treatment for claudication highlight your need for a concise, current, rigorous synthesis of the best available evidence on each of these topics: in brief, a systematic review.

9. Definition: Science … is the organized systematic activity that gathers knowledge about the world and condenses the knowledge into testablelaws and principles.(Edward O.Wilson; American Scientist, Jan 1998, pg.6.)

10. Medical literature Primary (analytic) studies Secondary (integrative) studies Systematic review Nonsystematic review Meta-analysis Practice guideline Decision analysis Economic analysis Editorial, commentary Experimental RCT Nonrandomized Controlled trial Observational cohort case-control cross sectional descriptive, surveys case reports

11. Medical literature Primary (analytic) studies Secondary (integrative) studies Systematic review Nonsystematic review Meta-analysis Practice guideline Decision analysis Economic analysis Editorial, commentary Experimental RCT Nonrandomized Controlled trial Observational cohort case-control cross sectional descriptive, surveys case reports

12. Definitions • Authors sometimes use the terms overview, systematic review, and meta-analysis interchangeably. • systematic review :any summary that attempts to address a focused clinical question using methods designed to reduce the likelihood of bias • meta-analysis: describes reviews that use quantitative methods to summarize the results.

13. Summarizing the evidence is needed whenever there is: • Substantial uncertainty: contradicting results of studies or effects which vary too much among different types of subjects

14. In a small randomized double-blind trial of a new treatment for acute myocardial infarction, the mortality in the treated group was half that in the control group, but the difference was not significant. We can conclude that: The treatment is useless there is no point in continuing to develop the treatment the reduction in mortality is so great that we should introduce the treatment immediately we should keep adding cases to the trial until the Normal test for comparison of two proportions is significant we should carry out a new trial of much greater size Paul Glasziou- University of Oxford

15. Which is the smallest study? Which is the largest study? How many are statistically significant? Which studies are “large enough”? The 17 studies : Forest Plot/Blobbogram Paul Glasziou- University of Oxford

16. The 17 studies: of streptokinase for MI Paul Glasziou- University of Oxford

17. Systematic review Or Overview Comprehensively • locates • evaluates • synthesizes all (?) the available literature on a given topic using a strict scientific design which must itself be reported in the review

18. A ‘systematic review’, therefore, aims to be: • Systematic (e.g. in its identification of literature) • Explicit (e.g. in its statement of objectives, materials and methods) • Reproducible (e.g. in its methodology and conclusions

19. Stages of a systematic review • Planning the review– i.e. identifying the need for a review, and documenting the methodology • Conducting the review – i.e. finding, selecting, appraising, extracting and synthesizing primary research studies • Reporting and dissemination – i.e. writing up and disseminating the results of the review

20. The Process of Conducting a Systematic Review Comprehensive and pre-defined data search Unbiased pre-defined selection and extraction process Critical appraisal of data Synthesisof data Well formulated question

21. Question components : PICO • What types of Participants? • What types of Interventions? • What types of Comparison? • What types of Outcomes?

22. Well formulated question

23. Comprehensive and pre-defined data search • study design (e.g., RCT’s?, DBPC?, Cohort & CCS?) • setting (emergency department, outpatient, inpatient) • age (adults only, > 60 only, etc) • year of publication or conduct (esp. if technology or typical dosing changes) • similarity of exposure or treatment (e.g., drug class, or dosage) • similarity of outcomes (case definitions) • minimum sample size or follow-up • languages? • pre-1966?

24. Comprehensive and pre-defined data search

25. Is finding all published studies enough? • Negative studies less likely to be published than ‘Positive’ • How does this happen? • Follow-up of 737 studies at Johns Hopkins (Dickersin, JAMA, 1992) • Positive SUBMITTED more than negative (2.5 times)

26. Bias in Systematic Reviews • Publication bias is the selective publication of manuscripts based on the magnitude, direction, or statistical significance of the study results.

27. Data sources for a systematic review Electronic databases MEDLINE and EMBASE The Cochrane Central Register of Controlled Trials (CENTRAL) Hand searching “Grey literature” ( thesis, Internal reports, pharmaceutical industry files) Checking reference lists Unpublished sources known to experts in the specialty seek by personal communication) Raw data (from published trials)

28. Which are biased? Which OK? • All studies published in BMJ, Lancet, JAMA or NEJM • All publicly funded studies • All studies with more than 100 patients • All studies conducted in the Northern Hemisphere • All studies registered studies

29. Registered vs Published StudiesOvarian Cancer chemotherapy: single v combined Simes, J. Clin Oncol, 86, p1529

30. Unbiased pre-defined selection and extraction process

31. Critical appraisal of data

32. Critical appraisal of data 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete?

33. Critical appraisal of data 1. Was there an independent, blind comparison with a reference standard? 2. Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice? 3. Did the results of the test being evaluated influence the decision to perform the reference standard? 4. Were the methods for performing the test described in sufficient detail to permit replication?

34. Synthesis of data • when an overview incorporates a specific statistical strategy for assembling the results of several studies into a single estimate • Systematic reviews do not have to have a meta-analysis • There are times when it is not appropriate or possible.

35. Systematic Review vs Meta-Analysis

36. Fig 2: Improvements up to two weeks after steroid injection in knee

37. Graphical and tabular summary of studies?

38. Synthesis of data

39. Synthesis of data

40. The Process of Conducting a Systematic Review Comprehensive and pre-defined data search Unbiased pre-defined selection and extraction process Critical appraisal of data Synthesis of data Well formulated question

41. Feature Narrative Review-Textbook Systematic Review Question often broad in scope a focused clinical question Not usually specified potentially biased Sources and search Comprehensive sources and explicit search strategy Criterion-based selection uniformly applied Not usually specified potentially biased Selection Rigorous critical appraisal Appraisal Variable Often a qualitative summary Quantitative summary Synthesis Sometimes evidence-based Usually evidence-based Inferences

42. Pros and cons of systematic reviews Advantages Larger numbers & power Robustness across PICOs Disadvantages May conclude small biases are real effects

43. Where to find systematic reviews • The Cochrane Library • DARE (Database of Abstracts of Reviews of Effectiveness) • The NHS Health Technology Assessment (HTA) Database

44.  The Cochrane Collaboration Preparing, maintaining and disseminating systematic reviews of the effects of health care

45. Thank You !