Presentation of Findings

1. Presentation of Findings Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov

2. Systematic Review Process Overview

3. Learning Objectives • To understand the goals of presenting systematic review data in tables that summarize data across studies • To become familiar with approaches to graphical presentations of findings not synthesized through statistical or meta-analytic techniques

4. Organizing and Reporting Findings • Once the evidence tables are completed, there is an enormous (sometimes overwhelming) amount of data yet to synthesize. • Various types of heterogeneity may preclude a quantitative synthesis (e.g., meta-analysis). • Synthesizing heterogeneous studies by using tables (nonquantitative synthesis) can capture underlying similarities to support conclusions. • Making sense of the data requires good tabular presentation, in addition to clear organization and writing of the text.

5. Why Not Meta-analysis? • Even studies meeting the same inclusion criteria can vary. • Clinical heterogeneity — variation in the study population, interventions, and outcomes • Methodological heterogeneity — variation in study design • Statistical heterogeneity — variation in observed treatment effect (for trials) • Many factors can contribute to variation in seemingly similar studies. • Some examples are evolving diagnostic criteria, evolving diseases, differences in baseline characteristics, and differences in care. • Not all studies can (or should) be combined statistically. • Various statistical models are useful for identifying the level of heterogeneity between studies. AHRQ . Methods reference guide for effectiveness and comparative effectiveness reviews. Version 1.0. Available at: http://effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf.

6. Solution • Nonquantitative synthesis using tables that summarize data across studies • Use of evidence maps to provide an overview of the data

7. PICOTS • Population: primary populations of interest are followed by subpopulations • Intervention: hierarchy of interventions may reflect most common to least common • Comparator: where multiple comparisons are made, one might present the more “usual” decision first • Outcome: a hierarchy of outcomes may reflect higher to lower acuity or target outcomes followed by collateral ones; harms are generally presented last • Timing: short term generally precedes long term • Setting: an intervention may be available in various settings (e.g., inpatient or outpatient)

8. Combining Studies • Evidence tables are the first step to summarization, but each evidence table represents the data in only one study. • Incorporating multiple studies into a single table allows entire subsets of the literature to be summarized and compared (e.g., by key question or study design). • Summary tables and evidence maps are two approaches with which information about individual studies and results are combined. • Authors should use caution when deciding which evidence to summarize so as not to introduce a biased perspective.

9. Summary Tables (I) • Combine data from multiple studies to illustrate trends in the data • May be focused on describing study characteristics, results, or both • Can be designed to include characteristics of all included studies • Examples: funding sources, assessment method, country of study • Can be designed for subsets of included studies • Examples: summary tables for randomized controlled trials, prevalence studies, harms/side effects, outcomes for specific treatments

10. Summary Tables (II) • Simplified entry (one row) for each study • Table columns may include, for example: • PICOTS (may be listed in table title or headers) • Methodological quality • Applicability • Study size (weight) • Magnitude of effect • A single study may be represented in multiple summary tables (e.g., different outcomes) PICOTS = population, intervention, comparator, outcomes, timing, and setting

11. Example:Summary Table of Study Characteristics • A basic summary table is the “study characteristics” table. • The overall summary provides an overview of the state of the available studies in the literature. Hartmann KE, et al. AHRQ Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/ bladder/bladder.pdf.

12. Example: Summary Table as High-Level Map Chung M, et al. AHRQ Evidence Report/Technology Assessment No. 183. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/vitadcal/vitadcal.pdf.

13. Example: Summary Table of Study Characteristics Balk EM, et al. Ann Intern Med 2003;139:670-82. Reprinted with permission from the American College of Physicians.

14. Visual Presentation of Available Comparisons Ip S, et al. AHRQ Evidence Report/Technology Assessment No. 124. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/rhinoupdate/rhinoup.pdf.

15. Example:Summary Table of Prevalence Findings Summary tables can be specialized for different types of questions. Hartmann KE, et al. AHRQ Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/ downloads/pub/evidence/ pdf/ bladder/bladder.pdf.

16. Example:Summary Table for Randomized Controlled Trials Summary tables can be specialized for different subgroups of the literature. Hartmann KE, et al. AHRQ Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/ downloads/pub/evidence/pdf/bladder/bladder.pdf.

17. Example: Summary Table for Cohort Studies Wang C, et al. AHRQ Evidence Report/Technology Assessment No. 94. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/o3cardio/o3cardio.pdf.

18. Example: Presentation of Harms Data Summary tables can be specialized for different subgroups of the literature. Hartmann KE, et al. AHRQ Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/ downloads/pub/evidence/ pdf/ bladder/bladder.pdf.

19. Example:Presentation of Data by Outcome and Age Chung M, et al. AHRQ Evidence Report/Technology Assessment No. 183. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/vitadcal/vitadcal.pdf.

20. Example:Presentation of Patient-Reported Outcomes Summary tables can be specialized for different types of outcomes. Hartmann KE, et al. AHRQ Evidence Report/Technology Assessment No. 187. Available at: http://www.ahrq.gov/ downloads/pub/evidence/ pdf/ bladder/bladder.pdf.

21. Evidence Maps • Are a succinct graphical presentation of available studies to address key questions along variables of interest • Results of outcomes typically not included • Can help to identify comparisons with sufficient evidence for analysis • Can help to guide readers in knowing what comparisons and outcomes are available in the report • Can help to identify evidence gaps

22. Example: Evidence Map Terasawa T, et al. Ann Intern Med 2009;151:556-65. Reprinted with permission from the American College of Physicians.

23. A Second Approach to an Evidence Map Dt: time (to stable dose) INR: international normalized ratio mean dose: mean maintenance dose Raman G, et al. AHRQ Technology Assessment No. GEND1206. Available at: http://www.cms.gov/determinationprocess/downloads/id61TA.pdf.

24. Forest Plot Without Summary Favors Wide-Area Circumferential Ablation Favors Pulmonary Vein Isolation Ip S, et al. AHRQ Comparative Effectiveness Review No. 15. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/51/114/2009_0623RadiofrequencyFinal.pdf.

25. Summary Matrix Wang C, et al. AHRQ Evidence Report/Technology Assessment No. 94. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/o3cardio/o3cardio.pdf.

26. Key Messages • Summary tables and evidence maps provide key information on study characteristics and study findings. Through table and graphical formats, respectively • Properly constructed summary tables: • Effectively convey results • Provide an overview of the literature in a given field • Enable the reader to grasp results for subsets of the literature

27. References (I) • Agency for Healthcare Research and Quality. Methods reference guide for effectiveness and comparative effectiveness reviews. Version 1.0. Rockville, MD: Agency for Healthcare Research and Quality, Draft Posted October 2007. Chapter 2, Finding evidence. Available at: http://effectivehealthcare.ahrq.gov/repFiles/ 2007_10DraftMethodsGuide.pdf. • Balk EM, Lau J, Goudas LC, et al. Effects of statins on nonlipid serum markers associated with cardiovascular disease. Ann Intern Med 2003;139:670-82. • Chung M, Balk EM, Brendel M, et al. Vitamin D and Calcium: A Systematic Review of Health Outcomes. Evidence Report/Technology Assessment No. 183. (Prepared by Tufts–New England Medical Center Evidence-based Practice Center under Contract No. HHSA 290-2007-10055-I). Rockville, MD: Agency for Healthcare Research and Quality, August 2009. AHRQ Publication No. 09-E015. Available at: http://www.ahrq.gov/downloads/pub/ evidence/pdf/vitadcal/vitadcal.pdf.

28. References (II) • Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of Overactive Bladder in Women. Evidence Report/Technology Assessment No. 187 (Prepared by Vanderbilt University Evidence-based Practice Center under Contract No. 290-2007-10065-I). Rockville, MD: Agency for Healthcare Research and Quality, August 2009. AHRQ Publication No. 09-E017. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/bladder/ bladder.pdf. • Ip S, Fu L, Balk E, et al. Update on Acute Bacterial Rhinosinusitis. Evidence Report/Technology Assessment No. 124. (Prepared by Tufts–New England Medical Center Evidence-based Practice Center under Contract No. 290-02-0022). Rockville, MD: Agency for Healthcare Research and Quality, June 2005. AHRQ Publication No. 05-E020-2. Available at: http://www.ahrq.gov/downloads/pub/ evidence/pdf/rhinoupdate/rhinoup.pdf .

29. References (III) • Ip S, Terasawa T, Balk EM, et al. Comparative Effectiveness of Radiofrequency Catheter Ablation for Atrial Fibrillation. Comparative Effectiveness Review No. 15 (Prepared by Tufts–New England Medical Center Evidence-based Practice Center under Contract No. 290-02-0022). Rockville, MD: Agency for Healthcare Research and Quality, July 2009. AHRQ Pub. No. 09-EHC015-EF. Available at: http://www.effectivehealthcare.ahrq.gov/ ehc/products/51/114/2009_0623RadiofrequencyFinal.pdf. • Raman G, Trikalinos TA, Zintzaras E, et al. Reviews of Selected Pharmacogenetic Tests for Non-Cancer and Cancer Conditions. Technology Assessment No. GEND1206 (Prepared by Tufts–New England Medical Center Evidence-based Practice Center under Contract No. 290-02-0022). Rockville, MD: Agency for Healthcare Research and Quality, November 2008. Available at: http://www.cms.gov/determinationprocess/downloads/id61TA.pdf.

30. References (IV) • Terasawa T, Dvorak T, Ip S, et al. Systematic review: charged-particle radiation therapy for cancer. Ann Intern Med 2009;151:556-65. • Wang C, Chung M, Balk E, et al. Effects of Omega-3 Fatty Acids on Cardiovascular Disease. Evidence Report/Technology Assessment No. 94 (Prepared by Tufts–New England Medical Center Evidence-based Practice Center under Contract No. 209-02-0022). Rockville, MD: Agency for Healthcare Research and Quality, March 2004. AHRQ Publication No. 04-E009-2. Available at: http://www.ahrq.gov/downloads/pub/evidence/pdf/o3cardio/o3cardio.pdf.

31. Authors • This presentation was prepared by Melissa L. McPheeters, Ph.D., M.P.H., and Jeff Seroogy, B.S., members of the Vanderbilt University Evidence-based Practice Center, and Joseph Lau, M.D., and Thomas Trikalinos, M.D., Ph.D., members of the Tufts Medical Center Evidence-based Practice Center. • The information in this module is currently not included in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/ehc/ products/60/294/2009_0805_principles1.pdf).