1 / 41

Quantitative Synthesis II

Quantitative Synthesis II. Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov. Systematic Review Process Overview. Learning Objectives.

raheem
Download Presentation

Quantitative Synthesis II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quantitative Synthesis II Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov

  2. Systematic Review Process Overview

  3. Learning Objectives • To understand how to explore between-study heterogeneity in a meta-analysis • To understand the pros and cons of subgroup analyses • To understand what meta-regression is and why it is useful • To understand the usefulness of control-rate meta-regression

  4. Statistical Homogeneity Barrington KJ. Cochrane Database Syst Rev 2000;(2):CD000505.

  5. Statistical Heterogeneity Patellar Resurfacing in Total Knee Arthroplasty for Pain RE = random effects model Reprinted from Pakos E, et al. J Bone Joint Surg Am 2005;87:1438-45, with permission from Rockwater, Inc..

  6. What Is Heterogeneity? • In meta-analysis, heterogeneity refers to between-study diversity. • This term is often used to refer to: • differences in study characteristics, and • variability of study results.

  7. Methodological and Clinical Diversity • Methodological diversity pertains to specifics of study design and analysis: • Type of study • Length of followup • Proportion and handling of dropouts • Clinical diversity pertains to differences in: • Populations • Interventions and cointerventions • Outcomes

  8. Statistical Heterogeneity • Statistical heterogeneity exists when the results of individual studies are not “consistent” among themselves. Clinical diversity Methodological diversity Biases Chance Statistical heterogeneity

  9. Clinical and Methodological DiversityVersus Statistical Heterogeneity • Clinical and methodological diversity is abundant: Our aim in a meta-analysis is to explore that diversity and use our observations to formulate interesting hypotheses. • Often clinical and methodological heterogeneity results in a statistically significant test. • Chance, technical issues, or biases may result in statistically significant results in heterogeneity tests.

  10. Exploration of Heterogeneity IsCentral to Evidence Synthesis • One of the most important roles of meta-analytic methodologies is to quantify statistical heterogeneity and to explore whether and to what extent it is explained by clinical and methodological diversity.

  11. Dealing With Heterogeneity (I)

  12. Dealing With Heterogeneity (II) SUBGROUP ANALYSES Differentiating Effects in Subgroups OVERALL ESTIMATE Combining Summary Data RESPONSE SURFACE Modeling Individual Patient Data META-REGRESSION Modeling Summary Data

  13. Promises of Subgroup Analyses • Subgroup analyses can help: • identify modifiers of the treatment effect, • recognize biologically interesting phenomena, or • formulate hypotheses. • We discuss two illustrative examples: • Time-to-thrombolysis affects the treatment effect of thrombolytic drugs in patients with acute myocardial infarction. • Effects of vitamin E supplementation on mortality may differ by vitamin E dose.

  14. Subgroup Analysis: A Meta-analysis of Thrombolytic Therapy for Acute Myocardial Infarction (I) Effects of Thrombolytic Therapy on Mortality in Patients With Acute Myocardial Infarction (Mean Time-to-Treatment) Risk Ratio (95% Confidence Interval)

  15. Subgroup Analysis: A Meta-analysis of Thrombolytic Therapy for Acute Myocardial Infarction (II) Largest Effect in the Subgroup of Trials With Mean Time-to-Treatment of 0 to 3 Hours

  16. Subgroup Analysis: A Meta-analysis of Thrombolytic Therapy for Acute Myocardial Infarction (III) Treatment Effect Diminishes When Mean Time-to-Treatment Is Between 3.1 and 5 Hours

  17. Subgroup Analysis: A Meta-analysis of Thrombolytic Therapy for Acute Myocardial Infarction (IV) Treatment Effect Diminishes Further When Mean Time-to-Treatment Is Between 5.1 and 10 Hours

  18. Subgroup Analysis: A Meta-analysis of Thrombolytic Therapy for Acute Myocardial Infarction (V) No Evidence of a Treatment Effect When Mean Time to Treatment Longer than 10 Hours

  19. Subgroup Analysis: A Meta-analysis of Vitamin E Doses and Mortality Miller ER 3rd, et al. Ann Intern Med. 2005;142:37-46. Reprinted with permission from the American College of Physicians.

  20. Hazards of Subgroup Analyses:Multiple Testing (I) • Subgroup analyses are a form of multiple testing. • When uncontrolled, multiple testing can yield spurious findings. • Most meta-analyses do not perform statistical adjustments for multiple testing.

  21. Hazards of Subgroup Analyses:Multiple Testing (II) • In the example that follows, we discuss subgroup analyses from the Second International Study of Infarct Survival (ISIS-2), a 2x2 factorial study of streptokinase versus placebo and aspirin versus placebo in more than 17,000 patients with myocardial infarction. • For streptokinase versus placebo, the treatment effect does not differ across subgroups by history of prior myocardial infarction. • For aspirin versus placebo, the treatment effect does differ.

  22. Subgroup Analysis:Second International Study of Infarct Survival (ISIS-2) (I) Reprinted from ISIS-2 Collaborative Group. Lancet 1988;2:349-60, with permission from Elsevier.

  23. Subgroup Analysis:Second International Study of Infarct Survival (ISIS-2) (II) Reprinted from ISIS-2 Collaborative Group. Lancet 1988;2:349-60, with permission from Elsevier.

  24. Can You Avoid the Hazardsof Subgroup Analyses? (I) • When analyzing data, it is important to distinguish subgroup analyses that are specified a priori (without knowing what the data are) versus those that are specified post hoc (after the researcher has been exposed to the data). • This distinction is very clear when analyzing a prospective study. • In most meta-analyses, the distinction is not as clear.

  25. Can You Avoid the Hazardsof Subgroup Analyses? (II) • Most meta-analyses use data that are published (and potentially known). • When researchers adequately prepare before embarking on a meta-analysis, they inevitably become acquainted with the data they will analyze. • This makes it difficult for researchers to claim that they specified subgroups without knowing anything about their data.

  26. Can You Avoid the Hazards ofSubgroup Analyses? (III) • Meta-analysts should do their best to define subgroups that make methodological and biological sense. • Treat the results of subgroup analyses with a healthy dose of skepticism, especially when adjustments for multiple testing are not performed.

  27. Beyond Subgroup Analyses:Meta-Regression • Meta-regression can help examine how the treatment effect changes across the levels of a variable. • All subgroup analyses can be formulated in a meta-regression framework, but meta-regression goes well beyond subgroup analyses.

  28. Subgroup Analysis: A Meta-analysis of Vitamin E Doses and Mortality Miller ER 3rd, et al. Ann Intern Med. 2005;142:37-46. Reprinted with permission from the American College of Physicians.

  29. Corresponding Univariate Meta-Regression:A Meta-analysis of Vitamin E Doses and Mortality Miller ER 3rd, et al. Ann Intern Med. 2005;142:37-46. Reprinted with permission from the American College of Physicians.

  30. ~τ’2 ~τ2 Meta-Regression: A Meta-analysis of Zidovudine Monotherapy Versus Placebo Ioannidis JPA. In: Publication bias in meta-analysis: prevention, assessment and adjustments. 2005.

  31. Two Types of Covariates in Meta-Regressions • Study level • Examples: presence/absence of blinding, intervention dose (in experimental studies) • Participant level • Examples: mean age, proportion of diabetic patients, mean intake of vitamin E (in observational studies)

  32. Spurious Associations in Meta-Regressions or Subgroup Analyses (I) • Aggregate-data meta-regressions on participant-level covariates can mislead, because they are susceptible to ecological fallacy. • The observed relationship between the study-level treatment effect and the mean of a patient-level factor does not necessarily reflect the corresponding true relationship at the individual patient level.

  33. Spurious Associations inMeta-Regressions or Subgroup Analyses (II) • Therefore, associations of treatment effect and participant-level covariates should be interpreted with caution. • Such associations can be biologically plausible and informative, or can mislead. • Unfortunately, there is no universal way to distinguish true from fallacious findings.

  34. Control-Rate Meta-Regression • In control-rate meta-regression, we examine whether the treatment effect changes across studies with different event rates in the controls. • The control rate is a surrogate for the baseline risk for the event of interest (severity of disease). • This approach examines whether the underlying risk of an event explains differences in the treatment effect across studies.

  35. Intravenous Streptokinase Therapy inPatients With Acute Myocardial Infarction (I)

  36. Intravenous Streptokinase Therapy inPatients With Acute Myocardial Infarction (II)

  37. Control Rate Meta-Regression in the Preceding Streptokinase Example Schmid CH, et al. Stat Med 1998;17:1923-42.

  38. Key Messages • Subgroup analyses, meta-regressions, and control-rate meta-regressions are tools to explore between-study heterogeneity. • Use them to understand data. • They are mostly hypothesis-forming tools. • Especially for meta-regressions on patient-level covariates, ecological fallacy may mislead. • Beware when interpreting their results.

  39. References (I) • Barrington KJ. Umbilical artery catheters in the newborn: effects of position of the catheter tip. Cochrane Database Syst Rev 2000;(2):CD000505. • DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-88. • ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomized trial of intravenous streptokinase, oral aspirin, both, or neither among 17,817 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988;2:349-60. • Miller ER 3rd, Pastor-Barriuso R, Dalal D, et al. Meta-analysis: high-dosage vitamin E supplementation may increase all-cause mortality. Ann Intern Med 2005;142:37-46.

  40. References (II) • Pakos E, Ntzani EE, Trikalinos TA. Patellar resurfacing in total knee arthroplasty. A meta-analysis. J Bone Joint Surg Am 2005;87:1438-45. • Ioannidis JPA. Differentiating biases from genuine heterogeneity: distinguishing artifactual from substantive effects. In: Rothstein HR, Sutton AJ and Borenstein M, eds. Publication bias in meta-analysis: prevention, assessment and adjustments. Chichester, England: Wiley; 2005. p. 287-302. • Schmid CH, Lau J, McIntosh MW, et al. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med 1998;17:1923-42.

  41. Authors • This presentation was prepared by Joseph Lau, M.D., and Thomas Trikalinos, M.D., Ph.D., members of the Tufts Medical Center Evidence-based Practice Center. • The information in this module is based on Chapter 9 in Version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf).

More Related