Meta-analysis Principles and practice in cardiovascular research

1. Meta-analysisPrinciples and practice in cardiovascular research Giuseppe Biondi Zoccai Istituto di Cardiologia, Università di Torino

2. Disclosure • No funding or conflict of interest to declare

3. Index • Introduction & definitions • Scientific hierarchy • The Cochrane Collaboration • Structured approach to systematic reviews • Additional topics • Statistical packages • Further examples

4. Exponential increase in PubMed citations PubMed search strategy: ("2001"[PDAT] : "2005"[PDAT]) AND (("systematic"[title/abstract] AND "review"[title/abstract]) OR ("systematic"[title/abstract] AND "overview"[title/abstract]) OR ("meta-analysis"[title/abstract] OR "meta-analyses"[title/abstract]))

5. Famous quotes “If I have seen further it is by standing on the shoulders of giants” Isaac Newton “The great advances in science usually result from new tools rather than from new doctrines” Freeman Dyson

6. Famous quotes “I like to think of the meta-analytic process as similar to being in a helicopter. On the ground individual trees are visible with high resolution. This resolution diminishes as the helicopter rises, and in its place we begin to see patterns not visible from the ground” Ingram Olkin

7. Baby steps of meta-analysis • 1904 - Karl Pearson (UK): correlation between inoculation of vaccine for typhoid fever and mortality across apparently conflicting studies • 1931 – Leonard Tippet (UK): comparison of differences between and within farming techniques on agricultural yield adjusting for sample size across several studies • 1937 – William Cochran (UK): combination of effect sizes across different studies of medical treatments • 1970s – Robert Rosenthal and Gene Glass (USA), Archie Cochrane (UK): combination of effect sizes across different studies of, respectively, educational and psychological treatments • 1980s – exponential development/use of meta-analytic methods

8. Minimal glossary • Review: viewpoint on a subject quoting different primary authors • Overview: as above • Qualitative review: deliberately avoids a systematic approach • Systematic review: deliberately uses a systematic approach to study search, selection, abstraction, appraisal and pooling • Quantitative review: uses quantitative methods to appraise or synthesize data • Meta-analysis: uses specific statistical methods for data pooling and/or exploratory analysis • Individual patient data meta-analysis: uses specific stastistical methods for data pooling or exploration exploiting individual patient data → Our goal:systematic review (± meta-analysis)

9. Qualitative review

10. Systematic review and meta-analyses • What is a systematic review? • A systematic appraisal of the methodological quality, clinical relevance and consistency of published evidence on a specific clinical topic in order to provide clear suggestions for a specific healthcare problem • What is a meta-analysis? • A quantitative synthesis that, preserving the identity of individual studies, tries to provide an estimate of the overall effect of an intervention, exposure, or diagnostic strategy

11. Systematic review

12. Systematic review and meta-analyses

13. Individual patient data meta-analysis • Ideally should be a systematic review and meta-analysis based on individual patient data • Major pros: • a unique database containing primary studies is created and used (consistency checks and homogenous variables are created) • the same analytical tools can be used across studies • subgroup analyses can be performed even for groups that were not reported in the original publications • Major cons: • some studies may have to be excluded (publication bias) because original authors may not provide source data • poses major logistical and financial challenges

14. Individual patient data meta-analysis BMJ 2002

15. Systematic review and meta-regression • A meta-regression employs meta-analytic methods to explore the impact of covariates or moderators on the main effect measure or on other • All the limitations of non-RCT studies applies, and thus they should mainly be regarded as hypothesis generating

16. Meta-regression

17. Cumulative and prospective meta-analyses • A cumulative meta-analysis recomputes and plots the pooled effect estimate every time a new study is added • A prospective meta-analysis is based on a specific a priori protocol for its conduct, analysis, and reporting, and may use also a cumulative design

18. Cumulative meta-analysis Antman et al, JAMA 1992

19. Cumulative meta-analysis

20. Pros • Systematic searches for clinical evidence • Explicit and standardized methods for search and selection of evidence sources • Thorough appraisal of the internal validity of primary studies • Quantitative synthesis with increased statistical power • Increased external validity by appraising the effect of an intervention (exposure) across different settings • Test subgroup hypotheses • Explore clinical and statistical heterogeneit Lau et al, Lancet 1998

21. Cons • “Exercise in mega-silliness” • “Mixing apples with oranges” • Not original research • Big RCTs definitely better • Pertinent studies might not be found, or may be of low quality or internal validity • Publication and small study bias • Average effect largely unapplicable to individuals Lau et al, Lancet 1998

22. Cons Smith et al, BMJ 2003

23. Biondi-Zoccai et al, BMJ 2006

24. Introduction • Introduction & definitions • Scientific hierarchy • The Cochrane Collaboration • Structured approach to systematic reviews • Additional topics • Statistical packages • Further examples

25. EBM hierarchy of evidence • N of 1 randomized controlled trial • Systematic reviews of homogeneous randomized trials • Single (large) randomized trial • Systematic review of homogeneous observational studies addressing patient-important outcomes • Single observational study addressing patient-important outcomes • Physiologic studies (eg blood pressure, cardiac output, exercise capacity, bone density, and so forth) • Unsystematic clinical observations Guyatt and Rennie, Users’ guide to the medical literature, 2002

26. Parallel hierarchy of scientific studies in cardiovascular medicine Qualitative reviews Case reports and series Observational studies Systematic reviews Observational controlled studies Meta-analyses from individual studies Randomized controlled trials Meta-analyses from individual patient data Multicenter randomized controlled trials Biondi-Zoccai, Ital Heart J 2003

28. Benson et al, NEJM 2000

29. Introduction • Introduction & definitions • Scientific hierarchy • The Cochrane Collaboration • Structured approach to systematic reviews • Additional topics • Statistical packages • Further examples

31. The Cochrane Collaboration Mission Statement: The Cochrane Collaboration is an world-wide organisation that aims to help people make well­informed decisions about healthcare by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of healthcare interventions

32. The Cochrane Collaboration • About 6000 contributors • 50 Collaborative Review Groups (CRGs) • 12 Centres throughout the world • 9 Fields • 11 Methods Groups • 1 Consumer Network • Campbell Collaboration

33. The Cochrane Collaboration Objectives: • Collaboration • Building on the enthusiasm of individuals • Avoiding duplication • Minimising bias • Keeping up to date • Striving for relevance • Promoting access • Ensuring quality • Continuity

34. The Cochrane Collaboration • Cochrane Database of Systematic Reviews (CDSR) – contains Cochrane systematic reviews • Database of Abstracts of Reviews of Effectiveness (DARE) – contains abstracts of non-Cochrane reviews • Cochrane Central Controlled Trials Register (CENTRAL) – contains titles or abstracts of RCTs from multiple sources • Cochrane Database of Methodology Reviews – contains Cochrane reviews of methods papers • Cochrane Methodology Register (CMR) – contains abstracts of non-Cochrane methods papers • Health Technology Assessment Database (HTA) – contains abstracts of HTA papers • NHS Economic Evaluation Database (NHS EED) – contains abstracts of economic analysis papers

35. Introduction • Introduction & definitions • Scientific hierarchy • The Cochrane Collaboration • Structured approach to systematic reviews • Additional topics • Statistical packages • Further examples

36. Algorithm for systematic reviews • Definition of question and hypothetical solution • Prospective design of the systematic review • Problem formulation (population, intervention or exposure, comparison, outcome [PICO]) • Data search • Data abstraction and appraisal • Data analysis ± quantitative synthesis • Result interpretation and dissemination FEED-BACK ON HYPOTHESIS Biondi-Zoccai et al, Ital Heart J 2004

37. Definition of question and prospective design • The clinical question should be clearly stated, being as much explicit as possible • The review should be designed in as much details as possible, and yet with a limited a priori knowledge of the subject Biondi-Zoccai et al, Ital Heart J 2004

38. Problem formulation according to the PICO approach • Population of interest - eg elderly male >2 weeks after myocardial infarction) • Intervention (or exposure) – eg intracoronary infusion of progenitor blood cells • Comparison – eg patients treated with progenitor cells vs standard therapy • Outcome(s) – eg change in echocardiographic left ventricular ejection fraction from discharge to 6-month control Biondi-Zoccai et al, Ital Heart J 2004

39. Data search • After definition of question according to PICO approach, the appropriate key-words are used to search several databases • Useful resources: BioMedCentral, CENTRAL, clinicaltrials.gov, EMBASE, LILACS, and PubMed • Conference proceedings • Cross-referencing (snowballing) • Contact with experts

40. Study search A simple PubMed strategy for clinical studies on LM PCI: left AND main AND coronary AND stent* NOT case reports [pt] NOT review [pt] NOT editorial [pt] A complex PubMed strategy for randomized clinical trials on invasive vs conservative strategies in ACS: (randomized controlled trial[pt] OR controlled clinical trial[pt] OR randomized controlled trials[mh] OR random allocation[mh] OR double-blind method[mh] OR single-blind method[mh] OR clinical trial[pt] OR clinical trials[mh] OR (clinical trial[tw] OR ((singl*[tw] OR doubl*[tw] OR trebl*[tw] OR tripl*[tw]) AND (mask*[tw] OR blind[tw])) OR (latin square[tw]) OR placebos[mh] OR placebo*[tw] OR random*[tw] OR research design[mh:noexp] OR comparative study[mh] OR evaluation studies[mh] OR follow-up studies[mh] OR prospective studies[mh] OR cross-over studies[mh] OR control*[tw] OR prospectiv*[tw] OR volunteer*[tw]) NOT (animal[mh] NOT human[mh]) NOT (comment[pt] OR editorial[pt] OR meta-analysis[pt] OR practice-guideline[pt] OR review[pt])) AND ((invasive OR conservative AND (coronary OR unstable angina OR acute coronary syndrome* OR unstable coronary syndrome* OR myocardial infarction))) Biondi-Zoccai et al, Int J Epidemiol 2005

41. Study search Reveiz et al, J Clin Epidemiol 2006

42. Study selection • 1st - screening of titles and abstracts • 2nd – potentially pertinent citations are then retrieved as full reports and appraised according to prespecified and explicit inclusion/exclusion criteria • 3rd – studies fullfilling both inclusion and exclusion criteria, are then included in the systematic review

43. Andreotti et al, Eur Heart J 2005

44. Data abstraction and appraisal • Abstraction of outcomes and moderator variables, possibly on prespecified data form • Appraisal of the internal validity of primary studies (eg the risk of selection, performance, adjudication and attrition bias) • Performed by single vs multiple reviewers, with divergences resolved by consensus (possibly after formal tests for agreement)

45. Data extraction Buscemi et al, J Clin Epidemiol 2006

46. Internal validity of primary studies • Many scales for the quality of included studies have been reported, but none is reliable or robust • The recommended approach is to individually appraise the potential risk of the 4 biases (eg A-low, B-moderate, C-high, D-unclear from reported data): • Selection bias(one group is different than the other) • Performance bias(treatment is systematically different) • Adjudication bias(outcome adjudication is selectively different) • Attrition bias(follow-up duration or completeness is different)

47. Quality scales are unreliable

48. Quality scales are unreliable

49. Internal validity of primary studies Hill et al, Eur Heart J 2004

50. Data synthesis • Quantitative data synthesis is central to the practice of meta-analysis, and is based on a major assumptio: individual studies that are going to be pooled are relatively homogeneous, both clinically and statistically, to provide a meaningful central tendency effect estimate