1 / 126

youtube/watch?v=QUW0Q8tXVUc

http://www.youtube.com/watch?v=QUW0Q8tXVUc. Quiz For each question choose at least one answer…. Question: What is the purpose of a systematic review?. to get a publication to explore the differences between available studies to help people make decisions about whether to use the intervention

casey-mcintosh
Download Presentation

youtube/watch?v=QUW0Q8tXVUc

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. http://www.youtube.com/watch?v=QUW0Q8tXVUc

  2. QuizFor each question choose at least one answer…

  3. Question:What is the purpose of a systematic review? • to get a publication • to explore the differences between available studies • to help people make decisions about whether to use the intervention • to reach a definite conclusion about whether the intervention works or not

  4. Answer:The purpose of a systematic review is to • to get a publication • to explore the differences between available studies • to help people make decisions about whether to use the intervention • to reach a definite conclusion about whether the intervention works or not

  5. Question: What are key features of a systematic review? • explicit, pre-defined eligibility criteria are used to select included studies • studies can be excluded if they come to the wrong conclusions • a systematic and comprehensive search to find all available evidence • a systematic and comprehensive search to find all the studies that meet the eligibility criteria • critical appraisal is used to assess the reliability of the included studies

  6. Answer: The key features of a systematic review are • explicit, pre-defined eligibility criteria are used to select included studies • studies can be excluded if they come to the wrong conclusions • a systematic and comprehensive search to find all available evidence • a systematic and comprehensive search to find all the studies that meet the eligibility criteria • critical appraisal is used to assess the reliability of the included studies

  7. Introduction tometa-analysis

  8. Steps of a Cochrane review • define the question • plan eligibility criteria • plan methods • search for studies • apply eligibility criteria • collect data • assess studies for risk of bias • analyse and present results • interpret results and draw conclusions • improve and update review

  9. Session outline • principles of meta-analysis • steps in a meta-analysis • presenting your results See Chapter 9 of the Handbook

  10. Study level ↓ Review level ↓ Outcome data Outcome data Outcome data Outcome data Effect measure Study A Effect measure Study B Effect measure Effect measure Study C Effect measure Study D Source: Jo McKenzie & Miranda Cumpston

  11. What is a meta-analysis? • combines the results from two or more studies • estimates an ‘average’ or ‘common’ effect • optional part of a systematic review Source: Julian Higgins

  12. Why perform a meta-analysis? • quantify treatment effects and their uncertainty • increase power • increase precision • explore differences between studies • settle controversies from conflicting studies • generate new hypotheses Source: Julian Higgins

  13. When not to do a meta-analysis • mixing apples with oranges • each included study must address same question • consider comparison and outcomes • requires your subjective judgement • combining a broad mix of studies answers broad questions • answer may be meaningless and genuine effects may be obscured if studies are too diverse Source: Julian Higgins

  14. When not to do a meta-analysis • garbage in – garbage out • a meta-analysis is only as good as the studies in it • if included studies are biased: • meta-analysis result will also be incorrect • will give more credibility and narrower confidence interval • if serious reporting biases present: • unrepresentative set of studies may give misleading result Source: Julian Higgins

  15. When can you do a meta-analysis? • more than one study has measured an effect • the studies are sufficiently similar to produce a meaningful and useful result • the outcome has been measured in similar ways • data are available in a format we can use

  16. Session outline • principles of meta-analysis • steps in a meta-analysis • presenting your results

  17. Steps in a meta-analysis • identify comparisons to be made • identify outcomes to be reported and statistics to be used • collect data from each relevant study • combine the results to obtain the summary of effect • explore differences between the studies • interpret the results

  18. Selecting comparisons • break your topic down into pair-wise comparisons • each review may have one or many • use your judgement to decide what to group together, and what should be a separate comparison Hypothetical review: Caffeine for daytime drowsiness vs decaffeinated coffee caffeinated coffee

  19. Selecting outcomes & effect measures • for each comparison, select outcomes • for each outcome, select an effect measure • may depend on the available data from included studies Hypothetical review: Caffeine for daytime drowsiness vs • asleep at end of trial (RR) • irritability (MD/SMD) • headaches (RR) decaffeinated coffee caffeinated coffee

  20. Calculating the summary result • collect a summary statistic from each contributing study • how do we bring them together? • treat as one big study – add intervention & control data? • breaks randomisation, will give the wrong answer • simple average? • weights all studies equally – some studies closer to the truth • weighted average

  21. Weighting studies • more weight to the studies which give more information • more participants, more events, narrower confidence interval • calculated using the effect estimate and its variance • inverse-variance method:

  22. For example

  23. For example

  24. Meta-analysis options • for dichotomous or continuous data • inverse-variance • straightforward, general method • for dichotomous data only • Mantel-Haenszel (default) • good with few events – common in Cochrane reviews • weighting system depends on effect measure • Peto • for odds ratios only • good with few events and small effect sizes (OR close to 1)

  25. Meta-analysis options

  26. Session outline • principles of meta-analysis • steps in a meta-analysis • presenting your results

  27. A forest of lines Trees Joyce Kilmer Forest by charlescleonard http://www.flickr.com/photos/charlescleonard/3754931947/

  28. Forest plots Headache at 24 hours • headings explain the comparison

  29. Forest plots Headache at 24 hours • list of included studies

  30. Forest plots Headache at 24 hours • raw data for each study

  31. Forest plots Headache at 24 hours • total data for all studies

  32. Forest plots Headache at 24 hours • weight given to each study

  33. Forest plots Headache at 24 hours • effect estimate for each study, with CI

  34. Forest plots Headache at 24 hours • effect estimate for each study, with CI

  35. Forest plots Headache at 24 hours • scale and direction of benefit

  36. Forest plots Headache at 24 hours • pooled effect estimate for all studies, with CI

  37. Interpreting confidence intervals • always present estimate with a confidence interval • precision • point estimate is the best guess of the effect • CI expresses uncertainty – range of values we can be reasonably sure includes the true effect • significance • if the CI includes the null value • rarely means evidence of no effect • effect cannot be confirmed or refuted by the available evidence • consider what level of change is clinically important

  38. Considering clinical significance Based on Sanders S, Glasziou PP, Del Mar C, Rovers MM. Antibiotics for acute otitis media in children. Cochrane Database of Systematic Reviews 2004, Issue 1. Art. No.: CD000219. DOI: 10.1002/14651858.CD000219.pub2.

  39. The Results section of your review • a systematic, narrative summary of results • forest plots • key forest plots linked as figures • usually primary outcomes • avoid forest plots with only one study • may also add other data tables • results of single studies • summary data for each group, effect estimates, confidence intervals • non-standard data • not helpful to report trivial outcomes or results at high risk of bias

  40. What to include in the protocol • how will you decide whether a meta-analysis is appropriate? • meta-analysis model to be used

  41. Take home message • there are several advantages to performing a meta-analysis but it is not always possible (or appropriate) • plan your analysis carefully, including comparisons, outcomes and meta-analysis methods • forest plots display the results of meta-analyses graphically • interpret your results with caution

  42. Question: How are studies in a meta-analysis are combined? • by giving more weight to the studies with larger sample sizes and lower standard deviations, because they are likely to more accurately estimate the true intervention effect • by a simple average, to give each study equal weight • by giving less weight to studies with lower variance in treatment effect • by giving more weight to studies with higher variance in treatment effect

  43. Answer: Studies in a meta-analysis are combined • by giving more weight to the studies with larger samples sizes and lower standard deviations, because they are likely to more accurately estimate the true intervention effect • by a simple average, to give each study equal weight • by giving less weight to studies with lower variance in treatment effect • by giving more weight to studies with higher variance in treatment effect

  44. Collecting data

  45. Steps of a systematic review • define the question • plan eligibility criteria • plan methods • search for studies • apply eligibility criteria • collect data • assess studies for risk of bias • analyse and present results • interpret results and draw conclusions • improve and update review

  46. Outline • data to be collected • putting it into practice See Chapter 7 of the Handbook

  47. What data should you collect? • comprehensive information about each study • population and setting (e.g. age, race, sex, socioeconomic details, disease status, duration, severity, comorbidities) • interventions and integrity of delivery • methods and potential sources of bias • outcomes, authors’ conclusions • citation, author contact details • sources of funding, etc. • information required for: • references • description of included studies • risk of bias assessment • analyses

  48. Collecting outcome data • focus on those outcomes specified in your protocol • be open to unexpected findings, e.g. adverse effects • measures used • definition (e.g. diagnostic criteria, threshold) • timing • unit of measurement • for scales – upper and lower limits, direction of benefit, modifications, validation, minimally important difference • numerical results • may be in many formats - conversion may be required • collect whatever is available • no. participants for each outcome & time point

  49. Data in many formats Source: Phil Wiffen

  50. Outline • data to be collected • putting it into practice

More Related