1 / 48

Critical appraisal Systematic R eview

Critical appraisal Systematic R eview. กิตติพันธุ์ ฤกษ์ เกษม ภาควิชาศัลยศาสตร์ มหาวิทยาลัยเชียงใหม่. Systematic review vs Meta-analysis. Systematic review: a systematic approach to minimising bias and error

jock
Download Presentation

Critical appraisal Systematic R eview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Critical appraisal Systematic Review กิตติพันธุ์ ฤกษ์เกษม ภาควิชาศัลยศาสตร์ มหาวิทยาลัยเชียงใหม่

  2. Systematic review vs Meta-analysis • Systematic review: a systematic approach to minimising bias and error • Meta-analysis: a statistical analysis, which aim to produce a single estimate of a treatment effect • Systematic review may or may not include Meta-analysis

  3. Why do we need systematic review? • Theearly1980suncomplicatedMI “Shouldpt receive a beta-blockerforsecondarypreventionbeforedischarge?” Library:4randomisedcontrolledtrials (RCT)

  4. Beta-blocker vs placebo RCT 1.Mortality and hospital readmission is not different RCT 2. Not conclusive RCT 3. Beta-blocker not shown benefit RCT 4. Long term beta-blocker reduces the mortality and rate of re-infarction

  5. A review in BMJ 1981 • There is no clear evidence that beta-blocker improves long tem survival after MI despite almost 20 yrs of clinical trials • Good enough!!

  6. Another review in European Heart Journal 1981 “itseemsperfectlyreasonabletotreatpatientswhohavesurvivedaninfarctionwithbeta-blocker”

  7. Limitation of a single study • Too small sample size false negative

  8. Problem of Conventional review • Prone to bias and error • Select only evidence support the author’s view • Not specify methodological quality of studies • Finally choose most vote ignore sample size, and design

  9. Meta-analysis = combining all available data • Attractive alternative to such large, expensive and problematic study • Weight average of the result large > small trial

  10. Meta-analysis Beta-blocker trials - MI Beta-blocker better Placebo better

  11. Cumulative meta-analysis of beta-blocker trials

  12. Cumulative meta-analysis Significant effect from 1980 onwards (OR not across 1

  13. Meta-analysis Beta-blocker trials - MI Maybe Unnecessary trials Beta-blockerbetter Placebobetter

  14. Benefit • Estimate the overall effect • Examine different result between studies (heterogeneity) • Identified insufficient data

  15. Cochrane collaboration • International organisation of health care profession • Promoting accessibility of systematic review • Foster development of systematic review • 50 collaborative review groups www.cochrane.org/cochrane/ccweb.htm

  16. Potentials of systematic review • Good • Bad

  17. Systematic review Basic structure and types กิตติพันธุ์ ฤกษ์เกษม ภาควิชาศัลยศาสตร์ มหาวิทยาลัยเชียงใหม่

  18. 1. Basic structure

  19. Like primary research • Why- Introduction, background • How-method • What we found-result • What it mean-discussion

  20. Basic structure • Abstract • Introduction • Background • Objectives • Method “treat a paper like a patient in 1 reseach” • Type of studies • Inclusion criteria type of participants • Exclusion criteria • Type of intervention • Type of outcome measures • Search strategy for identification of study • Method of analysis

  21. Basic structure • Result (special diagram) • Conclusion • Reference

  22. Forest plot Stroke rate ‘LA vs GA in carotid sx trial’ 1966-2001 LA better GA better Blacksquare= OR, horizontalline = 95%confidenceinterval Areaofblacksquare = weight, diamond = combinedORwith 95%CI

  23. 2. types

  24. Types • Systematic review of primary research • Observational studies • Diagnostic screening • RCT

  25. 3. Method “treat a paper like a patient in 1 reseach”

  26. The process (1) • Research question • Writing protocol • Searching • Article retrieval • Literature review

  27. The process (2) • Inclusion/ exclusion criteria • Validity and quality of articles • Data extraction/ synthesis • Interpretation

  28. The question • Is local anesthesia is better than general anesthesia during carotid endarterectomy?

  29. Writing the protocol • Background • Objectives • Type of studies • Inclusion criteria • Type of participants • Exclusion criteria • Type of intervention • Type of outcome measures • Search strategy for identification of study • Method of analysis • Reference

  30. Searching • Medline • Other database • Hand searching the literature • Writing to people

  31. Getting the article • Which ones to get? • It takes time • Libraries • Inter-library Loans

  32. Literature review • Youdon’thavetoreadthewholepaperyet! • Translation

  33. Validity and quality of articles • Do read the paper and see what the author thought was wrong • Unequal intervention/control size • Hidden loss to follow up

  34. Data extraction • Read method carefully • Design a form

  35. Synthesis/ Interpretation • Estimates and confidence intervals pool effect make by statistic method e.g. Peto method (fix method) give more weight effect for large study than small study (P value) • Difference between studies (Heterogeneity) Chi-squared test (P value)

  36. Small RCTs show LA is marginal lower mortality than GA

  37. Critical Appraisal 1. Are the result valid? 2. What are the results?

  38. 1. Are the result valid? • Did this review address a sensible clinical question • Was the search for relavant studies detailed and exhaustive? • Were selection and assessment of studies reproducible? • Were the primary studies of high methodological quality?

  39. Publication bias “A (significant) beneficialtreatmenteffectarepublished, butanequalresultremainunpublished” • Ingeneralmedicaljournalandpublicheathjournalreportedstatisticallysignificant 85.4% • Inpsychologicaljournal 95.6%

  40. Time lag bias • “Positiveresultwilldominatetheliteratureforseveralyearuntilthenegativewillreportlater” • HIVtrialinUSA, median timetopublishofpositiveresult 4.2years, butnegativeresult 6.4years

  41. Duplicate publication bias • “ onestudypresentsandreportsseveraltimes” “ includethisleadtooverestimationoftreatmenteffect” • Ondersetrontopreventpostoperativenauseavomitting 16studies 3duplicatedpapers • Sometimesdifficulttosay, sincenotsharesinglecommonauthors!!!!!!

  42. Language bias • “Authors tend to report positive result in international papers, English language journal, but if negative result are published in local journal”

  43. Outcome reporting bias • In trials many outcome is recorded but only favorable finding will be reported • Clinical trials by drug companies, unpublished trials gave information on adverse effect > published trials

  44. Selection Bias • Tend to happen in non Randomised controlled trials (non RCT) • For example select low risk group to new treatment group

  45. 2. What are the result? • Were the results similar from study to study? If yes, the credit of single estimates is OK. • Point estimates similar? • Overlapping confidence interval • Test for heterogeneity? (Chi square test) • Percentage of variability (I2 ) good < 20%, concern 20-50%,serious concern > 50%

  46. Look overlapping confidence interval Rerkasem Cochrane Database Syst Rev 2008; (4):CD000126.

  47. Small RCTs show LA is marginal lower mortality than GA Rerkasem Cochrane Database Syst Rev 2008 (4):CD000126.

  48. 2. What are the result? • Were the results similar from study to study? • What are the overall results of the reviews? • How precise were the results?

More Related