Introduction to Critical Appraisal

1. Introduction to Critical Appraisal Yulia Lin, MD, FRCPC Transfusion Medicine Specialist, Dept of Clinical Pathology Sunnybrook Health Sciences Centre University of Toronto TMR Journal Club, September 4, 2008

2. Why is Critical Appraisal important?

3. Vast amount of literature

4. New Studies in Transfusion

5. New Studies in Transfusion Past Studies in Transfusion

6. New Studies in Transfusion Studies in other Sub-specialties Past Studies in Transfusion

7. Why critical appraisal is important? • Impossible to keep up with all of the literature • Enables us to distinguish stronger evidence from weaker evidence • Allows us to appropriately incorporate the evidence into our practice so as to improve patient care

8. Medical Expert • Apply principles to evaluate quality of research publications • Scholar • Critically appraise sources of medical information • Understand the concepts of clinical research design • Demonstrate knowledge of basic statistics and epidemiology

9. Purpose of TMR Journal Club • To present a recent article in the transfusion literature • To learn how to critically appraise the medical literature

10. Steps to a successful JC • Choosing the article • Presenting the article • Critically appraising the article

11. Choosing an article… • Where to find an article? • Recent table of contents • NEJM, Lancet, JAMA • Transfusion, Vox Sanguinis, Blood, etc. • Transfusion Medicine Reviews – Journal Club • Transfusion Medicine Community • List of interesting articles

12. Choosing the article… • Which article? • Relevant • Interesting to you • What type of article? • Any article • But some are easier than others…

13. Type of articles • Randomized controlled trials – Therapy • Observational studies – Harm • Systematic Reviews • Guidelines • Diagnosis • Prognosis

14. First, a review…

15. Randomized Controlled Trial Intervention A Population selection Outcomes measured Randomization Intervention B

16. Randomized Controlled Trial Intervention A Population Outcomes measured Randomization Intervention B • Is aprotinin superior to lysine analogs in • high-risk cardiac surgery? • Fergusson DA et al. NEJM May 29, 2008.

17. Observational Case-Control Population Exposure to Risk Factor Exposed Cases (+Disease) Yes No Control (No disease) Yes No

18. Observational Case-Control Population Exposure to Risk Factor Exposed Cases (+Disease) Yes No Control (No disease) Yes No Retrospective

19. Observational Case-Control Population Exposure to Risk Factor Exposed Cases (+Disease) Yes No Control (No disease) Yes No Blood transfusion, anesthesia, surgery and risk of NHL in a population-based case-control study. Cerhan JR et al. Int J Cancer 2008 Aug 15.

20. Observational Cohort Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed

21. Cohort Studies Past Present Future Cohort assembled Follow-up Retrospective Cohort Study

22. Cohort Studies Past Present Future Cohort assembled Follow-up Retrospective Cohort Study The ratio of fibrinogen to red cells transfused affects survival in casualties receiving massive transfusions at an army combat support hospital. Stinger HK et al., J Trauma February 2008;64:S79-85. Red blood cell to plasma ratios transfused during massive transfusion are associated with mortality in severe multiply Injury: a retrospective analysis from Deutsche Gesellschaft. Maegele M et al. Vox Sang August 2008;95:112-119.

23. Cohort Studies Past Present Future Cohort assembled Follow-up Retrospective Cohort Study Cohort assembled Follow-up Prospective Cohort Study

24. Cohort Studies Past Present Future Cohort assembled Follow-up Retrospective Cohort Study Cohort assembled Follow-up Prospective Cohort Study Prospective evaluation of a transfusion policy of D+ red blood cells into D- patients. Gonzalez-Porras JR et al. Transfusion July 2008.

25. Systematic Review • Overview • Summary of the medical literature that attempts to address a focused clinical question • Systematic review • Using methods designed to reduce the likelihood of bias • Meta-analysis • Review that uses quantitative methods to summarize the results

26. Systematic Review • Systematic review • Using methods designed to reduce the likelihood of bias • Meta-analysis • Review that uses quantitative methods to summarize the results Comparing the efficacy and safety of apheresis and whole-blood derived platelet transfusions: a systematic review. Heddle NM et al. Transfusion July 2008.

27. Guidelines • “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” • Attempt to address all issues and values relevant to a clinical decision • Attempt to distill a large body of medical expertise into a convenient, readily usable format • Make explicit recommendations with definite intent to influence what clinicians do Field MJ. Clinical practice guidelines. 1990 Hayward et al. JAMA 1995;274:570-4 Cook et al. Ann Int Med 1997;127:210-6

28. Guidelines • “Systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” • Guidelines for policies on alternatives to allogeneic blood transfusion: • Predeposit autologous blood donation and transfusion • Boulton FE et al. Transfusion Medicine 2007;17:354-65

29. How to do a critical appraisal?

30. Critical appraisal methods • Most are based on Users’ Guides to the Medical Literature • Available in a textbook form (2nd ed) but also on-line at http://ugi.usersguides.org/usersguides/hg/hh_start.asp • Centre for Health Evidence Website: http://www.cche.net/usersguides/main.asp • For guidelines, different instruments • AGREE instrument: http://www.agreetrust.org/instrument.htm • GRADE instrument available at Blood Depot

34. Steps to Critical Appraisal • Are the results valid? • Have the results been influenced in a systematic fashion so as to lead to a false conclusion? • What are the results? • How can I apply the results to patient care?

35. RCT Intervention A Population Outcomes measured Randomization Intervention B

36. RCT - Are the results valid? Intervention A Population Outcomes measured Randomization Intervention B Did experimental & control groups begin the study with a similar prognosis? Were patients randomized? Was randomization concealed? Were patients analyzed in the groups to which they were randomized? Were groups similar with respect to known prognostic variables?

37. RCT - Are the results valid? Intervention A Population Outcomes measured Randomization Intervention B Did experimental & control groups retain a similar prognosis after the study started? Were patients, clinicians, outcome assessors blinded? Was follow-up complete?

38. RCT - What were the results? Intervention A Population Outcomes measured Randomization Intervention B How large was the treatment effect? How precise was the estimate of the treatment effect?

39. RCT - How can I apply the results to patient care? Intervention A Population Outcomes measured Randomization Intervention B Were the study patients similar to the patient in my practice?

40. RCT - How can I apply the results to patient care? Intervention A Population Outcomes measured Randomization Intervention B Were the study patients similar to the patient in my practice? Were all clinically important outcomes considered? Are the likely treatment benefits worth the potential harm and costs?

41. Observational Cohort Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed

42. Cohort - Are the results valid? Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed Did expt and control groups begin the study with a similar prognosis? Did the investigators demonstrate similarity in all known determinants of outcome? Did they adjust for differences in the analysis?

43. Cohort - Are the results valid? Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed Did expt and control groups retain a similar prognosis after the study started? Were the outcomes measured in the same way in the groups being compared? Was follow-up sufficiently complete?

44. Cohort - What are the results? Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed How strong is the association between exposure and outcome? Consider magnitude and dose response? How precise is the estimate of the risk?

45. Cohort - How can I apply the results? Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed Were the study patients similar to my patients?

46. Cohort - How can I apply the results? Cohort without Disease Population Exposure to Risk Factor Disease Yes No Exposed Cohort Yes No Not exposed Were the study patients similar to my patients? Was the duration of follow-up adequate? Was was the magnitude of the risk? Should I attempt to stop the exposure?

47. Putting it all together…

48. Tips for TMR Journal Club • Choose the article carefully • Present the article briefly (10 slides) • The main objective is the critical appraisal • Internal validity: • Focus on 1-2 key points • Try explaining a new concept (bias, statistical concept) • External validity: Take a stand • Allow ½ the time for discussion