實証醫學 Critical Appraisal In Evidence-Based Medicine

2. 實証醫學Critical Appraisal In Evidence-Based Medicine 柯德鑫, MD, PhD. 奇美醫學中心 神經內科

3. Evidence-Based Medicine (EBM) • The consistent use of current best evidence derived from published clinical and epidemiologic research in management of patients, with attention to the balance of risks and benefits of diagnostic tests and alternative treatment regimens, taking account of each patient’s unique circumstances, including baseline risk, comorbid conditions and personal preferences.

4. Evidence-Based Medicine (EBM) • Meta-analysis is an overview in which uses quantitative methods to summarise the results from different clinical trials. • Odds Ratio describes the odds of an experimental patient suffering an adverse event relative to a control patient.

5. What evidence-based medicine is: The practice of EBM is the integration of • individual clinical expertise with the • best available external clinical evidence from systematic research. and • patient’s values and expectations

6. What is Evidence-Based Medicine? • See a patient • Ask a question • Seek the best evidence for that question • Appraise that evidence • Apply the evidence • Monitor the change

7. 健康狀態 疾病 病人 病態生理變化 感覺不適 診斷條件： 特異性高的發現 統計上相關性 相近的病態生理/病理機制 病史詢問 理學檢查 臨時診斷思考 醫師 形成最可能的假設診斷 相關的檢驗與檢查 證實假設診斷

8. Evidence-Based Medicine • Ask a question • Acquire some articles • Appraise the evidence • Apply the findings • Assess your performance

9. 證據等級之定義

11. What is critical appraisal? Critical appraisal is the process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision.

12. Critical Appraisal • The process of deciding whether a piece of research can help you in answering your clinical question. Three questions you need to ask about any kind of research: • 1. Is it valid? • 2.Is it important? • 3. Is it applicable to the patient?

13. Critical Appraisal Skills • VIP Valid:研究方法的探討 Important:結論的分析 Practice:如何運用來照顧病人 RCT/Cohort study/Case-control study

14. How to Appraise Evidence? ·By Expertise 　-Secondary journal 　-CAT(critical appraisal topics) 　-Cochrane database 　-JAMA & BMJ Guide 　-Basic Clinical Statistics ·Have an appraisal course ·EBM style journal meeting

15. Evaluation of A Diagnostic Test • Sensitivity is the proportion of people with the disease who have a positive test. • Specificity is the proportion of people free of the disease who have a negative test. • Positive Predictive Value (+PV) is the proportion of people with a positive test who truly have the disease. • Negative Predictive Value (-PV) is the proportion of people with a negative test who are free of the disease.

16. Evaluation of A Diagnostic Test • SnNout when a sign/test has a high sensitivity, a negative result rules out the diagnosis; e.g. the sensitivity of a history of ankle swelling for diagnosing ascites is 92%, therefore if a person does not have a history of ankle swelling, it is highly unlikely that the person has ascites. • SpPin when a sign/test has a high specificity, a Positive result rules in the diagnosis; e.g. the specificity of fluid wave for diagnosing ascites is 92%. Therefore, if a person has a fluid wave, it is highly likely that the person has ascites.

18. Critical Appraisal for Diagnosis • Are the results of this diagnostic study valid? • Are the valid results of this diagnostic study important? • Can you apply this valid, important evidence about a diagnostic test in caring for your patient?

19. Are the results of this diagnostic study valid? • Was there an independent, blind comparison with a reference ("gold") standard of diagnosis? • Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom we would use it in practice)? • Was the reference standard applied regardless of the test result? • Was the test (or cluster of tests) validated in a second, independent group of patients?

20. Are the valid results of this diagnostic study important?

21. Are the valid results of this diagnostic study important? • Sensitivity = a / (a+c) = 371/809 = 90 % Specificity = d / (b+d) = 1500/1770 = 85 %LR+ = sens / (1-spec) = 90/15 = 6LR- = (1-sens) / (spec) = 10/85 = 0.12Positive Predictive Value = a / (a+b) = 731/1001 = 73 % Negative Predictive value = d / (c+d) = 1500/1578 = 95 % Prevalence = (a+c) / (a+b+c+d) = 809/2579 = 32 %Pre-test odds = prevalence / (1-prevalence) = 31/69 = 0.45Post-test odds = pre-test odds * LRPost-test Probability = post-test odds / (post-test odds + 1)

22. Can you apply this valid, important evidence about a diagnostic test in caring for your patient? • Is the diagnostic test available, affordable, accurate and precise in your setting? • Can you generate a clinically sensible estimate of your patient's pre-test probability (from practical data, from personal experience, from the report itself, or from clinical speculation)?

23. Can you apply this valid, important evidence about a diagnostic test in caring for your patient? • Will the resulting post-test probabilities affect your management and help your patient? (Could it move you across a test-treatment threshold? Would your patient be a willing partner in carrying it out?) • Would the consequences of the test help your patient?

25. Evidence-Based Medicine (EBM) in Clinical Trial • Number Needed to Treat (NNT) is the number of patients who need to be treated to prevent one bad outcome. • Number Needed to Harm (NNH) is the number of patients under treatment that is need to develop one bad outcome or adverse effect.

26. Evidence-Based Medicine (EBM) in Clinical Trial • Intention-to-treat analysis:including all patients in the analysis who are randomized to treatment, regardless of what occurs during the clinical trial. • Adherence-to-protocal (On treatment) analysis: analyzing patients based on the treatment they received finally.

27. Therapies • To treat or not to treat? • Validity • Importance • Applicability

28. Therapies • Validity • Was it randomised? • Was the allocation concealed? • Were the all the subjects analysed correctly? • Was it blinded? • Were the groups similar?

29. Therapies • Importance • What were the results? • Over what time period? • With what precision?

30. Therapies • Number needed to treat • Relative risk reduction • Absolute risk reduction • Event rates

31. Therapies • Event rates • n with event / total • Control event rate (CER) • Experimental event rate (EER)

33. Therapies • Absolute risk reduction • difference in two event rates • CER - EER = ARR • Relative risk reduction • proportion of control rate • CER-EER / CER = RRR

34. Therapies • Number needed to treat • number of extra patients you need to treat to prevent one bad outcome • 1 / ARR = NNT

35. Therapies • 95% confidence interval • range within which the true value falls with 95% confidence • use computer (e.g. CATMaker)

36. Occurrence of death, stroke, or other major complications

37. THERAPY WORKSHEETCitation:

38. THERAPY WORKSHEETAre the valid results of this randomized trial important? SAMPLE CALCULATIONS:

39. THERAPY WORKSHEETSAMPLE CALCULATIONS: 95% Confidence Interval (CI) on an NNT = 1 / (limits on the CI of its ARR) = YOUR CALCULATIONS:

40. Therapies • Application • Can it be applied to my patient? • Can it be done here? • How do patient values affect the decision?

41. Therapies • Is it valid? • Is it important? • NNT for what over how long with what precision • Does it apply?

43. Six guides to distinguish useful from useless or even harmful therapy 1. Was the assignment of patients to treatments really randomized？ 2. Was all clinically relevant outcomes reported？ 3. Were the study patients recognizably similar to your own？ 4. Were both clinical and statistical significance considered？ 5. Is the therapeutic maneuver feasible in your practice？ 6. Were all patients who entered the study accounted for its conclusion？

44. The likelihood of help vs. harm (LHH) In applying a SR or RCT to an individual patient, we need to consider: •our patient's risk, relative to patients in the trial, of the event we hope to prevent with the treatment: ft •our patient's risk, relative to patients in the trial, of the side-effect we might cause from the treatment: fh •our patient's perception of the severity of the event we're trying to prevent relative to the side-effect we might cause: s

45. The likelihood of help vs. harm is(1 /NNT) x ft x s vs. (1/NNH)x fh For example, suppose we're applying a trial with an NNT of 9 and an NNH of 12 and we think our patient is at just half the risk of the event but at twice the risk of the side-effect, then the "raw" LHH before we adjust it for our patient's perception of relative severity is 1/9 x 0.5 vs. 1/12 x 2 = 1/18 vs. 1/6, or three times as likely to harm vs. help the patient. However, if our patient regards the severity of the event that the treatment might prevent to be six times worse than the side-effect it might cause, then the final LHH = 1/18 x 6 vs. 1/6, or two times as likely to help vs. harm

46. Should these valid, potentially important results change the treatment of our patient? 1. Is our patient so different from those included in the study that its results don’t apply? 2. What is our patient’s risks of the adverse event? What is our patient’s potential benefit from the therapy? 3. What are our patient’s preferences, concerns and expectations from this treatment? 4. What alternative treatments are available?

47. Are the results of this systematic review of therapy valid? 1. Is this a systematic review of randomized trials? 2. Does it include a methods section that describes: (a) finding and including all the relevant trials? (b) assessing their individual validity? 3. Were the results consistent from study to study? (4. Were individual patient data used in the analysis (or aggregate data)? )

48. Is the systematic review important? Are the valid results of this systematic review important? Translating odds ratios to NNTs

49. Are the valid, important results of this systematic review applicable to our patient? 1.Is our patient so different from those in the study that its results cannot apply? 2.Is the treatment feasible in our setting? 3.What are our patient's potential benefits and harms from the therapy? 4.What are our patient's values and preferences for both the outcome we are trying to prevent and the side-effects we may cause?

50. Are the recommendations in this guideline valid? 1. Did its developers carry out a comprehensive, reproducible literature review within the past 12 months? 2. Is each of its recommendations both tagged by the level of evidence upon which it is based and linked to a specific citation?