Introduction to Evidence Based Medicine

1. Introduction to Evidence Based Medicine Bradford S. Pontz, M.D. Assistant Professor of Medicine Georgetown University Medical Center

2. Patient W.R. • 55 year old healthy male presents with 3 days right-sided back pain, 2 days rash • Exam shows vesicular lesions • Patient says his friend was given a medication to treat the shingles and prednisone and wants to know if this will make his experience less miserable

3. Overview and Objectives • 1. Definition • 2. How to ask Clinical Questions you can answer • 3. Searching for the Best Evidence • 4. Critically Appraising the Evidence • 5. Applying Evidence

4. Definition • Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients

5. Definition • Making the best decision requires sound judgment based on the following: • clinical expertise • knowledge of patient values and preferences • evidence from the literature

6. How to Ask Clinical Questions you can Answer - Four Elements of well-built clinical questions • 1. Patient or problem - ask “how would I describe a group of patients similar to mine?” compromise between precision and brevity • 2. Intervention - ask “what main intervention am I considering?” usually a treatment or exposure

7. Four Elements of well-built clinical questions • 3. Comparison intervention - usually vs. placebo or vs. established therapy • 4. Outcome - what could this treatment hope to accomplish or what could result from this exposure?

8. Search StrategiesAND/OR • AND ing searches for intersection of searches (studies that contain all search words) • OR ing searches for union of searches (simply adds them, but studies duplicated will appear only once in result of your OR)

9. Search Strategies - Medline • Enter topic as subject heading (check box labelled “Map Term to Subject Heading”) • Enter any limits (years, language, etc.) • “Explode” topic if necessary • Next enter topic as index text word (do not check box) • Next enter topic as “topic.af” (all fields) • Then, combine these three searches using OR (creates union of first three searches)

10. Search Strategies - Medline • Do this for as many aspects (in terms of search words) as seems necessary to limit search • Finally AND the results of these searches for individual topics

11. Search Strategies - Medline • Methodologic filter is a way to further refine your search • Searches for all studies that involve the parameter you want (often a large number) • In Medline, search for your parameter with a suffix

12. Search Strategies - Medline • Randomized controlled trial.pt (publication type) • Random.tw (text word) • Drug Therapy.sh (subject heading)

13. Critically Appraising the Evidence • Hierarchy of Types of Studies • A few general terms • Assessing validity and importance of three main types of studies • diagnosis • prognosis • treatment

14. Hierarchy of Types of Studies(in decreasing order of preference) • 1.Systematic reviews and meta-analyses • 2.Randomized, controlled clinical trials with definitive, significant results • 3.Randomized, controlled clinical trials with less definitive results (a point estimate suggesting a clinically significant effect, but with confidence intervals that suggest the possibility of more equivocal results)

15. Hierarchy of Types of Studies(in decreasing order of preference) • 4. Cohort studies • 5. Case-Control studies • 6. Case reports

16. General Terms • Hypothesis: “Purpose is to examine Treatment X in lowering blood pressure compared to standard Treatment Y” • Null hypothesis is no difference

17. General Terms • P value is a level of probability, deemed as statistically significant, chosen as grounds for rejecting the null hypothesis. • Traditionally p<.05 = less than 5% probability that difference between treatments is due to chance or unknown reason rather than true difference in treatments

18. General Terms • Validity - external validity is the degree to which the results of a study hold true in other settings and/or apply to populations beyond those included in a study, such as your own patients. Consider possible differences: • gender - compliance • stage or severity of disease

19. Primary End Points • Most directly and clearly portray the actual condition of interest. Should state how possibility of end point will be assessed: • cancer prevalence by biopsy • coronary artery disease by angiogram • peptic ulcer evaluated by endoscopy

20. Is this evidence about a diagnostic test valid? • Was there an independent, blind comparison with a reference “gold” standard of diagnosis? • Was it evaluated in an appropriate spectrum of patients?

21. Is this evidence about prognosis valid? • Was a defined, representative sample of patients assembled at a common (usually early) point in the course of their disease? • Was follow-up sufficiently long and complete?

22. If Follow-up not Optimal, do a “Worst-Case” Analysis • 100 patient enter, 4 die, 16 lost to follow-up • Death rate = 4/(100-16) = 4/84 = 4.8% • Survival rate = 100%-4.8% = 95.2% • Worst case - What if all 16 lost died? • Death rate=(4+16)/(84+16) = 20/100 = 20% • Survival rate = 100%-20% = 80%

23. Is this evidence about treatment valid? • Was the assignment of patients randomized and double-blind? • Were the groups similar at the start? • Apart from the experimental intervention, were the groups treated equally? • Were all accounted for at the end of the trial and analyzed in the groups to which they were randomized? (“intention to treat”)

24. No randomized trials found? • Refine your search • Consider whether a treatment effect is so large you can’t imagine it would be false(+) • Evidence from non-randomized trial showing that treatment is useless or harmful is somewhat acceptable. False (-) conclusions less likely than false (+)

25. Is evidence from a systematic review valid? • Are the trials randomized? • Were the results consistent from study to study? • Does it include a methods section that describes: • search methods • methods for assessing individual study validity

26. Critically Appraising the Evidence • Hierarchy of Types of Studies • A few general terms • Assessing validity and importance of three main types of studies • diagnosis • prognosis • treatment

27. Is evidence about a diagnostic test important?

29. Terms about diagnotic tests • Sensitivity = a/(a+c) = 731/809 = 90% • Specificity = d/(b+d) = 1500/1770 = 85% • LR+ = sens/(1-spec) = 90%/15% = 6 • LR- = (1-sens)/spec = 10%/85% = 0.12 • pos. pred. Value = a/(a+b) = 731/1001 = 73% • neg. pred. Value = d/(c+d) = 1500/1578 = 95%

30. Terms about diagnostic testsLikelihood Ratios • Definition - probability of that test result in people with the disease divided by the probability of the result in people without the disease • Can be calculated for a range of values of test results rather than just pos. vs. neg. • Can be used with pre-test odds to calculate post-test odds

31. Terms about diagnostic tests • Prevalence = (a+c)/(a+b+c+d) = 809/2579 = 32% • Pretest odds = prevalence/(1-prevalence) = 31%/69% = 0.45 • Post-test odds = pretest odds X LR • Post-test probability = post-test odds/(post-test odds + 1)

32. Example - Ferritin for Diagnosis of iron deficiency anemia • Assume a pre-test odds of 1:1 (a 50-50 chance) • Suppose Ferritin = 60 • Post-test odds = 1X6 = 6 • Post-test probability = 6/(6+1) = 6/7 = 86%

34. Is evidence about treatment important? • A statistically significant result (e.g. p<.05) may not be clinically significant. • May show that one treatment is better than another, but does not necessarily suggest the impact that treatment might have in your own clinical practice

35. Bottom Line Clinical Effects • Relative Risk (RR) • Relative Risk Reduction (RRR) • Absolute Risk Reduction (ARR) • Number Needed to Treat (NNT) • Confidence Intervals

37. Basic Statistics • CER = Control Event Rate = risk of outcome event of interest in the control group = A/(A+B) • EER = Experimental Event Rate = risk of outcome event rate in the experimental group = C/(C+D)

38. Relative Risk • Aka Risk Ratio • is the ratio of risk of the outcome event in the experimental (intervention or treated group) to the risk in control group • RR = EER/CER = [C/(C+D)]/[A/(A+B)] • RR = [350/(350+947)]/[404/(404+921)] • RR = 0.865 or about 87%

39. Relative Risk Reduction • Essentially the complement of RR • The percent reduction in the experimental group event rate compared with the control group event rate • RRR = [(CER-EER)/CER] X 100 OR • RRR = 1-RR • RRR = (1-0.865) = 13.5%

40. Absolute Risk Reduction • Aka Risk Difference = difference in the event rate between a control group and an experimental group • ARR = CER-EER • ARR = A/(A+B) - C/(C+D) • ARR = 404/(404+921) - 350/(350+947) • ARR = 0.041 or 4.1%

41. Number Needed to Treat • NNT = 1/ARR • NNT = 1/0.041 = 24 • NNT is particularly useful to clinicians who want to know whether the probable benefits of some treatments or intervention will be worthwhile in their patients

42. Confidence Intervals • Basic research concept - experiment repeated will yield slightly different results each time • Approximation of the true effect is called the point estimate • CI = larger neighborhood in which true effect is likely to reside

43. Confidence Intervals • Expressed with a given degree of expected certainty such as 95% • True result will lie outside the range only 5% of the time (2.5% of the time above and 2.5% of time below) • For example, an absolute risk reduction of 4.1% could have 95% CI of -1.0 to 9.2

44. Hypothetical treatment study • Suppose experimental group is group 1. 15 of 125 patients have a given outcome. Proportion of outcome is p1=r1/n1=15/125=12%. • Suppose control group is group 2. 30 of 120 patients have a given outcome. Proportion of given outcome is p2=r2/n2=30/120=25%.

46. Can you apply a diagnostic test? • Is it affordable, available, accurate and precise in your setting? • Can you estimate pretest probability? • Data from personal or practice experience • Data from the report itself - Speculation • Will the resulting post-test probabilities affect your management? Would you treat based on results? Would patient agree to?

47. Applying Treatment Evidence • Can you apply evidence about prognosis to your patient? • Can you apply evidence about treatment? • Is your patient so different from those in trial that its results cannot be applied? Usually can extrapolate at least a direction of effect • How great a benefit might you expect from treatment?

48. N of 1 Trial • Problems with classic trial of therapy • Many illnesses or lab abnormalities are self-limited • Placebo effect can lead to improvement in symptoms • Conclusions can be biased by our own expectations and those of the patient

49. N of 1 Trial • Clinician and patient agree to trial • Patient undergoes pairs of treatment periods • Both patient and clinician are blinded • Treatment targets are monitored (symptom diary, etc.) • Pairs of treatment periods are replicated until both pt. and clinician are convinced that treatments are really different or not

50. Patient W.R. • 1. Patient group/problem • 2. Intervention • 3. Comparison • 4. Outcome • Question: Does prednisone in addition to standard antiviral therapy, compared to standard antiviral therapy alone, improve pain in immunocompetent patients with acute herpes zoster?