860 likes | 1.09k Views
Understanding concepts of Evidenced Based Medicine. Frank J. Domino, M.D. Professor University of Massachusetts Medical School. Learning Objectives by the end of the session, you will. Appreciate the basic statistical concepts involved in EBM
E N D
Understanding concepts of Evidenced Based Medicine Frank J. Domino, M.D. Professor University of Massachusetts Medical School
Learning Objectivesby the end of the session, you will Appreciate the basic statistical concepts involved in EBM Contrast absolute risk reduction (ARR) with relative risk reduction (RRR) and Calculate the number needed treat (NNT); and Understand how sometimes statistical interpretation can leads us astray….
Outline of Talk • Types of Papers in the Medical Literature • Review vs. Systematic Review • RCT, Cohort, Case-Control Studies • Bias • Statistics: AR, RR, RRR, NNT • Patient vs Disease Oriented Evidence
What is Evidence-Based Medicine? Integration of Best research evidence (from Systematic Reviews) Clinical expertise Patient values Patient Oriented Outcomes [POE] vs Disease/Intermetdiate Outcomes [DOE] Research Evidence Patient Values Clinical Expertise 4 Sacket et al “How to Teach and Practice EBM”, Churchill Livingston, 2000 Len
Types of Research Information Review Articles Vs Systematic Reviews
Traditional Review Article - Summary of the literature Often valuable reviews for medical practice Problems: • Do not necessarily include all relevant evidence • Author bias mixed with the evidence • Publisher’s motives in ?
A Systematic Review Article– Utilizes quality standards to judge the literature Clear objective for evaluation of study Identify studies (Randomized Controlled Trials) meeting review criteria The results of acceptable studies combined Outcome of those studies published If Quantitative, combined -> Meta Analysis
Systematic Review Systematic Review Meta Analysis
Antibiotics and Acute SinusitisA Cochrane Systematic Reviewwww.cochrane.org 1. Objectives: ‘To determine whether antibiotics are indicated for acute sinusitis, and if so, which antibiotic classes are most effective.’ 2. Search strategy: Studies identified via searches of MEDLINE & EMABASE, contacts w/ pharmaceutical companies and bibliographies of included studies.
3. Selection criteria Randomized controlled trials n > 30 adults. Compare antibiotic to control or other Abx. DX confirmed by radiograph or aspiration. Outcomes: cure or symptom improvement. Of 2058 potentially relevant studies, only49 studies (13,660 pts) met review criteria! > 2000 of questionable significance……..
4. Data Combined Reviewers' conclusions: • For acute maxillary sinusitis, current evidence is limitedbut supports penicillin or amoxicillin for 7 to 14 days. • Clinicians should weigh the moderate benefits of antibiotic treatment against the potential for adverse effects. The Cochrane Library, Oxford
Estimates with 95% confidence intervals Line of no effect Kennedy 1997 Locke 1952A Estimate and confidence interval for each study Lopes 1997 Reynolds 1998 Estimate and confidence for the meta-analysis Seiberth 1994 Scale (effect measure) 0.2 1.0 5 Risk ratio Favours LR Favours control Direction of effect The Forest Plot Forest Plot: If <> to your Left, Intervention was Effective at Lowering Risk of Outcome. Len
Study Designs The Systematic Reviews use: 1. Randomized Controlled Trials Other Study types: 2. Cohort Studies 3. Case Control Studies
Randomized Controlled Clinical Trials A population is chosen, then randomly assigned to an intervention or not An intervention is given to the study population. The Non-Intervention group receives a placebo or some standard of care. Differences in pre identified outcomes are measured and produce ABSOLUTE RISK of those outcomes
Randomized Controlled Trials Bias in the Medical Literature Attrition bias – how were “drop outs” accounted for Publication bias – only positive results get published Comparator bias - new Tx compared to placebo rather than current standard Commercial Bias – who funded the study and what motivated the researchers
Attrition Bias What happened to everyone who was randomized? Was the study “Intention to Treat”? May need to review the data to determine if they “add up” 17 Bob
Publication BiasFactors that prevent publication40% of All RCT not published RCT of using 400 IU/day of Vitamin E to prevent Coronary Artery Disease. Intervention Placebo MI 4.2 3.9 p=0.7 No benefit to supplementing diet with Vitamin E in the prevention of CAD 18 Bob
Comparator BiasComparing new treatment to no treatment, rather than the current standard of care “Azithromycin is superior to placebo in the treatment of Acute Sinusitis” That is nice, but is it superior to Amoxicillin? Don’t know; they didn’t do (or won’t publish) that study. 19 Bob
Commercial Bias $$ Look for Disclosures of the Authors Who funded the study Bottom of The Front page and just before Reference Section Ex. JUPITER Study 20 BIAS: Assume it is Present Bob
P < 0.001 Statistically significant! AR MI placebo - 0.76% AR MI Crestor - 0.35%
Cohort Studies • Prospective, Observationalstudies with conclusions • Produces aRelative Risk (RR) RR=Incidence of disease in Exposed divided by Incidence of disease in Unexposed population Ex: Framingham Heart study
Relative Risk (RR) If < 1.0, risk is REDUCED If > 1.0, risk is INCREASED RR = Number of times more or less than 1 an event will happen in one group when compared to another
“The RR of death if involved in an MVA without a seat belt = 3.5” “Tea drinkers have a 0.6 RR of dying from CAD”
Correlationdoes not prove cause and effect! Does tea drinking prevent CAD???
Vitamin E & CAD 3 cohort studies mid-90’s concluded vitamin E use correlated with a lower risk for CAD. Antioxidant Vitamins and Coronary Heart Disease Vitamin E Consumption and Risk of Coronary Disease in Men Vitamin E Consumption and Risk of CAD in Women
The Heart Outcomes PreventionEvaluation (HOPE) Study 2000:Controlled Trial of Vitamin E for 5 Yr: rates of MI, CVA & CV death did not differ significantly from placebo (16.2% v 15.5 %). HOPE II (JAMA 2005) Long-term use of vit E in CHD or DM patients does not prevent cancer or major cardiovascular events and may increase the risk for HF
Case Control Studies • Identify a Potential trend in disease • Collect exposure history of “CASES” • Identify similar people age/gender like Cases but without disease • Compare Cases to Controls to determine if exposure increased ODDS of disease. Ex: Cigarette smoking
Outline of Talk • Types of Papers in the Medical Literature • Review vs. Systematic Review • RCT, Cohort, Case-Control Studies • Bias • Statistics: AR, RR, RRR, NNT • Patient vs Disease Oriented Evidence
Which drug would you take? A B C Drug A can reduce your MI risk by 1/3 Drug B can reduce your MI risk by 9 % Drug C every 11 patients who take Drug C, one MI will be prevented
The Scandinavian Simvastatin Survival Study (4S) 4444 pts with angina or previous MI and ↑ cholesterol Randomized to simvastatin or placebo. After 5 yrs, simvastatin reduced TC 25%, LDL 35% and increased HDL 8%. [DOE] MI or death: 622(28%) control group (placebo) 431pts (19%) simvastatin group [POE]
Disease Oriented vs. Patient Oriented Evidence DOE Lidocaine ↓ V. Tach HRT will ↓ LDL; ↑ HDL Fluoride ↑ bone density POE Prophylactic Lido ↑ CV death Increased risk of stroke & Mortality Does not prevent fractures
Absolute Risk Reduction (ARR) 4S Trial: The difference between the incidence of outcome in the control group (28%) and the incidence in the treatment group (19%). ARR = Incid Control – Incid Treatment 28%- 19% = 9% ARR
Relative Risk Reduction Absolute Risk Reduction (ARR) divided by Incidence in the control group RRR = (28-19)/28 = 33% RRR is not the same as AR or RR
Relative Risk Reduction A way to describe (and often over inflate) the relative impact of a treatment on an outcome 35 Bob
Number Needed to Treat (NNT) • NNT - the number of people needed to receive an intervention before one person gets the expected outcome! • NNT = 100 divided by Absolute Risk Reduction. NNT = 100/(28-19) = 11 11 pts w/ CAD need to be treated with simvastatin to prevent 1 from having a subsequent MI or death.
Which drug would you take? • Drug A can reduce your MI risk by 1/3rd • Relative Risk Reduction (RRR = 33%) • Drug B can reduce your MI risk by 9% • Absolute Risk Reduction (ARR = 9 %) • Drug C prevent an MI for every 11 patients who take it regularly • Number Needed to Treat (NNT = 11)
Another example of NNT:Does alendronate prevent hip fractures in postmenopausal women ? The published study: Low bone mass density One or more fractures at baseline Alendronate 5 mg/d x 24 mo, then 10mg/d Outcome - subsequent hip fractures
Outcomes P < 0.044 Statistically significant! Statistical Significant does NOT equal Clinical Significant
Statistical Significance An indication that the findings are not due to chance! P < 0.05 means there is < 5% chance the difference between the placebo and treated group is a chance occurrence. It is not an implication of clinical meaning (significance).
A Closer Look… Hip fracture rate in treated group - 1% Hip fracture rate in control group - 2.2%
Absolute Risk Reduction Placebo incidence – Treated incidence 2.2% - 1% = 1.2% A high risk patient can reduce her risk of a hip fracture by 1.2% from alendronate x 4 yrs
What happens to the other 82 women? • They receive the medication, incurring the cost of treatment along with the exposure to the potential side effects, but….. obtained no identified benefit!
What is this 56% ??? Relative Risk Reduction 2.2-1(ARR)/2.2 = 56%
Number Needed to Treat 100/ARR = 100/(2.2-1) = 83 83 high risk women would have to be treated with alendronate for 3 years to prevent one additional hip fracture • Evidence-Based Medicine
What’s a “good” NNT ? Depends on risks of intervention vs outcome The best NNT would be 1 - every treated patient benefited, but no placebo benefit NNTs < 5 indicate very effective treatments
Good NNT NNT is a VALUE BASED decision NNTs of 50 or 100 useful for interventions to reduce death after heart attack. NNS in the 1000’s as screened population includes those with and without disease. Decisions should also consider costs and risk of the intervention (NNH)
Some NNTs 25 unstable angina 700 mild HTN Anti-HTN X 1yr 15 severe HTN ACEi 18 CHF/post-MI 3500 strep throats Penicillin Ca/Vit D X 3yr 30 ambulatory♀