1 / 53

Mark Pletcher 6/10/2011

Quantifying Treatment Effects. Mark Pletcher 6/10/2011. Rationale. Any treatment involves tradeoffs Weigh benefits against risks/costs. Benefit. $$. Harm. Rationale. Sometimes the decision is difficult!. $$. Harm. Benefit. Rationale. And this one?. How big is this box?. $$. Harm.

zinna
Download Presentation

Mark Pletcher 6/10/2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quantifying Treatment Effects Mark Pletcher 6/10/2011

  2. Rationale • Any treatment involves tradeoffs • Weigh benefits against risks/costs Benefit $$ Harm

  3. Rationale • Sometimes the decision is difficult! $$ Harm Benefit

  4. Rationale And this one? How big is this box? $$ Harm Benefit

  5. Rationale • Tests can help us understand who is most likely to benefit from a treatment And this one? How big is this box? $$ Harm Benefit

  6. Rationale • Tests can help us understand who is most likely to benefit from a treatment • Rapid strep to decide who will benefit from penicillin • BNP to decide who will benefit from furosemide • CRP to decide who will benefit from statins

  7. Rationale • The utility of a test depends on: • How beneficial the treatment is • How harmful the treatment is • How much the test tells us about these benefits and harms in a given individual • Risk of harm from the test itself

  8. Rationale • The utility of a test depends on: • How beneficial the treatment is • How harmful the treatment is • How much the test tells us about these benefits and harms in a given individual • Risk of harm from the test itself The topic for this lecture

  9. Outline • Is an intervention really beneficial? • How beneficial is it? • Pitfalls • Examples

  10. Is the intervention beneficial? • Randomized trials compare an outcome in treated to untreated persons • MI in 10% vs. 15% • Duration of flu symptoms 3 vs. 5 days

  11. Is the intervention beneficial? • Randomized trials compare an outcome in treated to untreated persons • MI in 10% vs. 15% • Duration of flu symptoms 3 vs. 5 days • *Statistics* are used to decide if should reject the “null hypothesis” and accept that the intervention is beneficial

  12. Is the intervention beneficial? • But statistics cannot help us interpret effect size

  13. Quantifying the Benefit • Effect size • How do we summarize and communicate this? • What is really important for clinicians and policymakers?

  14. Quantifying the Benefit • Effect size • How do we summarize and communicate this? • What is really important for clinicians and policymakers? • Example: MI in 10% vs. 15% • Q: What do we do with these two numbers?

  15. Quantifying the Benefit • Two simple possibilities: • 10% / 15% = 0.66 • 15% - 10% = 5%

  16. Quantifying the Benefit • Two simple possibilities: • 10% / 15% = 0.66 • 15% - 10% = 5% Relative Risk (RR) Absolute Risk Reduction (ARR)

  17. Quantifying the Benefit • Relative risk as a measure of effect size • RR = 0.66 – is this big or small?

  18. Quantifying the Benefit • Relative risk as a measure of effect size • RR = 0.66 – is this big or small? • MI: 10% vs. 15% in 10 years • Death: 50% vs. 75% in 3 years • Basal Cell CA: 2% vs. 3% in lifetime

  19. Quantifying the Benefit • Relative risk as a measure of effect size • RR = 0.66 – is this big or small? • MI: 10% vs. 15% in 10 years • Death: 50% vs. 75% in 3 years • Basal Cell CA: 2% vs. 3% in lifetime Medium Big Small

  20. Quantifying the Benefit • Relative risk as a measure of effect size • RR = 0.66 – is this big or small? • MI: 10% vs. 15% in 10 years • Death: 50% vs. 75% in 3 years • Basal Cell CA: 2% vs. 3% in lifetime • RR is NOT the best measure of effect size

  21. Quantifying the Benefit • Absolute risk reduction (ARR) is better • ARR = Risk difference = Risk2 – Risk1

  22. Quantifying the Benefit • Absolute risk reduction (ARR) is better RRARR MI: 10% vs. 15% in 10 years .66 5% Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1%

  23. Q: What does the 34% reduction mean?

  24. Nimotop® Ad Graph Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11% 33% 22%

  25. Nimotop® Ad Graph Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11% 33% 22% What is 34%?

  26. Nimotop® Ad Graph Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11% 33% 22% Relative risk reduction (RRR) = 1 – RR = 1-.66 = .34 or 34%

  27. Quantifying the Benefit • RRR is no better than RR RRRRR MI: 10% vs. 15% in 10 years .66 34% Death: 50% vs. 75% in 3 years .66 34% Basal Cell CA: 2% vs. 3% in lifetime .66 34%

  28. Quantifying the Benefit • RRR is ALWAYS bigger than ARR • (unless untreated risk is 100%)

  29. Quantifying the Benefit • BEWARE of risk reduction language!! • ARR or RRR? • “We reduced risk by 34%” • “Risk was 34% lower”

  30. Quantifying the Benefit • BEWARE of risk reduction language!! • ARR or RRR? • “We reduced risk by 34%” can’t tell • “Risk was 34% lower” can’t tell • Very hard to be unambiguous!

  31. Quantifying the Benefit • Another reason that ARR is better: • Translate it into “Number Needed to Treat” • NNT = 1/ARR

  32. Why is NNT = 1/ARR? 100 SAH patients treated 67 no stroke anyway 33 strokes with no treatment R2 11 strokes prevented R1 22 strokes with with treatment 22 strokes with Nimotop®

  33. Why is NNT 1/ARR? Treat 100 SAH patients  prevent 11 strokes Ratio manipulation: 100 treated 1 treated 9.1 treated 11 prevented .11 prevented 1 prevented = =

  34. Why is NNT 1/ARR? Treat 100 SAH patients  prevent 11 strokes Ratio manipulation: 100 treated 1 treated 9.1 treated 11 prevented .11 prevented 1 prevented = = 1/ARR = NNT

  35. Why is NNT 1/ARR? NNT best expressed in a sentence: “Need to treat 9.1 persons with SAH using nimodipine to prevent 1 cerebral infarction”

  36. Quantifying the Benefit • NNT calculation practice RRARRNNT? MI: 10% vs. 15% in 10 years .66 5% Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1%

  37. Quantifying the Benefit • NNT calculation practice RRARRNNT? MI: 10% vs. 15% in 10 years .66 5% 20 Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1%

  38. Quantifying the Benefit • NNT calculation practice RRARRNNT? MI: 10% vs. 15% in 10 years .66 5% 20 Death: 50% vs. 75% in 3 years .66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime .66 1%

  39. Quantifying the Benefit • NNT calculation practice RRARRNNT? MI: 10% vs. 15% in 10 years .66 5% 20 Death: 50% vs. 75% in 3 years .66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime .66 1% 100

  40. Quantifying the Benefit • NNT expression practice RRARRNNT? MI: 10% vs. 15% in 10 years .66 5% 20 Death: 50% vs. 75% in 3 years .66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime .66 1% 100 Statins Chemo Sunscreen every day

  41. Quantifying the Benefit • NNT expression practice “Need to treat 20 patients with statins for 10 years to prevent 1 MI” “Need to treat 4 patients with chemo for 3 years to prevent 1 death” “Need to treat 100 patients with sunscreen every day for their whole life to prevent 1 basal cell”

  42. Example 1 • Randomized controlled trial of the effects of hip replacement vs. screws on re-operation in elderly patients with displaced hip fractures. Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

  43. Example 1 Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

  44. Example 1

  45. Example 1 RR = R1/R2 = 5.2% / 39.8% = .13 RRR = 1-RR = 1-.13 = 87% ARR = R2 – R1 = 39.8% - 5.2% = 34.6% NNT = 1/ARR = 1/.346 = 3 “Need to treat 3 patients with hip replacement instead of screws to prevent 1 from needing a re-do operation”

  46. Example 2 • JUPITER: Randomized controlled trial of high dose rosuvastatin in patients with LDL<130 and CRP>2.0 Ridker et al. NEJM 2008; 359:2195-207

  47. Example 2 Ridker et al. NEJM 2008; 359:2195-207

  48. Example 2 Ridker et al. NEJM 2008; 359:2195-207

  49. Example 2 HR = (R1/R2) (from regression) = .56 RRR = 1-HR = 1-.56 = 44% ARR = R2 – R1 = 1.36 - 0.77 = .59 / 100py* = .0059 / py NNT = 1/ARR = 1/.0059 = 100/.59 = 169 pys “Need to treat 169 patients for a year to prevent 1 CVD event” Or better: “Need to treat 85 patients for 2 years to prevent 1 CVD event” (average treatment duration in trial was 1.9 years) * py = person-years

  50. Example 4 Warfarin vs. placebo for atrial fibrillation WarfarinPlacebo Risk of major bleed (/yr) 1.2% 0.7% Ann Intern Med 1999; 131:492-501

More Related