1 / 95

Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Treatment Thresholds

Using Randomized Trials to Quantify Treatment Effects. Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Treatment Thresholds Revisited. 6 November 2008 Michael A. Kohn, MD, MPP.

meryle
Download Presentation

Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Treatment Thresholds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Randomized Trials to Quantify Treatment Effects Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Treatment Thresholds Revisited 6 November 2008 Michael A. Kohn, MD, MPP

  2. EBM is about using research studies to help in two related areas: Diagnosis: Evaluate a test and then use it to determine whether a patient has a given disease. (Chs. 2, 3, 4, 5, 8) Treatment: Determine if a treatment is beneficial in patients with a given disease, and if so, whether the benefits outweigh the costs and risks. (Chs. 9, 10) In screening programs (Ch. 6), diagnosis and treatment are the most closely intertwined. Prognostic testing (Ch. 7) requires longitudinal studies and evaluation of calibration as well as discrimination.

  3. EBM is about using research studies to help in two related areas: Diagnosis: Evaluate a test and then use it to determine whether a patient has a given disease. (Chs. 2, 3, 4, 5, 8) Treatment: Determine if a treatment is beneficial in patients with a given disease, and if so, whether the benefits outweigh the costs and risks. (Chs. 9, 10) In screening programs (Ch. 6), diagnosis and treatment are the most closely intertwined. Prognostic testing (Ch. 7) requires longitudinal studies and evaluation of calibration as well as discrimination.

  4. Quantifying the Benefit of a Treatment: Take Home Points • RCT Checklist • Need baseline incidence of bad outcome*. • Number Needed to Treat =NNT= 1/ARR • Number Needed to (treat to) Harm = NNH = 1/ARI • Back-of-the-envelope CEA: Treatment cost per bad outcome prevented = Treatment Cost x NNT *Unless the RR is 1 and RRR is 0.

  5. RCT Checklist

  6. RCT Checklist for Study Validity* Design and conduct • Randomization to address issues of confounding • Blinding of patients and clinicians to prevent differential co-interventions • Blinding of outcome assessors to prevent bias • Patient-Oriented Effect Measures (POEMs) vs. surrogate outcomes • Decompose composite outcomes • Good follow-up to eliminate differential losses to follow-up *For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

  7. RCT Checklist for Study Validity* Analysis • Intention-to-treat analysis (once randomized always analyzed) • Compare entire randomization groups, not subgroups • Between groups rather than within groups comparison *For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

  8. Randomization, Intention-to-Treat Analysis, and Follow-up: Hip Replacement vs. Screws • Pt is a 81-year-old woman with a hip fracture • Pt’s son is a physician. He asks about hip replacement vs. screws. Pubmed search  Parker MJ, Khan RJ, Crawford J, Pryor GA. Hemiarthroplasty versus internal fixation for displaced intracapsular hip fractures in the elderly. A randomised trial of 455 patients. J Bone Joint Surg Br. Nov 2002;84(8):1150-1155.

  9. Displaced Femoral Neck Fracture = Hip Fracture

  10. Internal Fixation = Screws

  11. Hemiarthroplasty = Hip Replacement

  12. Randomization, Intention-to-Treat Analysis, and Follow-up: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Randomized controlled trial of the effects of hip replacement vs. screws on re-operation and other outcomes in > 70-year-old patients with displaced, hip fractures.

  13. Randomization: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Why do a randomized experiment? Why not do an observational study comparing mortality, re-operation rates, etc. between patients who had hip replacements and patients who had screws?

  14. Intention-to-Treat: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Some patients randomized to the hip replacement group ended up getting screws. Why not include these patients’ outcomes in the screws group or at least exclude them from the hip replacement group?

  15. Losses to Follow-Up: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.* If each treatment group had 20% loss to follow-up, there could still be bias. What if those in the screws group were lost to follow-up because they got better and those in the hip replacement group were lost because they died? *In fact, there were no losses to follow-up in this study.

  16. Patient Oriented Endpoints, Blinding • Pt is a 34-year-old man who dislocated his shoulder while surfing at Ocean Beach. • He asks about early arthroscopic stabilization versus immobilization and PT. Pubmed search  Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514

  17. Endpoints, Blinding: Arthroscopy vs. immobilization for 1st shoulder dislocation Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514.

  18. Outcomes Affected by Treatments* • Dichotomous (e.g. recurrent dislocation) • Continuous (e.g. WOSI**) Endpoints • Patient relevant (e.g., ability to return to sports) • Surrogate (e.g., MRI findings) * Example: Arthroscopy vs. conservative tx for 1st Anterior Shoulder Dislocation (Arthroscopy. 1999 Jul-Aug;15(5):507-14. ) **Western Ontario Shoulder Disability Index

  19. Outcomes Affected by Treatments • Dichotomous (e.g. recurrent dislocation) • Continuous (e.g. WOSI) Endpoints • Patient relevant (e.g., ability to return to sports) • Surrogate (e.g., MRI findings)

  20. Blinding • Blinding of Patients and Clinicians • Eliminates differential co-interventions • Blinding of Outcome Assessment • Eliminates biased outcome assessment

  21. Blinding Blinding less important when opportunity for cointerventions that affect outcomes is minimal, and outcome is not subjective. • Hip Replacement vs Screws for hip fracture, with endpoints of mortality and re-operation: patients, clinicians, and outcome assessors not blinded. • Arthroscopy vs. non-operative management of shoulder dislocation, with endpoints of re-dislocation, and WOSI*: patients not blinded, but clinicians and outcome assessors (therapists) were blinded. *Western Ontario Shoulder Disability Index **Moseley JB, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002;347(2):81-88.

  22. Between-groups Comparison Nissen SE, Tsunoda T, Tuzcu EM, Schoenhagen P, Cooper CJ, Yasin M, et al. Effect of recombinant ApoA-I Milano on coronary atherosclerosis in patients with acute coronary syndromes: a randomized controlled trial. Jama 2003;290(17):2292-2300.

  23. Sub-group Analysis: ISIS II* 30-day mortality *Lancet 1988;2(8607):349-360.

  24. Sub-group Analysis: ISIS II* 30-day mortality *Lancet 1988;2(8607):349-360.

  25. Composite Endpoints Irradiation to prevent re-blockage after cleaning out a blocked coronary artery bypass graft. “At 12 months, … the rate of major cardiac events was 49 percent lower (32 percent vs. 63 percent, P<0.001). “ Waksman, R., A. E. Ajani, et al. (2002). "Intravascular gamma radiation for in-stent restenosis in saphenous-vein bypass grafts." N Engl J Med346(16): 1194-9.

  26. Composite Endpoints “Major Cardiac Event” = Death or MI or Revascularization Procedure Irradiation Group: 19 “Events” 4 deaths, 1 MI, 17 Revasc Proc Placebo Group: 38 “Events” 4 deaths, 2 MI, 37 Revasc Proc

  27. Composite Endpoints Could have been: Irradiation Group: 19 “Events” 4 deaths, 1 MI, 17 Revasc Proc Placebo Group: 38 “Events” 0 deaths, 1 MI, 37 Revasc Proc

  28. DONE: RCT Checklist for Study Validity* • Randomization to address issues of confounding • Blinding of patients and clinicians to prevent differential co-interventions • Blinding of outcome assessors to prevent bias • Patient-Oriented Effect Measures (POEMs) vs. surrogate outcomes • Take care with composite outcomes • Good follow-up to eliminate differential losses to follow-up • Intention-to-treat analysis (once randomized always analyzed) • Between groups rather than within groups comparison • Compare entire randomization groups, not subgroups *For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

  29. RCTs • Parker MJ, Khan RJ, Crawford J, Pryor GA. Hemiarthroplasty versus internal fixation for displaced intracapsular hip fractures in the elderly. A randomised trial of 455 patients. J Bone Joint Surg Br. Nov 2002;84(8):1150-1155. • Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514 • ISIS-2 (1988). "Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group." Lancet2(8607): 349-60. 4. Waksman, R., A. E. Ajani, et al. (2002). "Intravascular gamma radiation for in-stent restenosis in saphenous-vein bypass grafts." N Engl J Med346(16): 1194-9.

  30. Effect Size (Dichotomous Outcomes*) RR RRR ARR NNT ARI NNH * Not going to discuss continuous outcomes today

  31. Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. This study was properly randomized but not blinded, used an intention-to-treat analysis, and had NO losses to follow-up. Results follow…

  32. Reduced Re-operation

  33. Measures of Treatment Effect RR= Risk Ratio = RR < 1 means treatment is beneficial RRR = Relative Risk Reduction = 1-RR

  34. Beware of the Odds Ratio RR = Risk Ratio = (a/b) (a/c) OR = Odds Ratio = ------- = -------- = ad/bc (c/d) (b/d) In the hip replacement vs. screws example, the baseline risk of reoperation (with screws) is 40%, so the baseline odds are 67%. The risk (or odds) with replacement is about 5% , so RR ≈ 5/40 ≈ 1/8; but the OR ≈ 5/67 ≈ 1/13.

  35. Atraumatic vs. Standard Spinal Needle and Occurence of Headache Straus, S. E., K. E. Thorpe, et al. (2006). JAMA296(16): 2012-22.

  36. Measures of Treatment Effect ARR = Absolute Risk Reduction = c/(c+d) - a/(a+b) NNT = Number Needed to Treat (to prevent 1 bad outcome) = 1/ARR

  37. Q: What does the 34% reduction mean?

  38. Nimotop® Ad Graph RR = 21.8%/33% = .66 RRR = 1-0.66 = 34% ARR = 33% - 21.8% = 11.2% 33% 22%

  39. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Original figure To scale 11%

  40. Why is NNT = 1/ARR? 100 SAH patients treated 67 no stroke anyway 33 strokes with no treatment 11 strokes prevented 22 strokes with with treatment 22 strokes with Nimotop®

  41. Why is NNT 1/ARR? Treat 100 SAH patients; prevent 11 strokes. 100/11 = 1/11% = 1/ARR = 9 patients treated per stroke prevented.

  42. NNT Practice In patients < 30 years old with first-time acute anterior shoulder dislocation, prompt arthroscopic surgery (vs. standard conservative therapy) reduces the 2-year re-dislocation rate by almost 33% in absolute terms (from about 50% to about 17%).* How many first-time dislocation patients do we need to treat with arthroscopy to prevent one having re-dislocation at 2 years?

  43. NNT Practice ISIS- 2*. Aspirin therapy (one month of 160 mg/day) in patients with acute myocardial infarction (AMI) reduced 30-day cardiovascular mortality from 11.8% in the placebo group to 9.3% in the aspirin group. *Lancet 1988;2(8607):349-360.

  44. NNT Practice How many AMI patients do we need to treat with aspirin to prevent one CV death at 30 days? *Lancet 1988;2(8607):349-360.

  45. NNT Practice RCT of re-insertion of stylette to reduce post-LP headache. Of 300 patients who had stylette re-inserted, only 15 (5%) got headaches. Of 300 patients who did not have stylette re-inserted, 49 (16%) got headaches. How many stylettes do you have to re-insert to prevent one post-LP headache? J. Neurol. 1998 Sep;245(9):589-92

  46. NNT Practice

  47. Problem with the Relative Risk Reduction The risk ratio (RR) or relative risk reduction (RRR = 1-RR) associated with a treatment is of minimal use without knowing the baseline level of risk*. *The RR is not completely useless without the baseline risk. If RR=1, the tx is useless regardless of the baseline risk. If RR << 1, then the treatment is beneficial; if RR >> 1, the treatment is harmful. Also, if you already know the baseline risk in your own population, the RR may be all you need.

  48. Problem with the Relative Risk Reduction Irradiation to prevent re-blockage after cleaning out a blocked coronary artery bypass graft. “At 12 months, the rate of revascularization of the target lesion was 70 percent lower in the iridium-192 group than in the placebo group” After cleaning out blocked bypass graft, how many do we have to treat with iridium to prevent one revascularization procedure (of the target lesion)? Waksman, R., A. E. Ajani, et al. (2002). "Intravascular gamma radiation for in-stent restenosis in saphenous-vein bypass grafts." N Engl J Med346(16): 1194-9.

  49. Problem with the Relative Risk Reduction “At 12 months, the rate of revascularization of the target lesion was 70 percent lower in the iridium-192 group than in the placebo group” The baseline risk (i.e., risk in the placebo group) was 57% After cleaning out blocked bypass graft, how many do we have to treat with iridium to prevent 1 revascularization procedure in the next 12 months? Waksman, R., A. E. Ajani, et al. (2002). "Intravascular gamma radiation for in-stent restenosis in saphenous-vein bypass grafts." N Engl J Med346(16): 1194-9.

More Related