1 / 31

Changing Trial Designs on the Fly

Changing Trial Designs on the Fly. Janet Wittes Statistics Collaborative ASA/FDA/Industry Workshop September 2003. Context. Trial that is hard to redo Serious aspect of serious disease Orphan. Statistical rules limiting changes. To preserve the Type I error rate

shadow
Download Presentation

Changing Trial Designs on the Fly

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Changing Trial Designs on the Fly Janet WittesStatistics Collaborative ASA/FDA/Industry Workshop September 2003

  2. Context • Trial that is hard to redo • Serious aspect of serious disease • Orphan

  3. Statistical rules limiting changes • To preserve the Type I error rate • To protect study from technical problems arising from operational meddling

  4. Challenge sense rigor

  5. Challenge senseless rigor mortis

  6. Scale of rigor • Over rigid • Rigorous • Prespecified methods for change – preserves  • Unprespecified but reasonable change • Invalid analysis • responders analysis • outcome-outcome analysis • completers

  7. Consequences • No change during the study OR • Potential for the perception that change caused by effect

  8. Prespecified changes • Sequential analysis • Stochastic curtailing • Futility analysis • Internal pilot studies • Adaptive designs • Two-stage designs

  9. Problems • Technical Solved • Operational Risks accepted • Efficiency Understood

  10. Add a DMC • What if it acts inconsistently with guidelines? • Something really unexpected happens? • DMC initiates change • Steering Committee initiates change

  11. Reasons for unanticipated changes • Unexpected high-risk group • Changed standard of care • Statistical method defective • Too few endpoints • Assumptions of trial incorrect • Other

  12. Examples • Too much censoring; DMC extends trial • Boundary not crossed but DMC stops • Unexpected adverse event • Statistical method defective • Event rate too low; DMC changes design

  13. #1 Endpoint-driven trial • Trial designed to stop after 200 deaths • Observations different from expected • Recruitment • Mortality rate • At 200 deaths, fu of many people<2 mo • DMC: change fu to minimum 6 mo • P-value: 0.20 planned; 0.017 at end

  14. #2. Boundary not crossed • Endpoint • Primary: 7 day MI • Secondary: one-year mortality • Very stringent boundary

  15. What DMC sees • Very strong result at 7 days • No problem at 1 year • Clear excess of serious adverse events

  16. Haybittle-Peto bound (10%)

  17. Haybittle-Peto bound (30%)

  18. Haybittle-Peto bound (50%)

  19. Haybittle-Peto bound (70%)

  20. Haybittle-Peto bound (70%)

  21. #3. Unexpected adverse event: PERT study of the WHI • Prespecified boundaries for BenefitHarm Heart attack Stroke Fracture PE Colon cancer Breast cancer

  22. Observations BenefitHarm ----- Stroke Fracture PE Colon cancer Breast cancer Heart attack

  23. Actions • Informed the women about increased risk of stroke, heart attack, and PE • Informed them again • Stopped the study

  24. #4. Statistical method defective • Neurological disease • 20 question instrument • Anticipated about 20% would not come • Planned multiple imputation- results: • Scale: 0 to 80 • Value for ID 001: 30 38 ? 42 28 ? • MI values: -22, 176

  25. #5. Too few endpoints • Example: approved drug • Off-label use associated with AE • Literature: SOC event rate: 20 percent • Non-inferiority design -  = 5 • Sample size: 800/group

  26. Observation • 400 people randomized • 0 events • What does the DMC do?

  27. Choices • Continue to recruit 1600 • Stop and declare no excess • Choose some sample size • Tell the Steering Committee to choose a sample size • What if n=1? 2? 5? 10?

  28. Conclusions • Ensure that DMC understands role • Separate decision-making role of DMC and Steering Committee • Distinguish between reasonable changes on the fly and “cheating” • Expect fuzzy borders

  29. Technical • Changing plans can increase Type I error rate • We need to adjust for multiple looks • How do we adjust for changes?

  30. Operational • Unblind assessments • Subtle change in procedures • In clinical trials, the FDA and SEC

More Related