310 likes | 454 Views
Changing Trial Designs on the Fly. Janet Wittes Statistics Collaborative ASA/FDA/Industry Workshop September 2003. Context. Trial that is hard to redo Serious aspect of serious disease Orphan. Statistical rules limiting changes. To preserve the Type I error rate
E N D
Changing Trial Designs on the Fly Janet WittesStatistics Collaborative ASA/FDA/Industry Workshop September 2003
Context • Trial that is hard to redo • Serious aspect of serious disease • Orphan
Statistical rules limiting changes • To preserve the Type I error rate • To protect study from technical problems arising from operational meddling
Challenge sense rigor
Challenge senseless rigor mortis
Scale of rigor • Over rigid • Rigorous • Prespecified methods for change – preserves • Unprespecified but reasonable change • Invalid analysis • responders analysis • outcome-outcome analysis • completers
Consequences • No change during the study OR • Potential for the perception that change caused by effect
Prespecified changes • Sequential analysis • Stochastic curtailing • Futility analysis • Internal pilot studies • Adaptive designs • Two-stage designs
Problems • Technical Solved • Operational Risks accepted • Efficiency Understood
Add a DMC • What if it acts inconsistently with guidelines? • Something really unexpected happens? • DMC initiates change • Steering Committee initiates change
Reasons for unanticipated changes • Unexpected high-risk group • Changed standard of care • Statistical method defective • Too few endpoints • Assumptions of trial incorrect • Other
Examples • Too much censoring; DMC extends trial • Boundary not crossed but DMC stops • Unexpected adverse event • Statistical method defective • Event rate too low; DMC changes design
#1 Endpoint-driven trial • Trial designed to stop after 200 deaths • Observations different from expected • Recruitment • Mortality rate • At 200 deaths, fu of many people<2 mo • DMC: change fu to minimum 6 mo • P-value: 0.20 planned; 0.017 at end
#2. Boundary not crossed • Endpoint • Primary: 7 day MI • Secondary: one-year mortality • Very stringent boundary
What DMC sees • Very strong result at 7 days • No problem at 1 year • Clear excess of serious adverse events
#3. Unexpected adverse event: PERT study of the WHI • Prespecified boundaries for BenefitHarm Heart attack Stroke Fracture PE Colon cancer Breast cancer
Observations BenefitHarm ----- Stroke Fracture PE Colon cancer Breast cancer Heart attack
Actions • Informed the women about increased risk of stroke, heart attack, and PE • Informed them again • Stopped the study
#4. Statistical method defective • Neurological disease • 20 question instrument • Anticipated about 20% would not come • Planned multiple imputation- results: • Scale: 0 to 80 • Value for ID 001: 30 38 ? 42 28 ? • MI values: -22, 176
#5. Too few endpoints • Example: approved drug • Off-label use associated with AE • Literature: SOC event rate: 20 percent • Non-inferiority design - = 5 • Sample size: 800/group
Observation • 400 people randomized • 0 events • What does the DMC do?
Choices • Continue to recruit 1600 • Stop and declare no excess • Choose some sample size • Tell the Steering Committee to choose a sample size • What if n=1? 2? 5? 10?
Conclusions • Ensure that DMC understands role • Separate decision-making role of DMC and Steering Committee • Distinguish between reasonable changes on the fly and “cheating” • Expect fuzzy borders
Technical • Changing plans can increase Type I error rate • We need to adjust for multiple looks • How do we adjust for changes?
Operational • Unblind assessments • Subtle change in procedures • In clinical trials, the FDA and SEC