Optimal cost-effective Go-No Go decisions

1. Optimal cost-effective Go-No Go decisions Cong Chen*, Ph.D. Robert A. Beckman, M.D. *Director, Merck & Co., Inc. EFSPI, Basel, June 2010

2. Sorry for not being able to attend in person…

3. Outline • Introduction • Benefit-cost ratio analysis of POC design strategies • Discussion • POC strategy and risk mitigation • Phase III futility analysis 3

4. How to fish smartly? Numerous POC possibilities Biology and tech revolution Constraint on societal cost Low success rate and predictability 4

5. Proof-of-concept trial • A randomized double-blinded phase II trial with type I/II error rate (α, β) for detection of Δ based on a surrogate marker • Go to Phase III if p-value <α • Choice of (α, β, Δ) is based on a heuristic argument in practice and is under-explored in statistical literature 5

6. Issues to be addressed • What is a more cost-effective sample size for a POC trial? • What is the optimal bar for a Go decision to Phase III? • How to re-allocate resource when there are more POC trials of similar interest? 6

7. Benefit-cost ratio analysis • Probability of Go if probability of drug truly active in the setting is POS • (1-POS)*α+POS*(1-β) • Expected total sample size (SS) • Phase II SS + Prob(Go)*Phase III SS • Benefit cost ratio • Power of carrying active drug (1-β) to Phase III divided by expected total SS 7

8. Two designs Assumptions Same Δ of interest, e.g., 50% improvement in median progression-free-survival Sample size for Phase III is fixed at 800 once a Go decision is made after POC Two choices of (α, β) (10%, 20%) or a ~160 patient/~110 events trial (10%, 40%) or a ~80 patient trial but higher empirical bar (~0.8Δ vs 0.6Δ) for a Go decision 8

9. Results for comparison 9

10. Optimal designs under fixed Phase II resource

11. Resource optimization • Budgeted for conducting one 160 patient POC trial, but has two POC trials of similar interest • Consensus is that one has higher POS (P1=30%) than the other (P2=20%) • Phase III trial uses same design once Go • Two scenarios for comparison under varying ratio of POC budget (C2)/Phase III cost (C3) assuming sample size is proportional to cost • Two drugs have same value • The one with lower POS has 50% higher value 11

12. Optimal resource split under same value 12

13. Optimal resource split and Go bar under same value 13

14. Optimal resource split and Go bar under different value 14

15. Conclusions • Optimal (α, β) can be easily optimized from benefit-cost ratio analysis • Number of POC trials and respective Go bars depend on Phase II resource, Phase III cost, perceived POS and projected value • Similar analysis reveals that a greater Δ has to be consideredwhen relationship between surrogate marker and OS is less certain • Uncertainty is highest in non-randomized trials! 15

16. POC strategy • More smaller trials, each with a higher Go bar, are generally preferred • Adequately powered for larger Δ of true interest • Similar analysis shows that simultaneous investigation is more cost-effective than sequential investigation

17. Avastin POC strategy 17

18. PFS effect of recently approved innovative drugs 18

19. Pros and cons • Smaller trials • Easier to accrue, faster to complete, and have better quality control • Empirical findings of large treatment effect are more exciting, and help with Phase III accrual • More vulnerable to baseline imbalance • More trials • Reduces missed opportunities (type III error) and increases overall probability of success • May inflate program level type I error rate 19

20. Risk mitigation • Apply minimization or other randomization techniques for better baseline balance • Follow-up patients for survival after primary objective for Phase II is achieved • Initiation of Phase III may be delayed while waiting for Phase II OS data to mature • May revisit a Go or No-Go decision as necessary after OS data become available • Strength of OS data may be used for setting futility bar of Phase III trial as appropriate • Revisit those less promising ones from Phase II after leading indications of same drug achieve major milestones in Phase III 20

21. Futility bar at interim for an ongoing Phase III trial • A hypothetical Phase III trial • Designed to have 90% power for detection of Δ in OS before accounting for any futility analysis • Trial stops for futility at interim if one-sided p-value > α based on survival info of fraction r after 50% of the cost is spent • Benefit = overall power adjusted for futility • May be further adjusted with value as needed • Expected cost = 0.5+0.5*Prob(Go) • where Prob(Go)=(1-POS)*α+POS*(1-β) and β satisfies Zα+Zβ=r1/2(Z0.025+Z0.1)

22. Benefit-cost ratio analysis at 25% info for 30% POS

23. Optimal futility bars