© 2006 Millennium Pharmaceuticals Inc.

1. Stan Letovsky Senior Director, Computational Sciences Costs and Benefits of Biomarkers in Clinical Trials Washington D.C. September 29, 2006 © 2006 Millennium Pharmaceuticals Inc.

2. Drug Response/Toxicity Biomarkers • Biomarker is a measurement or test on a patient that can predict (with some probability) • Efficacy of a treatment • Toxicity of a treatment • Disease severity (independent of drug) • E.g. Gleevec/BCR-ABL, Iressa/EGFRmut • Drug-specific biomarkers need to be validated in clinical trials to affect approvals. ©2006 Millennium Pharmaceuticals, Inc.

3. ©2006 Millennium Pharmaceuticals, Inc.

4. Question Under what circumstances does it make sense to include a biomarker efficacy hypothesis as part of the main study objectives of a clinical trial? • What are the costs? • Assays, logistics • P-value / sample-size adjustments • What are the benefits? • Increased probability of drug approval ©2006 Millennium Pharmaceuticals, Inc.

5. Possible Trial Designs • Traditional – efficacy only, no biomarker component • Biomarker Discovery – hitchhike on phase 2-3 trial, resulting biomarkers not validated. • Static Biomarker trial – specific biomarker hypotheses tested as part of trial design, could yield validated biomarkers and stratified market. Patient population not biased by biomarker. • Adaptive Validation – a form of adaptive trial in which a biomarker hypothesis is formulated at an interim point. May yield a validated biomarker. No biased sampling. • Adaptive Sampling – a form of adaptive trial in which a biomarker hypothesis is evaluated at an interim point, and subsequent patient selection may be biased by the biomarker. • for Response: Sampling biased towards responding subset / away from adversely-responding subset • for Speed: Sampling biased towards severest disease for faster trial. • for Power: Sampling is biased to allocate more sample to the hypothesis that is most likely to benefit. ©2006 Millennium Pharmaceuticals, Inc.

6. Multiple Comparison Corrections • Study Design#1: • Hypothesis H: “drug not efficacious” • Significance threshold a=.05 • Study Design#2: • Hypothesis H0: “drug not efficacious” • Significance threshold a=.04 • Hypothesis H1: “drug not efficacious in biomarker positive population” • Significance threshold a=.05-.04=.01 ©2006 Millennium Pharmaceuticals, Inc.

7. $$ $ n=Max affordable study size or duration H0 powered at a0 <a : (f0>f , N0) H powered at a: (f,,n) Hi powered at ai : (fi>f0 , Ni=N0*pi) or Min clinically acceptable effect f f0 f1 Power Curves for Static Design (schematic) For a given choice of a (significance) and b (power) get curve of N vs F. 6.8% for a1=.01!! N = Sample Size Adding biomarker hypothesis imposes a multiple comparisons “tax” that must be paid in dollars (by increasing sample size), sensitivity (increasing F) or risk (decreasing power). F = Effect size (e.g. TTP for new drug + SOC / SOC alone) ©2006 Millennium Pharmaceuticals, Inc.

8. F0>=F1*p1 Impossible to be left of blue line F0<F1 Biomarker not predictive on green line, antipredictive below Possible Partial Backfire#2: Apparent success of H0 explained by H1 Trial outcome is a point in the F0,F1 plane f1 Slope of line is biomarker enrichment B1 Must have f1*p1 < f for biomarker strategy to be viable. The steeper the line, the smaller market. f1*p1 f f0 Parameter Space for Static Design Possible Partial Backfire: Reject H1 only, would have rejected H0 w/o biomarker. Market may be stratified; payoff=p1 or 1 vs. 1. Biomarker Win: Reject H1 only: biomarker pays off; stratified market better than none. Payoff=p1 vs. 0 Redundant: Reject both: didn’t need biomarker. Payoff = 1 vs. 1 F1 = mean effect size in biomarker population Biomarker Backfire: Fail to reject H0 but would have if you hadn’t used the biomarker. Total loss of market. Payoff = 0 vs. 1 Biomarker Failure: Reject H0 only; biomarker useless, no harm done. Payoff=1 vs. 1 Drug Failure: Reject none, drug is no good, biomarker didn’t help. Payoff = 0 vs. 0 F0 = mean effect size in entire study population ©2006 Millennium Pharmaceuticals, Inc.

9. Likelihood: Outcomes Are Not Equally Probable Given prior pdf for F0 (e.g. from phase II results, literature) and B1, (made up), can infer (assuming independence) joint distribution of F0 X F1 and pdf of F1. NB: F=T/C. Biomarker Enrichment ©2006 Millennium Pharmaceuticals, Inc.

10. Utility: Outcomes Are Not Equally Valuable X = ©2006 Millennium Pharmaceuticals, Inc.

11. A Biomarker-Favorable Scenario If unlikely to succeed in main trial, but likely in biomarker subpopulation. Better redesign trial? ©2006 Millennium Pharmaceuticals, Inc.

12. Parsing the Parameter Space • Simply by assuming reasonable values of f,f0,f1 and looking at different plausible priors one can learn a lot: • If the F0 prior makes it likely that F0 > f1, there is no need to bother with a biomarker. • If it is likely that F0 > f but it may not be > f1, you may be better off not risking the multiple comparison “tax”. • If there is substantial risk that F0 < f and you have a biomarker with substantial likelihood of significant enrichment, the biomarker strategy may have higher EPV. ©2006 Millennium Pharmaceuticals, Inc.

13. Multiple Comparison Tax Relief • Suppose regulator wants to encourage biomarker validation… • What is consequence of ignoring a=.01 worth of multiple comparison correction to main efficacy hypothesis? • No change to drug approvals in main study population – false positive rate of 5% already deemed societally acceptable. • 1% Probability of false positive “biomarker wins” already deemed acceptable in 4%/1% split. • Assuming something like 10% of biomarkers tested really are predictive, precision=91%, FDR=9%. • Social cost of biomarker backfire avoided ©2006 Millennium Pharmaceuticals, Inc.

14. Adaptive Biomarker Validation Add Biomarker Hypothesis To Trial Design good Initial Unbiased Recruiting Interim Evaluation Of Biomarker No good Continue As Before • Advantages: • Can validate biomarker during phase III • Disadvantages: • Never been done, breaking new regulatory ground • Some complex statistical issues – bias, multiple comparisons… ©2006 Millennium Pharmaceuticals, Inc.

15. E.g. Freidlin and Simon Adaptive Signature Design, Clinical Cancer Research Vol. 11, 7872-7878, Nov 2005 Biomarker-driven Adaptive Sampling Recruit Biomarker Positive Population good Initial Unbiased Recruiting Interim Evaluation Of Biomarker No good Continue Normal Recruiting • Advantages: • Can validate biomarker during phase III • If biomarker works, save money and/or improve chances of approval • Disadvantages: • Never been done, breaking new regulatory ground • Some complex statistical issues – bias, multiple comparisons… ©2006 Millennium Pharmaceuticals, Inc.

16. Uncertainty radius varies inversely with interim sample size Interim outcome gives estimate of final outcome Interim outcome is a point in the F0,F1 plane f1 Adaptive strategy is triggered if interim point falls in a predefined region. Decision analysis optimizes shape of region. Want final point in same (or better) region as interim point. f f0 Parameter Space View of Adaptive Validation ©2006 Millennium Pharmaceuticals, Inc.

17. Conclusions • The requirement of correcting for multiple comparisons has a significant impact on the incentives for including biomarkers in clinical trial designs. • The circumstances under which a cost/benefit analysis favors inclusion of a biomarker hypothesis in the main study objectives may be surprisingly rare. • Adaptive designs combining biomarker discovery, validation and use warrant further investigation. ©2006 Millennium Pharmaceuticals, Inc.

18. Acknowledgements Millennium • Mark Chang • Barb Bryant • Chris Hurff • Bill Trepicchio • Andy Boral FDA (CDER) • Gene Pennello ©2006 Millennium Pharmaceuticals, Inc.

19. SM Breakthrough science. Breakthrough medicine. ©2006 Millennium Pharmaceuticals, Inc.