Development of Prognostic and Predictive Biomarkers and their Use in Clinical Trial Design

1. Development of Prognostic and Predictive Biomarkers and their Use in Clinical Trial Design Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov

2. BRB Websitehttp://brb.nci.nih.gov • Powerpoint presentations and audio files • Reprints & Technical Reports • Sample Size Planning for Targeted Clinical Trials • BRB-ArrayTools software • BRB-ArrayTools Data Archive

3. Many cancer treatments benefit only a small proportion of the patients to which they are administered • Advantages of targeting treatment to the right patients • Treated patients benefit • Greater chance of success in clinical development • Societal benefit on health care costs

4. Biomarkers • Prognostic classifiers • Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment • Predictive classifiers • Measured before treatment to select good patient candidates for a particular treatment

5. Biomarkers • Pharmacodynamic endpoint • Measured before and after treatment to monitor effect of treatment • Surrogate endpoints • Measured before and after treatment to determine whether the treatment is providing clinical benefit

6. Surrogate Endpoints • It is extremely difficult to properly validate a biomarker as a surrogate for clinical outcome. • It requires a series of randomized trials with both the candidate biomarker and clinical outcome measured • Pharmacodynamic endpoints need not be validated surrogates of clinical benefit for use in phase I or phase II studies

7. Most prognostic factors are not used because they are not therapeutically relevant • Most prognostic factor studies use a convenience sample of patients for whom tissue is available. Often the patients are too heterogeneous to support therapeutically relevant conclusions • Prognostic factors in focused population can be therapeutically useful • Oncotype DX

8. Validation=Fit for Purpose • FDA terminology of “valid biomarker” and “probable valid biomarker” are not appropriate for prognostic or predictive classifiers • “Validation” should mean fitness for intended purpose and the purpose of prognostic/predictive classifiers are completely different than for surrogate endpoints

9. Types of Validation for Prognostic and Predictive Biomarkers • Analytical validation • When there is a gold standard • Sensitivity, specificity, ppv, npv, ROC • Pre-analytical and post-analytical robustness • Clinical validation • Does the biomarker predict what it’s supposed to predict for independent data • Clinical utility • Does use of the biomarker result in patient benefit

10. Clinical Utility • Benefits patient by improving treatment decisions • Depends on context of use of the biomarker • Treatment options and practice guidelines • Other prognostic factors • With new drug of for guiding use of standard treatments

12. The Roadmap • Develop a completely specified genomic classifier of the patients likely to benefit from a new drug • Pre-clinical, phase I, II studies • Establish analytical validity of the classifier • Use the completely specified classifier to design and analyze a new clinical trial to evaluate effectiveness of the new treatment with a pre-defined analysis plan.

13. Guiding Principle • The data used to develop the classifier must be distinct from the data used to test hypotheses about treatment effect in subsets determined by the classifier • Developmental studies are exploratory • Studies on which treatment effectiveness claims are to be based should be definitive studies that test a treatment hypothesis in a patient population completely pre-specified by the classifier

14. Development of Genomic Classifiers • Single gene or protein based on knowledge of therapeutic target • Empirically determined based on evaluation of a set of candidate classifiers • e.g. EGFR assays • Empirically determined based on genome-wide correlating gene expression or genotype to patient outcome after treatment

15. New Drug Developmental Strategy (I) • Develop a predictive classifier that identifies the patients likely to benefit from the new drug • Technical validation of the test • Use the test to restrict eligibility to a randomized evaluation of the new drug

16. Develop Predictor of Response to New Drug Using phase II data, develop predictor of response to new drug Patient Predicted Responsive Patient Predicted Non-Responsive Off Study New Drug Control

17. Applicability of Design I • Primarily for settings where the classifier is based on a single gene whose protein product is the target of the drug • eg Herceptin • Without strong biological rationale or adequate phase II data on unselected patients, FDA may have difficulty approving the test

18. Evaluating the Efficiency of Strategy (I) • Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004. • Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-339, 2005. • reprints and interactive sample size calculations at http://linus.nci.nih.gov/brb

19. Two Clinical Trial Designs • Un-targeted design • Randomized comparison of T to C without screening for expression of molecular target • Targeted design • Assay patients for expression of target • Randomize only patients expressing target

20. Efficiency relative to trial of unselected patients depends on proportion of patients test positive, and effectiveness of drug (compared to control) for test negative patients • When less than half of patients are test positive and the drug has little or no benefit for test negative patients, the targeted design requires dramatically fewer randomized patients

21. No treatment Benefit for Test - Patientsnstd / ntargeted

22. Treatment Benefit for Test – Pts Half that of Test + Pts nstd / ntargeted

23. Trastuzumab • Metastatic breast cancer • 234 randomized patients per arm • 90% power for 13.5% improvement in 1-year survival over 67% baseline at 2-sided .05 level • If benefit were limited to the 25% assay + patients, overall improvement in survival would have been 3.375% • 4025 patients/arm would have been required • If assay – patients benefited half as much, 627 patients per arm would have been required

24. Web Based Software for Comparing Sample Size Requirements • http://brb.nci.nih.gov

29. Develop Predictor of Response to New Rx Predicted Responsive To New Rx Predicted Non-responsive to New Rx New RX Control New RX Control Developmental Strategy (II)

30. Developmental Strategy (II) • Do not use the diagnostic to restrict eligibility, but to structure a prospective analysis plan • Having a prospective analysis plan is essential; “stratifying” (balancing) the randomization ensures that all randomized patients will have tissue available • The purpose of the study is to evaluate the new treatment overall and for the pre-defined subsets; not to modify or refine the classifier • The purpose is not to demonstrate that repeating the classifier development process on independent data results in the same classifier

31. Analysis Plan A • Compare the new drug to the control for classifier positive patients • If p+>0.05 make no claim of effectiveness • If p+ 0.05 claim effectiveness for the classifier positive patients and • Compare new drug to control for classifier negative patients using 0.05 threshold of significance

32. Sample size for Analysis Plan A • 88 events in test + patients needed to detect 50% reduction in hazard at 5% two-sided significance level with 90% power • If 25% of patients are positive, then when there are 88 events in positive patients there will be about 264 events in test negative patients • 264 events provides 90% power for detecting 33% reduction in hazard at 5% two-sided significance level • Sequential futility monitoring may enable early cessation of accrual of test negative patients

33. Study-wise false positivity rate is limited to 5% with analysis plan A • It is not necessary or appropriate to require that the treatment vs control difference be significant overall before doing the analysis within subsets

34. Analysis Plan B • Compare the new drug to the control overall for all patients ignoring the classifier. • If poverall 0.03 claim effectiveness for the eligible population as a whole • Otherwise perform a single subset analysis evaluating the new drug in the classifier + patients • If psubset 0.02 claim effectiveness for the classifier + patients.

35. This analysis strategy is designed to not penalize sponsors for having developed a classifier • It provides sponsors with an incentive to develop genomic classifiers

36. Sample size for Analysis Plan B • To have 90% power for detecting uniform 33% reduction in overally hazard at 3% two-sided level requires 297 events (instead of 263 for similar power at 5% level) • If 25% of patients are test positive, then when there are 297 total events there will be approximately 75 events in positive patients • 75 events provides 75% power for detecting 50% reduction in hazard at 2% two-sided significance level • By delaying evaluation in test positive patients, 80% power is achieved with 84 events and 90% power with 109 events

37. Analysis Plan C • Test for interaction between treatment effect in test positive patients and treatment effect in test negative patients • If interaction is significant at level int then compare treatments separately for test positive patients and test negative patients • Otherwise, compare treatments overall

38. Sample Size Planning for Analysis Plan C • 88 events in test + patients needed to detect 50% reduction in hazard at 5% two-sided significance level with 90% power • If 25% of patients are positive, then when there are 88 events in positive patients there will be about 264 events in negative patients • 264 events provides 90% power for detecting 33% reduction in hazard at 5% two-sided significance level

39. Simulation Results for Analysis Plan C • Using int=0.10, the interaction test has power 93.7% when there is a 50% reduction in hazard in test positive patients and no treatment effect in test negative patients • A significant interaction and significant treatment effect in test positive patients is obtained in 88% of cases under the above conditions • If the treatment reduces hazard by 33% uniformly, the interaction test is negative and the overall test is significant in 87% of cases

40. Development of Genomic Classifiers • During phase II development or • After failed phase III trial using archived specimens. • Adaptively during early portion of phase III trial.

41. Adaptive Signature Design An adaptive design for generating and prospectively testing a gene expression signature for sensitive patients Boris Freidlin and Richard Simon Clinical Cancer Research 11:7872-8, 2005

42. Adaptive Signature DesignEnd of Trial Analysis • Compare E to C for all patients at significance level 0.03 • If overall H0 is rejected, then claim effectiveness of E for eligible patients • Otherwise

43. Otherwise: • Using only the first half of patients accrued during the trial, develop a binary classifier that predicts the subset of patients most likely to benefit from the new treatment E compared to control C • Compare E to C for patients accrued in second stage who are predicted responsive to E based on classifier • Perform test at significance level 0.02 • If H0 is rejected, claim effectiveness of E for subset defined by classifier

44. Biomarker Adaptive Threshold Design Wenyu Jiang, Boris Freidlin & Richard Simon JNCI 99:1036-43, 2007

45. Biomarker Adaptive Threshold Design • Randomized phase III trial comparing new treatment E to control C • Survival or DFS endpoint

46. Biomarker Adaptive Threshold Design • Have identified a predictive index B thought to be predictive of patients likely to benefit from E relative to C • Eligibility not restricted by biomarker • No threshold for biomarker determined

47. Good Microarray Studies Have Clear Objectives • Class Comparison (gene finding) • Find genes whose expression differs among predetermined classes • Class Prediction • Prediction of response using gene expression profile • Class Discovery • Discover clusters of specimens having similar expression profiles • Discover clusters of genes having similar expression profiles

48. Class Comparison and Class Prediction • Not clustering problems • Supervised methods

49. Class prediction methods usually have gene selection as a component • The criteria for gene selection for class prediction and for class comparison are different • For class comparison false discovery rate is important • For class prediction, predictive accuracy is important

50. Gene Selection • Genes which are univariately most correlated with outcome • Small subset of genes which together give most accurate predictions • Little evidence that complex feature selection methods are useful in microarray problems