Integration of Diagnostic Markers into the Development Process of Targeted Agents

1. Integration of Diagnostic Markers into the Development Process of Targeted Agents Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov

2. BRB Websitehttp://brb.nci.nih.gov • Powerpoint presentations • Reprints • BRB-ArrayTools software • Web based Sample Size Planning Programs • Optimal 2-stage phase II designs • Phase III designs using predictive biomarkers • Phase II/III designs

3. Diagnostic Markers • Predictive biomarkers • Measured before treatment to identify who is likely or unlikely to benefit from a particular treatment • Prognostic biomarkers • Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment • Can be used to identify patients with such good prognosis on limited treatment that they do not require more aggressive approaches

4. Prognostic & Predictive Biomarkers • Most cancer treatments benefit only a minority of patients to whom they are administered • Being able to predict which patients are or are not likely to benefit would • Save patients from unnecessary toxicity, and enhance their chance of receiving a drug that helps them • Control medical costs • Improve the success rate of clinical drug development

5. Prognostic and Predictive Biomarkers in Oncology • Single gene or protein measurement • ER expression, HER2 amplification, EGFR or KRAS mutation • Score or classifier that summarizes expression levels of multiple genes • OncotypeDx, Mammaprint

6. Traditional Approach to Clinical Development a New Drug • Small phase II trials to find primary sites where the drug appears active • Phase III trials with broad eligibility to test the null hypothesis that a regimen containing the new drug is not better than the control treatment overall for all randomized patients • If you reject H0 then treat all future patients satisfying the eligibility criteria with the new regimen, otherwise treat no such future patients with the new drug • Perform subset hypotheses but don’t believe them

7. Traditional Clinical Trial Approaches • Based on assumptions that • Qualitative treatment by subset interactions are unlikely • “Costs” of over-treatment are less than “costs” of under-treatment • Neither of these assumptions is valid with most new molecularly targeted oncology drugs

8. Traditional Clinical Trial Approaches • Have protected us from false claims resulting from post-hoc data dredging not based on pre-defined biologically based hypotheses • Have led to widespread over-treatment of patients with drugs to which many don’t need and from which many don’t benefit • May have resulted in some false negative results

9. Clinical Trials Should Be Science Based • Cancers of a primary site may represent a heterogeneous group of diverse molecular diseases which vary fundamentally with regard to • their oncogenecis and pathogenesis • their responsiveness to specific drugs • The established molecular heterogeneity of human cancer requires the use new approaches to the development and evaluation of therapeutics

10. How Can We Develop New Drugs in a Manner More Consistent With Modern Tumor Biology and ObtainReliable Information About What Regimens Work for What Kinds of Patients?

11. Guiding Principle • The data used to develop the classifier should be distinct from the data used to test hypotheses about treatment effect in subsets determined by the classifier • Developmental studies can be exploratory • Studies on which treatment effectiveness claims are to be based should not be exploratory

12. Co-development of drugs and companion diagnostics increases the complexity of drug development • It does not make drug development simpler, cheaper and quicker • But it may make development more successful and it has great potential value for patients and for the economics of health care

13. Science Based Development of Molecularly Targeted Agents Often Requires • Larger phase II studies focused on more than just evaluating whether a drug is active in a primary site • Phase II trials that permit the development and refinement of a candidate predictive biomarker to be used as part of the design and primary analysis plan of the pivotal phase III trials of the drug • New phase III designs that use predictive biomarkers prospectively in their design and which are sized and structured to evaluate new treatment effectiveness in a patient population focused by the predictive biomarker

14. Some of the conventional wisdom about statistical analysis of clinical trials is not applicable to trials dealing with co-development of drugs and diagnostic • e.g. subset analysis if the overall results are not significant or if an interaction test is not significant or if the randomization was not stratified by the subset defining variable • The current approach to evaluating predictive markers as post-hoc analyses of phase III trials is not an adequate basis for developing a reliable science based predictive oncology

15. Roadmap for Prospective Evaluation • Develop a completely specified genomic classifier of the patients likely to benefit from a new drug • Establish analytical validity of the classifier • Use the completely specified classifier in the primary analysis plan of a phase III trial that permits evaluation of the medical utility of the classifier

16. Develop Predictor of Response to New Drug Using phase II data, develop predictor of response to new drug Patient Predicted Responsive Patient Predicted Non-Responsive Off Study New Drug Control

17. Applicability of Targeted/Enrichment Design • Primarily for settings where the classifier is based on a single gene whose protein product is the target of the drug or the biology seems well understood • eg trastuzumab • With a strong biological basis for the classifier, it may be unacceptable to expose classifier negative patients to the new drug • Analytical validation, biological rationale and phase II data provide basis for regulatory approval of the test if required

18. Evaluating the Efficiency of the Targeted Design • Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004; Correction and supplement 12:3229, 2006 • Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-339, 2005. • reprints and interactive sample size calculations at http://linus.nci.nih.gov

19. When less than half of patients are test positive and the drug has limited or no benefit for test negative patients, the targeted design requires many fewer randomized patients • 88 total events (e.g. death or progression) in test + patients needed to detect 50% reduction in hazard at 5% two-sided significance level with 90% power

20. Web Based Software for Designing RCT of Drug and Predictive Biomarker • http://brb.nci.nih.gov

21. Objections and Risks to Using the Targeted Design • We won’t know whether test negative patients might also benefit • We can study that in a subsequent trial if the drug is effective for the test positive patients • Restricting eligibility to test positive patients forces one to size the trial with adequate power and avoids objections that the test positive subset should not be looked at unless the results are significant overall

22. Objections and Risks to Using the Targeted Design • Since physicians might use the drug without the test, the trial should include broad eligibility and the primary analysis should be the overall null hypothesis. • Trials should be science based. If there is a biologically compelling reason to evaluate the treatment in test positive patients, the trial should be sized with adequate power for test positive patients. If test negative patients are included, they should be analyzed in a separate analysis.

23. Objections and Risks to Using the Targeted Design • We may have the wrong predictive biomarker and using it to restrict eligibility may make it more difficult to use archived specimens for later analysis with other candidate biomarkers • e.g. cetuximab and panitumumab in advanced colorectal cancer • There can be a delicate balance between the desire to use the biomarker to restrict eligibility in order to protect patients whom we do not expect to benefit from the new treatment and the desire to include test negative patients to protect against having the wrong biomarker • Phase II results in test negative and test positive patients can help us decide whether the targeted design is appropriate

24. DevelopPredictor of Response to New Rx Predicted Responsive To New Rx Predicted Non-responsive to New Rx New RX Control New RX Control Stratification Design

25. Stratification Design • Use the test to structure a prospective specified primary analysis plan • Having a prospective analysis plan is essential • “Stratifying” (balancing) the randomization is useful to ensure that all randomized patients have tissue available but is not a substitute for a prospective analysis plan • Not stratifying provides more time for analytical validation of the test • The purpose of the study is to evaluate the new treatment overall and for the pre-defined subsets; not to modify or refine the classifier • The purpose is not to demonstrate that repeating the classifier development process on independent data results in the same classifier

26. R Simon. Using genomics in clinical trial design, Clinical Cancer Research 14:5984-93, 2008 • R Simon. Designs and adaptive analysis plans for pivotal clinical trials of therapeutics and companion diagnostics, Expert Opinion in Medical Diagnostics 2:721-29, 2008

27. Analysis Plan A • Compare the new drug to the control for classifier positive patients • If p+>0.05 make no claim of effectiveness • If p+ 0.05 claim effectiveness for the classifier positive patients and • Compare new drug to control for classifier negative patients using 0.05 threshold of significance

28. Sample size for Analysis Plan A • 88 events in classifier + patients needed to detect 50% reduction in hazard at 5% two-sided significance level with 90% power • The same sample size as might be used for the targeted design • Including the test negative patients does not delay obtaining an answer for test positive patients • If 25% of patients are positive, then when there are 88 events in positive patients there will be about 264 events in negative patients • 264 events provides 90% power for detecting 33% reduction in hazard at 5% two-sided significance level • Sequential futility monitoring may have enabled early cessation of accrual of classifier negative patients • Not much earlier with time-to-event endpoint • Bayesian futility monitoring with a skeptical prior can provide a basis for earlier cessation of accrual of classifier negative patients

29. Analysis Plan B(Limited confidence in test) • Compare the new drug to the control overall for all patients ignoring the classifier. • If poverall 0.03 claim effectiveness for the eligible population as a whole • Otherwise perform a single subset analysis evaluating the new drug in the classifier + patients • If psubset 0.02 claim effectiveness for the classifier + patients.

30. Sample size for Analysis Plan B • To have 90% power for detecting uniform 33% reduction in overall hazard at 3% two-sided level requires 297 events (instead of 263 for similar power at 5% level) • If 25% of patients are positive, then when there are 297 total events there will be approximately 75 events in positive patients • 75 events provides 75% power for detecting 50% reduction in hazard at 2% two-sided significance level • By delaying evaluation in test positive patients, 80% power is achieved with 84 events and 90% power with 109 events

31. Does the RCT Need to Be Significant Overall for the Treatment Comparison to Justify the Pre-planned Focused Subset Analysis? • No • That requirement has been traditionally used to protect against data dredging. It is inappropriate for focused trials of a treatment with a companion test with a pre-planned subset analysis if the analysis plan protects the overall type I error at 5%. .

34. It is difficult to have the right completely defined predictive biomarker identified and analytically validated by the time the phase III trial is ready to start accrual • It requires changes in the way we do phase I/II trials • Adaptive methods for the refinement and evaluation of predictive biomarkers in the pivotal trials in a non-exploratory manner • Use of archived tissues in focused “prospective-retrospective” designs based on randomized pivotal trials • R Simon, S Paik, DF Hayes, JNCI (In Press)

36. Multiple Biomarker DesignA Generalization of the Biomarker Adaptive Threshold Design • Have identified K candidate binary classifiers B1 , …, BK thought to be predictive of patients likely to benefit from T relative to C • Eligibility not restricted by candidate classifiers • Let the B0 classifier classify all patients positive

37. Test T vs C restricted to patients positive for Bk for k=0,1,…,K • Let S(Bk) be log partial likelihood ratio statistic for treatment effect in patients positive for Bk • It measures treatment effect restricted to the subset of patients positive for marker Bk • Let S* = max{S(Bk)} , k* = argmax{S(Bk)} • S* is the largest treatment effect observed • k* is the marker that identifies the patients where the largest treatment effect is observed

38. For a global test of significance • Randomly permute the treatment labels and repeat the process of computing S* for the shuffled data • Repeat this to generate the distribution of S* under the null hypothesis that there is no treatment effect for any subset of patients • The statistical significance level is the area in the tail of the null distribution beyond the value of S* obtained for the un-suffled data • If the data value of S* is significant at 0.05 level, then claim effectiveness of T for patients positive for marker k*

39. Repeating the analysis for bootstrap samples of cases provides • an estimate of the stability of k* (the indication)

40. Seamless Phase II/III Design With Candidate Predictive Biomarkers • Objective is to compare new treatment E to control treatment C with regard to survival or DFS • At start of trial there are K candidate predictive biomarkers • Eligible patients have biomarkers measured and are randomized to receive either E or C • Biomarkers are not used to restrict eligibility • Size trial for final analysis after N total events to consist of two parts • comparison of E to C with regard to phase III endpoint for all randomized patients using statistical significance threshold o<0.05 • Comparison of E to C with regard to phase III endpoint for a subset S of randomized patients using statistical significance threshold 0.05 - o

41. Seamless Phase II/III DesignInterim Analysis • A single interim analysis is based on a phase II type endpoint (tumor response) or biomarker intermediate response endpoint (e.g. based on optical imaging, functional imaging, circulating tumor cell count, …) • Interim analysis will be based on patients randomized and with sufficient minimal follow-up by the time of the interim analysis. • Interim analysis will have two parts • Predictive biomarker selection • Futility analysis

42. Seamless Phase II/III DesignInterim Analysis • Predictive biomarker selection • Let pk denote statistical significance level for comparing outcomes of treatment groups E vs C for patients positive for candidate marker k. • If biomarkers are not binary, compute pk as representing the interaction between marker score and treatment effect • Based on the values p1 , p2 , …, pK , identify the single most promising candidate predictive marker or note that none are promising

43. Seamless Phase II/III DesignInterim Analysis • Futility analysis • Let pinterimdenote the statistical significance level comparing outcomes for the phase II endpoint for all patients treated with E to those treated with C • If pinterim > p* and none of the candidate predictive biomarkers are sufficiently promising, then terminate accrual to the trial • Otherwise, continue accrual as initially planned, with no change to eligibility criteria

44. Final analysis will consist of two parts • comparison of E to C with regard to phase III endpoint for all randomized patients using statistical significance threshold o<0.05 • Comparison of E to C with regard to phase III endpoint for a subset S of randomized patients using statistical significance threshold 0.05 - o • Subset S consists of the patients positive for the biomarker selected as most promising during the interim analysis. The patients included in the interim analysis are excluded from the final subset analysis

45. Information about a predictive biomarker may develop following completion of the pivotal trials • It may be infeasible to conduct a new prospective trial for a previously approved drug • KRAS for anti-EGFR antibodies in colorectal cancer • HER2 for doxorubicin in breast cancer

46. Use of Archived SamplesProspective – Retrospective Studies • In some cases the benefits of a prospective trial can be closely achieved by the carefully planned use of archived tissue from a previously conducted randomized clinical trial

47. Use of Archived Specimens in Evaluation of Prognostic and Predictive BiomarkersRichard M. Simon, Soonmyung Paik and Daniel F. Hayes • Claims of medical utility for prognostic and predictive biomarkers based on analysis of archived tissues can be considered to have either a high or low level of evidence depending on several key factors. • Studies using archived tissues, when conducted under ideal conditions and independently confirmed can provide the highest level of evidence. • Traditional analyses of prognostic or predictive factors, using non analytically validated assays on a convenience sample of tissues and conducted in an exploratory and unfocused manner provide a very low level of evidence for clinical utility.

48. Use of Archived Specimens in Evaluation of Prognostic and Predictive BiomarkersRichard M. Simon, Soonmyung Paik and Daniel F. Hayes • For Level I Evidence: • (i) archived tissue adequate for a successful assay must be available on a sufficiently large number of patients from a phase III trial so that the appropriate analyses have adequate statistical power and that the patients included in the evaluation are clearly representative of the patients in the trial. • (ii) The test should be analytically and pre-analytically validated for use with archived tissue. • (iii) The analysis plan for the biomarker evaluation should be completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be focused on evaluation of a single completely defined classifier. • iv) the results from archived specimens should be validated using specimens from a similar, but separate, study.

49. Acknowledgements • NCI Biometric Research Branch • Boris Freidlin • Sally Hunsberger • Wenyu Jiang • Aboubakar Maitournam • Yingdong Zhao • Soon Paik, NSABP • Daniel Hayes, U. Michigan