Cancer Clinical Trials in the Genomic Era

1. Cancer Clinical Trials in the Genomic Era Richard Simon, D.Sc. Chief, Biometric Research Branch National Cancer Institute http://brb.nci.nih.gov

2. Prognostic biomarkers • Measured before treatment to indicate long-term outcome for patients untreated or receiving standard treatment • May reflect both disease aggressiveness and effect of standard treatment • Used to determine who needs more intensive treatment • Predictive biomarkers • Measured before treatment to identify who will benefit from a particular treatment

3. Endpoint • Measured before, during and after treatment to monitor pace of disease and treatment effect • Pharmacodynamic (phase 0-1) • Does drug hit target • Intermediate response (phase 2) • Does drug have anti-tumor effect • Surrogate for clinical outcome (phase 3)

4. Prognostic & Predictive Biomarkers • Single gene or protein measurement • Scalar index or classifier that summarizes contributions of multiple genes

5. Prognostic & Predictive Biomarkersin Genomic Oncology • Many cancer treatments benefit only a minority of patients to whom they are administered • Being able to predict which patients are likely to benefit can • Help patients get an effective treatment • Help control medical costs • Improve the success rate of clinical drug development

6. Validation = Fitness for Intended Use

7. Biomarker Validity • Analytical validity • Measures what it’s supposed to • Reproducible and robust • Clinical validity (correlation) • It correlates with something clinically • Medical utility • Actionable resulting in patient benefit

8. Clinical Utility • Biomarker informs action that benefits patient by improving treatment decisions • Identify patients who have very good prognosis on standard treatment and do not require more intensive regimens • Identify patients who are likely or unlikely to benefit from a specific regimen

9. Objective:Use biomarkers to • Develop effective treatments • Know who needs these treatments and who benefits from them

10. Prognostic markers • There is an enormous published literature on prognostic markers in cancer. • Very few prognostic markers (factors) are recommended for measurement by ASCO, are approved by FDA or are reimbursed for by payers. Very few play a role in treatment decisions.

11. Prognostic Biomarkers Can be Therapeutically Relevant • <10% of node negative ER+ breast cancer patients require or benefit from the cytotoxic chemotherapy that they receive

12. OncotypeDx Recurrence Score • Intended use: • Patients with node negative estrogen receptor positive breast cancer who are going to receive an anti-estrogen drug following local surgery/radiotherapy • Identify patients who have such good prognosis that they are unlikely to derive much benefit from adjuvant chemotherapy

13. Selected patients relevant for the intended use • Analyzed the data to see if the recurrence score identified a subset with such good prognosis that the absolute benefit of chemotherapy would at best be very small in absolute terms • Used an analytically validated test

15. Major problems with prognostic studies of gene expression signatures • Inadequate focus on intended use • Reporting highly biased estimates of predictive value

16. Major problems with prognostic studies of gene expression signatures • Inadequate focus on intended use • Cases selected based on availability of specimens rather than for relevance to intended use • Heterogeneous sample of patients with mixed stages and treatments. Attempt to disentangle effects using regression modeling • Too a great a focus on which marker is prognostic or independently prognostic, not whether the marker is effective for intended use

17. Goodness of fit is not a proper measure of predictive accuracy • Odds ratios and hazards ratios are not proper measures of prediction accuracy • Statistical significance of regression coefficients are not proper measures of predictive value

18. Goodness of Fit vs Prediction Accuracy • For p>n problems, fit of a model to the same data used to develop it is no evidence of prediction accuracy for independent data

20. Validation of Prognostic Model • Completely independent validation dataset • Splitting dataset into training and testing sets • Evaluate 1 completely specified model on test set • Cross-validation

21. Leave-one-out Cross Validation for Classifier of Two Classes • Full dataset P={1,2,…,n} • Omit case 1 • V1={1}; T1={2,3,…,n} • Develop classifier using training set T1 • Classify cases in V1 and count whether classification is correct or not • Repeat for case 2,3,… • Total number of mis-classified cases

22. Complete cross Validation • Cross-validation simulates the process of separately developing a model on one set of data and predicting for a test set of data not used in developing the model • All aspects of the model development process must be repeated for each loop of the cross-validation • Feature selection • Tuning parameter optimization

23. Cross Validation • The cross-validated estimate of misclassification error is an estimate of the prediction error for the model fit applying the specified algorithm to full dataset

24. Prediction on Simulated Null DataSimon et al. J Nat Cancer Inst 95:14, 2003 • Generation of Gene Expression Profiles • 20 specimens (Pi is the expression profile for specimen i) • Log-ratio measurements on 6000 genes • Pi ~ MVN(0, I6000) • Can we distinguish between the first 10 specimens (Class 1) and the last 10 (Class 2)? • Prediction Method • Compound covariate predictor built from the log-ratios of the 10 most differentially expressed genes.

26. Cross-validation Estimate of Prediction Error

28. Partition data set D into K equal parts D1,D2,...,DK • First training set T1=D-D1 • Develop completely specified prognostic model M1 using only data T1 • eg • Using M1, compute prognostic score for cases in D1 • Develop model M2 using only T2 and then score cases in D2

29. Repeat for ... TK -> MK -> DK • Group patients into 2 or more risk groups based on their cross-validated scores • Calculate Kaplan-Meier survival curve for each risk-group

31. To evaluate significance, the log-rank test cannot be used for cross-validated Kaplan-Meier curves because the survival times are not independent

32. Statistical significance can be properly evaluated by approximating the null distribution of the cross-validated log-rank statistic • Permute the survival times and repeat the entire cross-validation procedure to generate new cross-validated K-M curves for low risk and high risk groups • Compute log-rank statistic for the curves • Repeat for many sets of permutations

33. Predictive Biomarkers • Cancers of a primary site often represent a heterogeneous group of diverse molecular entities which vary fundamentally with regard to • the oncogenic mutations that cause them • their responsiveness to specific drugs

34. In most positive phase III clinical trials comparing a new treatment to control, most of the patients treated with the new treatment did not benefit. • Adjuvant breast cancer: 70% long-term disease-free survival on control. 80% disease-free survival on new treatment. 70% of patients don’t need the new treatment. Of the remaining 30%, only 1/3rd benefit.

35. Predictive Biomarkers • Estrogen receptor over-expression in breast cancer • Anti-estrogens, aromatase inhibitors • HER2 amplification in breast cancer • Trastuzumab, Lapatinib • OncotypeDx gene expression recurrence score in N+ ER+ breast cancer • Low score -> not responsive to chemotherapy • KRAS in colorectal cancer • WT KRAS = cetuximab or panitumumab • EGFR mutation in NSCLC • EGFR inhibitor • V600E mutation in BRAF of melanoma • vemurafenib • ALK translocation in NSCLC • crizotinib

39. Standard Paradigm of Phase III Clinical Trials • Broad eligibility • Base primary analysis on ITT eligible population • Don’t size for subset analysis, allocate alpha for subset analysis or trust subset analysis • Only believe subset analysis if overall treatment effect is significant and interaction is significant

40. Standard Paradigm Sometimes Leads to • Treating many patients with few benefiting • Small average treatment effects • Problematic for health care economics • Inconsistency in results among studies • False negative studies

41. The standard approach to designing phase III clinical trials is based on two assumptions • Qualitative treatment by subset interactions are unlikely • “Costs” of over-treatment are less than “costs” of under-treatment

42. Subset Analysis • In the past generally used as secondary analyses • Numerous subsets examined • No control of type I error • Trial not sized for subset analysis

43. Neither conventional approaches to subset analysis nor the broad eligibility paradigm are adequate for genomic based oncology clinical trials • We need a prospective approach that includes • Preserving study-wise type I error • Sizing the study for the primary analysis that includes any subset analysis • If there are multiple subsets, replacing subset analysis with development and internal unbiased evaluation of an indication classifier

44. Although the randomized clinical trial remains of fundamental importance for predictive genomic medicine, some of the conventional wisdom of how to design and analyze rct’s requires re-examination • The concept of doing an rct of thousands of patients to answer a single question about average treatment effect for a target population presumed homogeneous with regard to the direction of treatment efficacy in many cases no longer has an adequate scientific basis

45. How can we develop new drugs in a manner more consistent with modern tumor biology and obtain reliable information about what regimens work for what kinds of patients?

46. Development is Most Efficient When the Scientific Basis for the Clinical Trial is Strong • Having an important molecular target • Having a drug that is deliverable at a dose and schedule that can effectively inhibit the target • Having a pre-treatment assay that can identify the patients for whom the molecular target is driving progression of disease

47. When the Biology is Clear the Development Path is Straightforward • Develop a classifier that identifies the patients likely (or unlikely) to benefit from the new drug • Develop an analytically validated test • Measures what it should accurately and reproducibly • Design a focused clinical trial to evaluate effectiveness of the new treatment in test + patients

48. Off Study Control New Drug Patient Predicted Non-Responsive Develop Predictor of Response to New Drug Patient Predicted Responsive Using phase II data, develop predictor of response to new drug Targeted (Enrichment) Design

49. Predictive Biomarkers • Estrogen receptor over-expression in breast cancer • Anti-estrogens, aromatase inhibitors • HER2 amplification in breast cancer • Trastuzumab, Lapatinib • OncotypeDx gene expression recurrence score in breast cancer • Low score for ER+ node - -> no chemotherapy • KRAS in colorectal cancer • WT KRAS = cetuximab or panitumumab • EGFR mutation in NSCLC • EGFR inhibitor • V600E mutation in BRAF of melanoma • vemurafenib • ALK translocation in NSCLC • crizotinib

50. Evaluating the Efficiency of Targeted Design • Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004; Correction and supplement 12:3229, 2006 • Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-339, 2005.