Statistical challenges in the validation of surrogate endpoints

1. Statistical challenges in the validation of surrogate endpoints Marc Buyse International Drug Development Institute (IDDI), Brussels Limburgs Universitair Centrum, Diepenbeek, Belgium marc.buyse@iddi.com FDA Industry Workshop, September 22-23, 2004

2. Outline • Need for surrogates • Definitions • Validation criteria • Single trial • Several trials (meta-analysis) • Case studies • PSA and survival (advanced prostatic cancer) • 3-year PFS and 3-year OS (early colorectal cancer)

3. Why do we need surrogates? • Practicality of studies: • Shorter duration • Smaller sample size (?) • Availability of biomarkers: • Tissue, cellular, hormonal factors, etc. • Imaging techniques • Genomics, proteomics, other-ics Ref: Schatzkin and Gail, Nature Reviews (Cancer) 2001, 3.

4. Validity of a surrogate endpoint Evidence that biomarkers predict clinical effects • Epidemiological • Pathophysiological • Biological • Statistical What are the conditions required to show this? Ref: Biomarkers Definition Working Group, Clin Pharmacol Ther 2001, 69: 89.

5. Definitions • Clinical endpoint: a characteristic or variable that reflects how a patient feels, functions, or survives • Biomarker: a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention • Surrogate endpoint: a biomarker that is intended to substitute for a clinical endpoint. A surrogate endpoint is expected to predict clinical benefit (or harm or lack of benefit or harm) Ref: Temple, JAMA 1999;282:790.

6. Single trial Parameters of interest • effect of treatment on surrogate endpoint () • effect of treatment on true endpoint () • effect of surrogate on true endpoint () • adjusted effect of treatment on true endpoint (S) • adjusted effect of surrogate on true endpoint (Z) Ref: Buyse and Molenberghs, Biometrics 1998;54:1014.

7. Surrogateendpoint Treatment Trueendpoint

8. Correlation of endpoints is not enough Key point: “A correlate does not a surrogate make”    0 is not a sufficient condition for validity Ref: Fleming and DeMets, Ann Intern Med 1996, 125: 605.

9. A first formal definition and criteria Prentice’s definition H0S :  = 0  H0T : = 0 Prentice’s criteria An endpoint can be used as a surrogate if • it predicts the final endpoint (  0) • it fully captures the effect of treatment upon the final endpoint (  0 and S = 0) Ref: Prentice, Statist in Med 1989;8:431.

10. A first formal definition and criteria Problems with Prentice’s approach • rooted in hypothesis testing • require significant treatment effects • overly stringent • criteria not equivalent to definition (except for binary endpoints) • one can never prove the null (S = 0) Ref: Buyse and Molenberghs, Biometrics 1998;54:1014.

11. The proportion explained Freedman’s “proportion explained” is defined as PE = 1 - S / • if S =, PE = 0 and the surrogate explains nothing • if S =0, PE = 1 and the surrogate explains the entire effect of treatment on the true endpoint Ref: Freedman et al, Statist in Med 1989;8:431.

12. The proportion explained Problems with the proportion explained • PE is not a proportion (can be <0 or >1) • PE confuses two sources of variability, one at the individual level, the other at the trial level: PE = Z / • PE can be anywhere on the real line, depending on precision of S and T… Ref: Molenberghs et al, Controlled Clin Trials 2002;23:607.

13. Statistical validation of surrogate endpoints “The effect of treatment on a surrogate endpoint must be reasonably likely to predict clinical benefit” Ref: Biomarkers Definitions Working Group, Clin Pharmacol Ther 2001;69:89.

14. The relative effect Interest now focuses on the two components of PE: • the surrogate must predict the true endpoint (Z  0) • the relative effect, defined as RE = / allows prediction of the effect of treatment on the true endpoint () based on the effect of treatment on the surrogate () Ref: Buyse and Molenberghs, Biometrics 1998;54:1014.

15. Prediction of true endpoint from surrogate endpoint Endpoints observed on individual patients R² indicates quality of regression True Endpoint Slope =  Surrogate Endpoint

16. Prediction of treatment effect: one trial Treatment effect observedin the trial 1 .5 Slope = / 0 Treatment Effect onTrue Endpoint () Regression through origin; only one point! -.5 -1 -1 0 1 Treatment Effect on Surrogate Endpoint ()

17. Several trials For a marker to be used as a surrogate, we need “repeated demonstrations of a strong correlation between the marker and the clinical outcome” Ref: Holland, 9th EUFEPS Conference on “Optimising Drug Development: Use of Biomarkers”, Basel, 2001.

18. Prediction of treatment effect: several trials Treatment effects observedin all trials 1 .5 Slope = / 0 Treatment Effect onTrue Endpoint () -.5 R² indicates quality of regression -1 -1 0 1 Treatment Effect on Surrogate Endpoint ()

19. Validation criteria using several trials Parameters of interest • effect of treatment on surrogate endpoint () • effect of treatment on true endpoint () • effect of surrogate on true endpoint () • measure of association between surrogate endpoint and true endpoint (R²individual) • measure of association between effects of treatment on surrogate endpoint and on true endpoint (R²trial) Ref: Buyse et al, Biostatistics 2000;1:49; Gail et al, Biostatistics 2000;1:231.

20. Technical difficulties: the endpoints are not normally distributed In practice, endpoints are often of the following type : response, survival, longitudinal. Such endpoints are not normally distributed, and therefore complex modelling is required to characterize the association between endpoints (“individual level association”). At the trial level, however, simple linear models are still adequate to characterize the association between treatment effects on the endpoints (“trial level association”). Refs: Molenberghs et al, Stat Med 20:3023, 2001; Burzykowski et al, J Royal Stat Soc A 50: 405, 2001; Renard et al, J Applied Statist 30:235, 2002.

21. A case study in advanced prostatic cancer:the trials • Two multicentric trials for patients in relapse after first-line endocrine therapy (596 patients) • Unit of analysis for treatment effects: country (19 units) • Patients randomized between two treatments: • Experimental (retinoic acid metabolism-blocking agent) • Control (anti-androgen) Ref: Buyse et al, in: Biomarkers in Clinical Drug Development (Bloom JC, ed.): Springer-Verlag, 2003.

22. A case study in advanced prostatic cancer:the endpoints Potential surrogate endpoints: • Longitudinal PSA measurements taken at pre-defined time points • PSA response (decrease of at least 50%) • Time to PSA progression (TPP) True endpoint: • Overall survival

23. A case study in advanced prostatic cancer Surrogateendpoint Treatment Experimental Rz Control Trueendpoint

24. PSA response as surrogate for survival Very weak association between treatment effects R² = 0.05

25. TTP as surrogate for survival Weak association between treatment effects R² = 0.22

26. Longitudinal PSA as surrogate for survival Moderate association between treatment effects R²trial = 0.45

27. Individual-level association between PSA and survival [95% C.I.] Trial-level association between treatment effects on PSA and survival [S.E.] PSA response Survival odds ratio = 5.5 [2.7 - 8.2] R²trial = 0.05 [0.13] Time to PSA progression Survival odds ratio = 6.3 [4.4 – 8.2] R²trial = 0.22 [0.18] Longitudinal PSA Coefficient of determination R²(t) > 0.84 at all times t R²trial = 0.45 [0.18] Individual-level and trial-level measures of association

28. A case study in early colorectal cancer:the trials • Fifteen collaborative group trials for patients after resection of colorectal tumor (12,915 patients) • Unit of analysis for treatment effects: 18 comparisons between 33 treatment arms • Patients randomized between various 5-FU regimens and/or control

29. A case study in early colorectal cancer:the endpoints Potential surrogate endpoint: • 3-year disease-free survival True endpoint: • 5-year overall survival Ref: Sargent et al, Proceedings ASCO (Abstract # 3502), 2004. Acknowledgement: the following slides are based on Dr Daniel Sargent’s presentations to ODAC on May 5 and at ASCO on June 6

30. Most recurrences occur before 3 years

31. Strong association between endpoints

32. Strong association between treatment effects

33. Predicted versus actual OS hazard ratios

34. Overview of validation approaches • Single trial • full capture (Prentice) • proportion explained (Freedman et al) • relative effect (Buyse & Molenberghs) • likelihood reduction factor (Alonso et al) • Several trials (meta-analysis) • concordance (Begg & Leung) • correlation of effects (Daniels & Hughes) • trial-level measures of association (Gail et al) • individual- and trial-level measures of association (Buyse et al) • predicted treatment effect (Baker) • surrogate threshold effect (Burzykowski & Buyse)

35. Conclusions on surrogate validation • Ideally, statistical validation requires the following: • data from randomized trials • replication at the trial or center level • at least some observations of T • large numbers of observations • range of therapeutic questions (Z1, Z2, …) • Hence: • individual patient data meta-analyses are needed • access to such data is a problem when they are proprietary Ref: Burzykowski, Molenberghs and Buyse (eds.), “The Evaluation of Surrogate Endpoints”, Springer-Verlag (in press).