Thoughts on Biomarker Discovery and Validation

1. Thoughts on Biomarker Discovery and Validation Karla Ballman, Ph.D. Division of Biostatistics October 29, 2007

2. Outline • General guidelines • Objectives of screening studies • Phase I: Pre-clinical exploratory studies • Phase II: Clinical assay development for clinical disease • Phase III: Retrospective longitudinal repository studies • Phase IV: Prospective screening studies • Phase V: Prevention/control studies

3. General Guidelines • Biomarker / Marker • indicator of a particular disease state of a patient • individual marker, panel of markers, signature, etc. • Clinical development process series of well-defined steps from identification of a potentially useful biomarker through systematic evaluation of its clinical utility

4. General Guidelines (2) • What is the intended final use of biomarker? • clinical versus population screen • stand-alone biomarker versus panel • asymptomatic versus symptomatic normals • diagnosis versus prognosis versus prediction • Need discrete decision points: pursue or not • Criteria • identify markers that have promise to be clinically useful • assess the best methodology for clinical evaluation of markers in question • confirm or validate that additional clinical utility is gained by using marker compared to standard practice

5. Screening Study Objectives • Non-invasive • Inexpensive • Secreted by disease tissue only • Highly sensitive • Highly specific Most likely requires the use of multiple markers to obtain high sensitivity and specificity.

6. Pre-clinical Exploratory Studies • Comparison of disease tissue versus non-disease tissue • Identify unique disease characteristics that might lead to ideas for clinical assays • immunohistochemistry • western blot • gene expression profiles • protein expression profiles • levels of circulating antibodies

7. Pre-clinical Exploratory Studies (2) • Primary objectives • identify leads for potential useful markers • prioritize identified leads • Specimen selection • (case) disease tissue before treatment • (control) non-disease tissue matched to case samples

8. Pre-clinical Exploratory Studies (3) • Primary outcome measure • biomarker value • assay reliability / reproducibility • Analysis • binary • TPR: true positive rate • FPR: false positive rate • continuous • sensitivity (TPR) • specificity (1 – FPR) • ROC curve

10. Pre-clinical Exploratory Studies (4) • Analysis (2) • selection of candidate markers • find all that are statistically significant • rank based on summary statistic • confirmatory analysis • training / test samples • cross-validation

11. Pre-clinical Exploratory Studies (5) • Sample size considerations • number and relative prevalence of disease subtypes • ability of markers to discriminate among different disease subtypes • number of candidate markers under study • number of case / control samples • statistical methodology being used Best to select sample sizes based on simulation studies.

12. Clinical Assay Development • Develop (non-invasive) clinical assay • Primary objective estimate the TPR and FPR (or ROC curve) of the clinical biomarker assay • Other objectives • optimize assay performance • determine relationship between assay levels on disease tissue and clinical specimen • assess patient/subject characteristics associated with biomarker status (level) in control subjects • assess disease characteristics associated with biomarker status (level) in case subjects

13. Clinical Assay Development (2) • Specimen selection • case samples before treatment • control samples matched to case samples • Primary outcome measure result of clinical marker assay

14. Clinical assay development (3) • Analysis • estimate of TPR and FPR (or ROC curve) • test of TPR is too low and/or FPR is too high • select minimally acceptable FPR and determine whether TPR is about the acceptable threshold

15. Clinical assay development (4) • Sample Size • depends on the precision wanted for TPR and FPR • choose size for adequate power to determine whether TPR and FPR are acceptable

16. Retrospective (Longitudinal) Repository Studies • Idea: compare the assay values of case samples collected before their diagnosis to control samples • Primary objectives • evaluate, as a function of time before clinical diagnosis, the capacity of the biomarker to detect preclinical disease • define criteria for a positive screening test

17. Retrospective (Longitudinal) Repository Studies (2) • Other objectives • explore the impact of covariates (demographics, disease-related characteristics, etc.) on the discriminatory abilities of the biomarker before clinical diagnosis • compare markers with a view to selecting those that are most promising • to develop algorithms for screen positivity based on combinations of markers • determine a screening interval if repeated screening is of interest

18. Retrospective (Longitudinal) Repository Studies (3) • Specimen selection • should be protocol driven • cases/controls should be obtained from target population • controls are those that develop disease • should match on all variables, including follow-up

19. Retrospective (Longitudinal) Repository Studies (4) • Primary outcome result of clinical marker assay • Analysis • comparison TPR and FDR (or ROC curves) • consider restricting analysis to TPR at the (maximally) acceptable FPR rate • ROC curves should be time-dependent (to account for time from test to disease presentation)

20. Retrospective (Longitudinal) Repository Studies (5) • Sample size • number of case subjects • number of control subjects • number of clinical specimens per subject The sample sizes should ensure that, for each preclinical time lag of interest (e.g., 1 year, 2 years, 4 years), there are enough specimens from control subjects and from case subjects taken close to those intervals so that biomarker accuracy can be estimated with sufficient precision.

21. Prospective Screening Studies • Idea: the screen is applied to individuals and definitive diagnostic procedures are applied at that time to those screening positive • the number and nature of cases detected with the screening tool are determined • as are the numbers of subjects falsely screening positive and referred for work-up

22. Prospective Screening Studies (2) • Primary objective determine the operating characteristics (TPR and FPR) of the biomarker based screening test in a relevant population • Other objectives • describe the characteristics of disease detected by screening test • assess practicability of applying the screening program • make preliminary assessments of effects of screening on costs/mortality/morbidity • monitor disease that occurs but is not detected by the screening protocol

23. Prospective Screening Studies (3) • Subject selection • target population • inclusion/exclusion criteria • also consider inclusion of unscreened control arm • Primary outcome measure • screening test positive and disease confirmed • screening test positive and disease not confirmed • screening test negative

24. Prospective Screening Studies (4) • Analysis • estimate of detection rate: those screened positive who are positive • estimate of false-referral rate: those screened positive but do not have disease • multivariable analysis to adjust for covariates • comparison of multiple screening tests • Sample size • depends on desired precision, or • depends on relative performance if comparing different screening assays

25. Disease Control Studies • Idea: determine whether screening reduces the burden of disease on the population • Primary objective estimate the reductions in disease mortality afforded by the screening test

26. Disease Control Studies (2) • Other objectives • obtain information about the costs of screening and treatment and the cost per life saved • evaluate compliance with screening and work-up in a diverse range of settings • compare different screening protocols and/or to compare different approaches to treating screen-detected subjects in regard to effects on mortality and costs

27. Disease Control Studies (3) • Subject selection • randomly selected from populations in which the screening program is likely to be implemented • Ideal: standard parallel-arm randomized clinical trial, with one arm consisting of subjects undergoing the screening protocol and the other arm consisting of unscreened subjects

28. Disease Control Studies (4) • Primary outcome time from entry into the study until death • Analysis • survival analysis methods are used to compare the study arms with regard to overall mortality • methods for comparing costs and quality of life for randomized trials

29. Disease Control Studies (5) • Sample size To detect a 20% reduction in cause-specific mortality with 80% power at the .05 two-sided significance level, standard calculations indicate that 650 deaths would need to be observed

30. Discussion • Not all studies need to undergo all the described phases • Need for formal guidelines • For Phase III studies (retrospective repository), need criteria to allocate scarce resources in sensible/fair fashion • Choices of cases and controls in all phases is complex, requires thought • Need new statistical methodology