Metrological Experiments in Biomarker Development (Mass Spectrometry—Statistical Issues)

1. Metrological Experiments inBiomarker Development(Mass Spectrometry—Statistical Issues) Walter Liggett Statistical Engineering Division Peter Barker Biotechnology Division National Institute of Standards and Technology

2. Biomarker(Clinical Pharmacology & Therapeutics, 2001) A characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention. Two parts of a biomarker • Execution of measurement protocol • Interpretation of measured response

3. Metrology • Development and evaluation of a measurement protocol, the first part of a biomarker • Diverse lessons learned from varied applications • Focus on general purpose protocols which may be adequate for a particular purpose • The use of metrology in biomarker development is the subject of this talk

4. Metrological Experiments • Experimental units (specimens) • Knowledge of their characteristics • Relation to unknowns of future interest • Response • Univariate—interval-scale variable • Multivariate/Functional • Protocol parameters—parameter design • Cost of experimental runs—high throughput?

5. Outline • Alternative statistical formulations • Classification based on cases and controls • Measurement of an interval-scale variable • Aspects of protocol development • Property of interest • Realization of protocol • Multivariate and functional measurements

6. Statistics for Classification • Assume gold standard for disease status • Evaluate marker on training data • Sensitivity—true positive rate • Specificity—1 – false positive rate • Continuous test result—ROC curves • Multivariate test result—classification, discriminant analysis

7. Pepe, et al., J. National Cancer Institute, 2001Specimen Selection • Wide spectrum of tumor and non-tumor tissue • Serum from cases and controls in a target screening population • Apparently healthy subjects monitored for development of cancer • Cohort from a population that might be targeted • Subjects randomly selected from populations in which the screening program is likely

8. Thinking Outside the Box • Bottom line is prediction of disease status • Definitive gold standard may not be available • Including laboratory sources of error in training data is a problem • There are metrological experiments that do not require a gold standard

9. The Role of Science • Given valid training data, statisticians can proceed without scientific knowledge • In the classification approach, scientific thought must go into specimen selection • In the metrological approach, focus is on a property to be measured • Scientific thought must go into the relation of the metrological property to biomarker goals

10. Statistics for Metrology • Focus (as best one can) on the property to be measured, an interval- or ratio-scale variable • Specify a baseline measurement protocol • Experiment with realizations of alternative protocols • Optimize repeatability (at least) and then ask if the measurement protocol is adequate for the purpose

11. Framework of Metrology • Relation between property and protocol obtained scientifically or through realization • Metrology explores faithfulness of realization before adequacy for the purpose Property Realization Protocol

12. Some Metrological Experiments • Protocol development through classes of units known to differ in the property of interest • Protocols linked to a scientific definition of the property of interest in such a way that all sources of error can be assessed (definitive methods) • Sets of protocols that measure the same property but are based on different scientific principles (independent methods)

13. Aspects of Performance • Repeatability • All manner of reproducibility • Operator, equipment • Inter-laboratory • Noise factors, effect of sample matrix • Calibration • Measurement assurance • Uncertainty components, type A and type B uncertainties

14. Experimental Units(Reference Materials) • Homogeneity (solution versus particles) • Quantity (cost) • Adaptable to high-throughput experiments • Known value of the property of interest • Classes with different values of the property of interest

15. From Univariate to Functional • Carryover has been done for classification • Extending measurement performance concepts to multivariate and functional responses is still a challenge • Chemometrics is the key word for much of the literature in this area

16. Functional Principal Components Analysis (Ramsay and Silverman) • Metrologists like to look at the spread of a batch of measurements (outliers, more than one mode) • For functional measurements, functional PCA provides a way to look at the spread • Consider results of functional PCA on Petricoin’s Lancet…/Normal Healthy (SPLUS, Ramsay’s software) • Main purpose is to illustrate metrological thinking

21. Conclusion • Producing large data sets has become easier except perhaps for selecting individuals with a particular disease status • With scientific and statistical reasoning, the advances in experimentation technology can be used to speed biomarker development • Statisticians have a role in formulating overall experimental strategy, allocating effort among different approaches