1 / 40

Mark Pletcher 6/9/2011

Prognostic and Genetic Tests. Mark Pletcher 6/9/2011. An Example. “Mammaprint” Gene expression profiling for Breast CA Grind up the tumor, extract RNA Incubate with a microarray of DNA fragments to estimate expression for each gene 70 previously identified genes predict outcomes.

sveta
Download Presentation

Mark Pletcher 6/9/2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prognostic and Genetic Tests Mark Pletcher 6/9/2011

  2. An Example • “Mammaprint” • Gene expression profiling for Breast CA • Grind up the tumor, extract RNA • Incubate with a microarray of DNA fragments to estimate expression for each gene • 70 previously identified genes predict outcomes

  3. Van de Vijver et al. NEJM 2002;347(25):1999-2009

  4. An Example • “Mammaprint” • Pattern of expression correlates with disease-free and overall survival

  5. Van de Vijver et al. NEJM 2002;347(25):1999-2009

  6. An Example • “Mammaprint” 10-year probability of: SurvivalFree of mets “Good” pattern 95% 85% “Bad” pattern 55% 51% Van de Vijver et al. NEJM 2002;347(25):1999-2009

  7. Outline • Prognostic vs. Diagnostic Tests • Evaluating a Prognostic Test • Accuracy • Utility • Genetic Tests (very briefly)

  8. Prognostic vs. Diagnostic Tests • How is a prognostic test different from a diagnostic test?

  9. Prognostic vs. Diagnostic Tests Identify Prevalent Disease Predict Incident Disease/Outcome Prior to Test After Test Cross-Sectional Cohort 1 (gold standard) <1 (not clairvoyant)

  10. Prognostic vs. Diagnostic Tests • Classic prognosis: • Prediction of death after diagnosis of a disease

  11. Prognostic vs. Diagnostic Tests • Prognosis, broadly speaking: • Prediction of any future event • Death or recurrence of cancer • Stroke after presentation for TIA • Peri-operative MI in surgical patients • First MI in asymptomatic persons

  12. Prognostic vs. Diagnostic Tests • Prognosis vs. Diagnosis: A Spectrum • Grey areas • Pre-clinical disease: Coronary calcium • “Reversible” disease: Tiny lung CA • Irreversible predisposition: Huntington’s gene

  13. Prognostic vs. Diagnostic Tests • Prognostication ≠ Etiology • Risk factor • Causes the disease • Reducing it may prevent disease • Confounding is crucial issue in observational studies • Risk marker (i.e., prognostic factor) • Predicts the disease • Need not be concerned about unmeasured confounders • Not all risk markers are risk factors…(e.g., CRP)

  14. Evaluating Prognostic Tests • Test Performance • Association • Discrimination • Calibration • Reclassification • Pitfalls • Test Utility

  15. Evaluating Prognostic Tests • Association • Is the marker associated with development of the disease? • Odds ratio, relative risk, hazard ratio • “Independently associated” means after adjustment for other known predictors

  16. Evaluating Prognostic Tests • HRadj = 4.6 • P<.001 Van de Vijver et al. NEJM 2002;347(25):1999-2009

  17. Evaluating Prognostic Tests • Discrimination • Ability to distinguish between people with higher or lower risk of disease • Metrics: just like diagnostic tests!? • Sensitivity/specificity • ROC curves

  18. Evaluating Prognostic Tests • Mammaprint Sensitivity = 28/30 = 93% Specificity = 41/83 = 49% Mets <5yrNo mets

  19. Evaluating Prognostic Tests • Coronary artery calcium • Predictor of CHD events • Adds discrimination • AUROC .63.68 FRS = Framingham Risk Score CACS = Coronary Artery Calcium Score Greenland et al. JAMA 2004;291(2):210-215

  20. Evaluating Prognostic Tests • Discrimination • Results are specific to a particular time point • 5-year risk of metastases or death • 90-day risk of stroke

  21. Evaluating Prognostic Tests • Discrimination Different results at 5 years….

  22. Evaluating Prognostic Tests • Discrimination …than at 10 years

  23. Evaluating Prognostic Tests • Discrimination • Often 1 time point is most relevant or easily communicated, but information is lost… • Can think of a “set” of discrimination statistics/ROC curves • Harell’s C-Statistic • Integrated C-statistic for survival data • Similar interpretation as AUROC Harrell et al. Stat Med 1996;15(4):361-87.

  24. Evaluating Prognostic Tests • Calibration • How close is predicted risk to actual risk?

  25. Evaluating Prognostic Tests • Prognostic test results are often converted into absolute risk estimates • Like post-test probabilities in diagnosis • Required for clinical interpretation • Estimated directly in a longitudinal study

  26. Evaluating Prognostic Tests • But absolute risk estimates can be “off” • When derivation population different than target population, etc • Framingham example

  27. D’Agostino et al. JAMA 2001;286(2):180-187

  28. Evaluating Prognostic Tests • Calibration is “orthogonal” to discrimination • Awful discrimination but good calibration • Awful calibration but good discrimination • Miscalibration leads to worse errors, but it’s easier to fix…

  29. Evaluating Prognostic Tests • Reclassification • How often does the test lead to reclassification across a treatment threshold? • i.e., how often might the test lead to a change in treatment? • CRP reclassification example

  30. Evaluating Prognostic Tests • Reclassification • How often does the test lead to reclassification across a treatment threshold? Cook et al. Annals of Int Med 2006;145(1):21-29

  31. Evaluating Prognostic Tests • Reclassification metrics • Net Reclassification Improvement (NRI) • Net % reclassified correctly • Depends on specified treatment thresholds/categories

  32. Evaluating Prognostic Tests • Pitfalls for prognostic test studies • Loss to follow-up and competing risks • Especially problematic if loss is “differential”

  33. Evaluating Prognostic Tests • Pitfalls for prognostic test studies • Bias if clinician knows the test result • e.g. – persons with coronary calcium+ are: • More likely to get revascularization • More likely to get referred to ED if they have chest pain

  34. Evaluating Prognostic Tests • Pitfalls for prognostic test studies • Overfitting • Test will perform best in sample from which it is derived • More variables and “choices”  more danger of overfitting • Gene expression arrays, proteomics

  35. Evaluating Prognostic Tests • Clinical Utility • Does it improve health?

  36. Evaluating Prognostic Tests Better patient understanding of disease/risk 1 2 Healthier patient behaviors Better health Test Result 3 Better clinical decisions Pletcher et al. Circulation 2011;123;1116-1124

  37. Evaluating Prognostic Tests • Clinical Utility • Cannot be estimated from test performance metrics alone • Need to understand downstream consequences, including • Benefits and harms of interventions based on test result • Harms from test itself • Quality and length of life • Costs

  38. Evaluating Prognostic Tests • Clinical Utility • Can be estimated directly… • Randomized trial of test-and-treat strategy • …or indirectly • Decision analysis/cost-effectiveness modeling • Same issues for diagnostic tests, and especially important when screening apparently healthy people… Pletcher et al. Circulation 2011;123;1116-1124

  39. Genetic Tests • Potentially useful for mechanistic insight • Prognostic implications across individuals in a family • Otherwise, must meet same standards for prognostic utility as other tests • Single gene studies often disappointing

  40. Key concepts • For prognostic tests, an element of time and chance remain (perfect test impossible) • Discrimination vs. Calibration • Reclassification indices help us understand how often a test might change management • Clinical utility depends on accounting for net benefits and harms (and costs)

More Related