@2003 Martin L Lesser, PhD

1. SCREENING AND DIAGNOSTIC TESTINGMartin L Lesser, PhDBiostatistics UnitFeinstein Institute for Medical ResearchNorth Shore – LIJ Health System @2003 Martin L Lesser, PhD

3. OUTLINE • What is a Screening test? • Objectives of Screening • Features of a Good Screening test? • Diagnostic Testing • Calculations: Sensitivity, Specificity, PPV, NPV, Accuracy, Prevalence, Bayes’ Theorem, ROC Curves

4. What is a Screening test? • A test administered to a group of asymptomatic people to detect the signs of a disease (does not diagnose… if positive, need further evaluation) • Usually a secondary prevention technique -improve outcome of illness in ‘affecteds’ -reduce severity of disease -reduce mortality

5. SCREENING vs DIAGNOSIS -asymptomatic -possibly symptomatic -usually in a high risk -not necessarily in high group (f-hx, lifestyle) risk group -community setting -clinical setting -inexpensive -can be expensive -easy to administer -may be complex -less invasive -may be invasive -relatively safe -may be risky -does not diagnose per se -goal is definitive diagnosis

6. Features of a Good Screening Test • Serious (important) disease • Detects disease prior to clinical symptoms • Effective therapy or treatment must exist for the disease detected (accessible and acceptable to ‘screenee’) • Early detection would likely lead to a cure or effective treatment • Safe to administer (and quick) • Not very costly • Must not cause undue anxiety • Preferably, follow-up diagnostic test must not be harmful, cumbersome or expensive • Results must be valid, reliable and reproducible

7. Screening Test Examples • Sphygmomanometer: Hypertension, CAD, CVA • Pap Smear: Cervical Cancer • PPD Test: Tuberculosis • Cholesterol test: Hypercholesterolemia, CAD • Mammogram: Breast Cancer • Chest X-ray: Lung Cancer • Fecal Occult Blood Test: Colon Cancer • PSA: Prostate Cancer

8. Who should be screened? -NOBODY -SOME -EVERYBODY -no benefit -who might benefit? -wasteful (low risk) -harmful (no cure) -costly -cheap! -high risk grp! -family hx!

9. Diseases Appropriate for Screening • Must be serious • Beneficial pre-symptomatic treatment • High prevalence of preclinical disease

10. Standard 2x2 Table Disease Disease + Disease - Test + a (TP) b (FP) PPV*** =a/(a+b) PPV*** =a/(a+b) Test Result Test - c (FN) d (TN) NPV*** =d/(c+d) Sensitivity =a/(a+c) Specificity =d/(b+d) ***formula applicable only when sampling is cross-sectional -may have to use Bayes’ Rule!!!

11. SENSITIVITY Persons with the disease who test positive x 100% = _______________________________________________ Total number of persons with the disease a x 100% = _______________ (a + c )

12. SPECIFICITY Persons without the disease who test negative x 100% = _______________________________________________ Total number of persons without the disease d x 100% = _______________ (b + d)

13. POSITIVE PREDICTIVE VALUE (PPV) Persons with a positive test who have the disease x 100% = _______________________________________________ Total number of persons who test positive a x 100% = _______________ (a + b)

14. NEGATIVE PREDICTIVE VALUE (NPV) Persons with a negative test who don’t have disease x 100% = _______________________________________________ Total number of persons who test negative d x 100% = _______________ (c + d)

15. ACCURACY Persons with a correct diagnosis x 100% = _______________________________________________ Total number of persons tested (a+d) x 100% = (TP + TN) x 100% = _______________ ____________________ (a + b + c + d) (a + b + c + d)

16. Test Characteristics • Fixed (Population Independent) -Sensitivity -Likelihood that someone with disease has a positive test -Specificity -Likelihood that someone without disease has a negative test • Relative (Population Dependent) -Positive Predictive Value -Likelihood that someone with a positive test has the disease -Negative Predictive Value -Likelihood that someone with a negative test does not have the disease

17. Fixed Characteristics Highly Sensitive Test -picks up most people with disease who truly have disease • Screening  Rule Out  High Sensitivity • Diagnosis Rule In High Specificity -good for screening!!! Highly Specific Test -unlikely to mislabel people as having disease when in fact they do not have the disease -avoid unnecessary treatment!!!

18. Examples Example 1 : PPD Test for TB Diameter > 1 mm => TB+ -results in too many TB+’s (high FP) Example 2 : CA125 in Ovarian Ca -elevated CA125 even in non-Ovarian Cancer cases!!! -unlikely to miss a true case of Ovarian Ca (high sensitivity) -many people who don’t have the disease will test positive (low specificity)

19. Relative Characteristics PPV and NPV are related to the overall prevalence of the disease in the population you are testing! NOTE! We normally assume that Sensitivity and Specificity remain constant regardless of the prevalence of the disease in the population you are testing .

20. Prevalence and PPV Example: HIV Testing 1. Drug Rehab Center 2. Monastery of St. Claire

21. Bayes’ Theorem PPV/NPV: influenced by 3 quantities: • Sensitivity • Specificity • Prevalence (prior odds) **As Prevalence increases-> PPV increases! **As Prevalence increases-> NPV decreases!

23. Bayes’ Formula for PPV PPV= Pr (D+| T+) x 100% Pr (T+|D+) x Pr (D+) x 100% = _______________________________________________ {Pr (T+|D+) x Pr (D+)} + {Pr(T+|D-) x Pr(D-)} sens x prev x 100% = _____________________________________ {sens x prev} + {(100-spec) x (100-prev)}

24. Bayes’ Formula for NPV NPV= Pr (D-| T-) x 100% Pr (T-|D-) x Pr (D-) x 100% = _______________________________________________ {Pr (T-|D-) x Pr (D-)} + {Pr(T-|D+) x Pr(D+)} spec x (100-prev) x 100% = _________________________________________ {spec x (100-prev)} + {(100-sens) x prev}

25. How to Set Cut Points -It’s like tuning your radio! -Want to pick up certain frequencies (disease) Want to catch disease, i.e. Minimize missing disease Avoid too many false positives Attain a balance of Sensitivity and Specificity!!!

26. Receiver Operating Characteristic Curves (ROC Curves) Legend: England--Battle of Britain -performance of radar receiver operators TP: Correct early warning of German planes coming over the English Channel FP: Receiver operator sent out alarm but no enemy planes appeared FN: German planes appeared without previous warning from the radar operators.

27. Constructing an ROC Curve Simply: Plot Sensitivity on the Y-axis against FP (or 100-Specificity) on the X-axis

28. Hypothetical Example:Blood Pressure Screening to Predict 10-Year Stroke Risk in Subjects 50 Years and Older -Take single blood pressure measurement in a large number of subjects -Follow subjects for 10 years to determine stroke status Cutoff for + Test (SBP) Sensitivity FP Specificity > 0 mm Hg 100% 100% 0% > 120 mm Hg ?? ?? ?? > 130 mm Hg ?? ?? ?? > 140 mm Hg ?? ?? ?? > 500 mm Hg 0% 0% 100%

30. Competing Screening Tests -Plot the ROC curves on the same graph Example: SBP vs. Cholesterol vs. HgbA1c -Area Under the ROC curve: is the probability that a randomly selected pair of normal and abnormal subjects can be correctly classified

32. Do Screening Tests Work? • Analysis of Outcomes Survival of those diagnosed by screening prior to symptoms versus Survival of those diagnosed at the time of symptomatic presentation • Other ways: (Randomized trial, Population based study)

33. Lead Time Bias Precancerous Cells  Small Nodule  Advanced Disease  Death  Time from Dx at Screening to Death  Lead Time   Time from Dx at Clinical Presentation to Death

34. Length Bias • ------------------------------------------------------- • ----------------------------------------------------------- • -------- • --------- • ---------------------------------------------- • --------------- • --------------- • ----- -------------- • ---- • ------------------------------------------------ • ------- • --------------------- • --------------------------------- 

35. Other Sources of Bias (Source: Begg CB, Statistics in Medicine 1987) Subject Selection • Case Mix • Verification bias • Uninterpretable test results • Inter-observer variation • Temporal changes Methodology • Influence of clinical factors on interpretation • Variation in positivity criterion • Absence of a definitive reference test • Cutoff point validation bias

36. REFERENCES • Jekel, JK, Katz, DL, Elmore JG. Epidemiology, Biostatistics, and Preventive Medicine. 2nd Ed. 2001. WB Saunders Company-Harcourt Health Sciences. • Hennekens CH MD DrPH, Buring JE, ScD. Edited by Mayrent SL, PhD.Epidemiology in Medicine. 1st Ed. 1987. Little Brown and Company, Boston/Toronto. • Dawson B, Trapp RG. Basic & Clinical Biostatistics. 3nd Ed. 2000. McGraw-Hill Medical Publishing Division. • Lesser, ML in Fishman-Javitt MC, MD Stein HL, MD Lovecchio JL, MD (eds). 1990.Imaging of the Pelvis-MRI and Correlations to CT and Ultrasound.

37. @2002 Cristina P. Sison, PhD

38. Thanks! For Statistical consulting, call:NORTH SHORE-LIJ HEALTH SYSTEM: BIOSTATISTICS UNIT (516) 240-8300CORNELL PEOPLE: (212) 746-8544CORNELL GCRC: (212) 746-6291 @2002 Cristina P. Sison, PhD