Diagnostic Tests

Diagnostic Tests Patrick S. Romano, MD, MPH Professor of Medicine and Pediatrics

The Two-by-two Table Disease+ Disease- Test+ TP FP TP+FP Test- FN TN FN+TN TP+FN FP+TN Total

The Two-by-two Table (cont) True positives: Patients with disease who test positive False negatives: Patients with disease who test negative True negatives: Patients without disease who test negative False positives: Patients without disease who test positive

Test Characteristics Sensitivity: TP/(TP + FN) Test accuracy (or probabilityof correct classification) among patients with disease Specificity: TN/(TN + FP) Test accuracy (or probabilityof correct classification) among patients without disease

Test Characteristics (cont) Positive predictive value: TP/(TP + FP) Predictive value of a positive (abnormal) result OR post-test probability of disease, given positive test Negative predictive value: TN/(TN + FN) Predictive value of a negative (normal) result OR post-test probability of non-disease, given negative test

CAGE Questionnaire Have you ever felt you should CUT down on your drinking? Have peopleANNOYED you by criticizing your drinking? Have you ever felt bad or GUILTY about your drinking? Have you ever had a drink first thing in the morning to steady your nerves or to get rid of a hangover (EYE opener?)

Prevalence of Alcoholismby CAGE Score No ‘Yes’ Alcoholism Alcoholism responses (n) (%) (n) (%) 4 23100 00 3 379713 2 28671433 1 11282872 0 185 35895

Performance Characteristics of CAGE:3-4 “yes” Responses

Performance Characteristics of CAGE:2-4 “yes” Responses

What Affects Sensitivity? • Choice of cutoff value • Quality of administration of test – Equipment, technique, reagents, questionnaire • Quality of interpretation of test • Spectrum of disease (severity distribution) – A truncated sample may result fromusing a test measure to select recipientsof the “gold standard” measure • NOT prevalence

What Affects Specificity? • Choice of cutoff value: • Sensitivity-specificity tradeoff • Quality of administration of test • Quality of interpretation of test • Spectrum of non-disease • Other prevalent diseases may cause false positive values • NOT prevalence

What Affects Predictive Values? • Sensitivity • Specificity • Prevalence (Sensitivity)(Prevalence) PV+ = (Sens)(Prev) + (1-Spec)(1-Prev) (Specificity)(1-Prevalence) PV- = (Spec)(1-Prev) + (1-Sens)(Prev)

Performance Characteristics of CAGE:High Prevalence of Alcoholism

Performance Characteristics of CAGE:Low Prevalence of Alcoholism

Test Characteristics • Likelihood ratio (positive): = Sensitivity / (1-Specificity)= (TP/Disease +) / (FP/Disease –) • Likelihood of a (true) positive testamong patients with disease, relative tothe likelihood of a (false) positive test among those without disease • How much more likely are you to find a positive test result in a person with disease than in a person without disease?

Test Characteristics (cont) • Likelihood ratio (positive): = Sensitivity/(1-Specificity) = (TP/Disease +)/(FP/Disease –) If ODDS = p(event)/[1-p(event)], then: • Pre-test odds x Likelihood ratio = Post-test odds • Prior odds x Likelihood ratio = Posterior odds

PSA Performance (ROC) Curve 100 80 Urologic practice 60 Sensitivity (TP/[TP+FN]) 40 Community screening 20 0 0 20 40 60 80 100 1-Specificity (FP/[TN+FP])

Specificity 1.0 0.8 0.6 0.4 0.2 0 Stage D Stage C 0.2 0.8 Stage B 0.4 0.6 1–Sensitivity Sensitivity Stage A 0.6 0.4 2.5 ng/ml 5.0 ng/ml 10.0 ng/ml 0.8 0.2 0 0.2 0.4 0.6 0.8 0 1–Specificity

Using Bayes Theorem Problem: • In your study, you are using a diagnostic test of unknown accuracy. • A better "gold standard" test is available, but is too expensive or too complicated for you to adopt. • How accurate is your classification of patients based on the cheaper test?

Using Bayes Theorem (cont) Solution—Step 1: Review the literature (or checkwith your instrument supplier or manufacturer) to ascertain the sensitivity and specificity of the measure in previous studies.

Using Bayes Theorem (cont) Solution—Step 2: If possible, do your own "validation.” This usually involves applying the gold standard to a subset of your sample and comparing the results with those of the cheaper test. A 5–10% subsample may suffice (depending on sample size).

Using Bayes Theorem (cont) Solution—Step 3: Apply Bayes theorem to calculate the predictive values of positive and negative tests, based on sensitivity, specificity, and prevalence. • Sensitivity = P(disease)|Test + • Specificity = P(no disease)|Test - • Prevalence = Prior probability of disease in your sample

Using Bayes Theorem (cont) Solution—Step 3 (cont) (Sensitivity)(Prevalence) PV+ = (Sens)(Prev) + (1-Spec)(1-Prev) (Specificity)(1-Prevalence) PV- = (Spec)(1-Prev) + (1-Sens)(Prev)

Using Bayes Theorem—Example You are using daily urinary ratios of pregnanediol-3-glucuronide to creatinine, indexed against each patient's baseline value, to identify anovulatorymenstrual cycles. The “gold standard” involves serum progesterone determinations, butcannot be applied to a largecommunity-based sample.

Using Bayes Theorem—Example (cont) Cycles with a low ratio are labeled anovulatory. The test has a sensitivity of 90% anda specificity of 90%. In the real world, where only 5-10% of cycles are anovulatory, how often willyou misclassify cycles?

Using Bayes Theorem— Example (cont) PV+ = (0.9)(0.1)/[(0.9)(0.1)+(0.1)(0.9)] =0.50 Assuming 10% prevalence PV+ = (0.9)(0.05)/[(0.9)(0.05)+(0.1)(0.95)] =0.32 Assuming 5% prevalence In other words, 50–68% of all cycleslabeled as anovulatory will actuallybe false positives (e.g., ovulatory).

Thank you !

Diagnostic Tests