DIAGNOSTIC TESTS

DIAGNOSTIC TESTS Assist. Prof. E. Çiğdem Kaspar Yeditepe University Faculty of Medicine Department of Biostatistics and Medical Informatics

Why we need a diagnostic test? • We need “information” to make a decision • “Information” is usually a result from a test • Medical tests: • To screen for a risk factor (screen test) • To diagnosse a disease (diagnostic test) • To estimate a patient’s prognosis (pronostic test) • When and in whom, a test should be done? • When “information” from test result have a value.

Value of a diagnostic test • The ideal diagnostic test: • Always give the right answer: • Positive result in everyone with the disease • Negative result in everyone else • Be quick, safe, simple, painless, reliable & inexpensive • But few, if any, tests are ideal. • Thus there is a need for clinically useful substitutes

Is the test useful ? • Reproducibility (Precision) • Accuracy (compare to “gold standard”) • Feasibility • Effects on clinical decisions • Effects on Outcomes

Determining Usefulnessof a Medical Test

Common Issues for Studies of Medical Tests • Spectrum of Disease Severity and Test Results: • Difference between Sample and Population? • Almost tests do well on very sick and very well people. • The most difficulty is distinguishing Healthy & early, presymtomatic disease. • Subjects should have a spectrum of disease that reflects the clinical use of the test.

Common Issues for Studies of Medical Tests • Sources of Variation: • Between patients • Observers’ skill • Equipments => Should sample several different institutions to obtain a generalizable result.

Common Issues for Studies of Medical Tests • Importance of Blinding: (if possible) • Minimize observer bias • Ex. Ultrasound to diagnose appendicitis (It is different to clinical practice)

Studies of the Accuracy of Tests • Does the test give the right answer? • “Tests” in clinical practice: • Symptoms • Signs • Laboratory tests • Imagine tests • To find the right answer. • “Gold standard” is required

Howaccurate is the test? • Validating tests against a gold standard: • Newtests should be validated by comparison against an established goldstandard in an appropriate subjects • Diagnostic tests are seldom 100% accurate (false positives and false negatives will occur)

Describing the performance of a new diagnostic test Physicians are often faced with the task of evaluation the merit of a new diagnostic test. An adequate critical appraisal of a new test requires a working knowledge of the properties of diagnostic tests and the mathematical relationships between them.

Limitations: 1) The gold standard is often the most risky, technically difficult, expensive, or impractical of available diagnostic options. 2) For some conditions, no gold standard is available. The gold standard test:Assessing a new diagnostic test begins with the identification of a group of patients known to have the disorder of interest, using an accepted reference test known as the gold standard.

The basic idea of diagnostic test interpretation is to calculate the probability a patient has a disease under consideration given a certain test result. A 2 by 2 table can be used for this purpose. Be sure to label the table with the test results on the left side and the disease status on top as shown here:

The sensitivity of a diagnostic test is the probability that a diseased individual will have a positive test result. Sensitivity is the true positive rate (TPR) of the test. Sensitivity = P(T+|D+)=TPR = TP / (TP+FN)

The specificityof a diagnostic test is the probability that a disease-free individual will have a negative test result. Specificity is the true negative rate (TNR) of the test. Specificity=P(T-|D-) = TNR =TN / (TN + FP).

FPR = P(T+|D-)= = FP / (FP+TN) False-positive rate: The likelihood that a nondiseased patient has an abnormal test result.

FNR = P(T-|D+)= = FN / (FN+TP) False-negative rate: The likelihood that a diseased patient has a normal test result.

Pretest Probabilityis the estimated likelihood of disease before the test is done. It is the same thing as prior probabilityand is often estimated. If a defined population of patients is being evaluated, the pretest probability is equal to theprevalence of disease in the population. It is the proportion of total patients who have the disease. P(D+) = (TP+FN) / (TP+FP+TN+FN)

Sensitivity and specificity describe how well the test discriminates between patients with and without disease. They address a different question than we want answered when evaluating a patient, however. What we usually want to know is: given a certain test result, what is the probability of disease? This is the predictive valueof the test.

Predictive value of a positive testis the proportion of patients with positive tests who have disease. PVP=P(D+|T+) = TP / (TP+FP) This is the same thing as posttest probability of disease given a positive test. It measures how well the test rules in disease.

Predictive value of a negative testis the proportion of patients with negative tests who do not have disease. In probability notation: PVN = P(D-|T-) = TN / (TN+FN) It measures how well the test rules out disease. This is posttest probability of non-disease given a negative test.

Evaluating a 2 by 2 table is simple if you are methodical in your approach.

General form of Bayes’ rule is Bayes’ Rule Method Using Bayes’ rule, PVP and PVN are defined as Bayes’ rule is a mathematical formula that may be used as an alternative to the back calculation method for obtaining unknown conditional probabilities such as PVP or PVN from known conditional probabilities such as sensitivity and specificity.

ExampleThe following table summarizes results of a study to evaluate the dexamethasone suppression test (DST) as a diagnostic test for major depression. The study compared results on the DST to those obtained using the gold standard procedure (routine psychiatric assessment and structured interview) in 368 psychiatric patients. What is the prevalence of major depression in the study group? For the DST, determine a-Sensitivity and specificity b-False positive rate (FPR) and false negative rate (FNR) c-Predictive value positive (PVP) and predictive value negative (PVN)

FNR=1-Sensitivity=1-0.391=0.609 FPR=1-Specificity=1-0.967=0.033

Validating tests against a gold standard • A test is valid if: • It detects most people with disorder (high Sen) • It excludes most people without disorder (high Sp) • a positive test usually indicates that the disorder is present (high PV+) • The best measure of the usefulness of a test is the LR: how much more likely a positive test is to be found in someone with, as opposed to without, the disorder

ROC (ReceiverOperatingCharacteristic ) CURVE • We want to be able to compare the accuracy of diagnostic tests. • Sensitivity and specificity are candidate measures for accuracy, but have some problems, as we’ll see. • ROC curves are an alternative measure We plot sensitivity against 1 – specificity to create the ROC curve for a test

ROC (Receiver Operating Characteristic ) CURVE The ROC Curve is a graphic representation of the relationship between sensitivity and specificity for a diagnostic test. It provides a simple tool for applying the predictive value method to the choice of a positivity criterion. ROC Curve is constructed by plottting the true positive rate (sensitivity) against the false positive rate (1-specificty) for several choices of the positivity criterion.

Plotting the ROC curve is a popular way of displaying the discriminatory accuracy of adiagnostic test for detecting whether or not a patient has a disease or condition. ROC methodology is derived from signal detection theory [1] where it is used to determine if an electronic receiver is able to satisfactory distinguish between signal and noise. It has been used in medical imaging and radiology , psychiatry , non-destructive testing andmanufacturing, inspection systems .

Specific Example Pts without the disease Pts with disease Test Result

Call these patients “negative” Call these patients “positive” Threshold Test Result

Call these patients “negative” Call these patients “positive” Some definitions ... True Positives Test Result without the disease with the disease

Call these patients “negative” Call these patients “positive” False Positives Test Result without the disease with the disease

Call these patients “negative” Call these patients “positive” True negatives Test Result without the disease with the disease

Call these patients “negative” Call these patients “positive” False negatives Test Result without the disease with the disease

Moving the Threshold: right ‘‘-’’ ‘‘+’’ Test Result without the disease with the disease

Moving the Threshold: left ‘‘-’’ ‘‘+’’ Test Result without the disease with the disease

ROC curve 100% True Positive Rate (sensitivity) 0% 100% 0% False Positive Rate (1-specificity)

RECEIVER OPERATING CHARACTERISTIC (ROC) curve • ROC curves (Receiver Operator Characteristic) • Ex. SGPT and Hepatitis Sensitivity 1 1 1-Specificity

ROC curve comparison 100% 100% True Positive Rate True Positive Rate 0% 0% 100% 100% 0% 0% False Positive Rate False Positive Rate A poor test: A good test:

ROC curve extremes 100% 100% True Positive Rate True Positive Rate 0% 0% 100% 100% 0% 0% False Positive Rate False Positive Rate Best Test: Worst test: The distributions don’t overlap at all The distributions overlap completely

‘Classical’ estimation • Binormal model: • X ~ N(0,1) in nondiseased population • X ~ N(a, 1/b) in diseased population • Then ROC(t) = (a + b-1(t)) for 0 < t < 1 • Estimate a, b by ML using readings from sets of diseased and nondiseased patients

ROC curve estimation with continuous data • Many biochemical measurements are in fact continuous, e.g. blood glucose vs. diabetes • Can also do ROC analysis for continuous (rather than binary or ordinal) data • Estimate ROC curve (and smooth) based on empirical ‘survivor’ function (1 – cdf) in diseased and nondiseased groups • Can also do regression modelingof the test result • Another approach is to model the ROC curve directlyas a function of covariates

The most commonly used global index of diagnostic accuracy is the area under the ROC curve (AUC).

DIAGNOSTIC TESTS