1 / 29

STATISTICAL EVALUATION OF DIAGNOSTIC TESTS

STATISTICAL EVALUATION OF DIAGNOSTIC TESTS. Describing the performance of a new diagnostic test.

presta
Download Presentation

STATISTICAL EVALUATION OF DIAGNOSTIC TESTS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STATISTICAL EVALUATION OF DIAGNOSTIC TESTS

  2. Describing the performance of a new diagnostic test Physicians are often faced with the task of evaluation the merit of a new diagnostic test. An adequate critical appraisal of a new test requires a working knowledge of the properties of diagnostic tests and the mathematical relationships between them.

  3. Limitations: 1) The gold standard is often the most risky, technically difficult, expensive, or impractical of available diagnostic options. 2) For some conditions, no gold standard is available. The gold standard test: Assessing a new diagnostic test begins with the identification of a group of patients known to have the disorder of interest, using an accepted reference test known as the gold standard.

  4. The basic idea of diagnostic test interpretation is to calculate the probability a patient has a disease under consideration given a certain test result.  A 2 by 2 table can be used for this purpose. Be sure to label the table with the test results on the left side and the disease status on top as shown here: 

  5. The sensitivityof a diagnostic test is the probability that a diseased individual will have a positive test result. Sensitivity is the true positive rate (TPR) of the test. Sensitivity = P(T+|D+)=TPR = TP / (TP+FN)

  6. The specificity of a diagnostic test is the probability that a disease-free individual will have a negative test result. Specificity is the true negative rate (TNR) of the test. Specificity=P(T-|D-) = TNR =TN / (TN + FP).

  7. FPR = P(T+|D-)= = FP / (FP+TN) False-positive rate: The likelihood that a nondiseased patient has an abnormal test result.

  8. FNR = P(T-|D+)= = FN / (FN+TP) False-negative rate: The likelihood that a diseased patient has a normal test result.

  9. Pretest Probability is the estimated likelihood of disease before the test is done. It is the same thing as prior probability and is often estimated. If a defined population of patients is being evaluated, the pretest probability is equal to theprevalence of disease in the population. It is the proportion of total patients who have the disease. P(D+) = (TP+FN) / (TP+FP+TN+FN)

  10. Sensitivity and specificity describe how well the test discriminates between patients with and without disease. They address a different question than we want answered when evaluating a patient, however. What we usually want to know is: given a certain test result, what is the probability of disease? This is the predictive value of the test.

  11. Predictive value of a positive testis the proportion of patients with positive tests who have disease. PVP=P(D+|T+) = TP / (TP+FP) This is the same thing as posttest probabilityof disease given a positive test. It measures how well the test rules in disease.

  12. Predictive value of a negative test is the proportion of patients with negative tests who do not have disease. In probability notation: PVN = P(D-|T-) = TN / (TN+FN) It measures how well the test rules out disease. This is posttest probability of non-disease given a negative test.

  13. Evaluating a 2 by 2 table is simple if you are methodical in your approach. 

  14. Bayes’ Rule Method General form of Bayes’ rule is Bayes’ rule is a mathematical formula that may be used as an alternative to the back calculation method for obtaining unknown conditional probabilities such as PVP or PVN from known conditional probabilities such as sensitivity and specificity. Using Bayes’ rule, PVP and PVN are defined as

  15. Example The following table summarizes results of a study to evaluate the dexamethasone suppression test (DST) as a diagnostic test for major depression. The study compared results on the DST to those obtained using the gold standard procedure (routine psychiatric assessment and structured interview) in 368 psychiatric patients. What is the prevalence of major depression in the study group? For the DST, determine a-Sensitivity and specificity b-False positive rate (FPR) and false negative rate (FNR) c-Predictive value positive (PVP) and predictive value negative (PVN)

  16. P(D+) =215/368 =0.584 Sensitivity = P(T+|D+)=TPR=TP/(TP+FN)=84/215=0.391 Specificity=P(T-|D-)=TNR=TN / (TN + FP)=148/153=0.967 FPR = P(T+|D-)=FP/(FP+TN)=5/153=0.033 FNR = P(T-|D+)=FN/(FN+TP)=131/215=0.609 PVP=P(D+|T+) = TP / (TP+FP)=84/89=0.944 PVN = P(D-|T-) = TN / (TN+FN)=148/279=0.53

  17. FNR=1-Sensitivity=1-0.391=0.609 FPR=1-Specificity=1-0.967=0.033

  18. ROC (Receiver Operating Characteristic ) CURVE The ROC Curve is a graphic representation of the relationship between sensitivity and specificity for a diagnostic test. It provides a simple tool for applying the predictive value method to the choice of a positivity criterion. ROC Curve is constructed by plottting the true positive rate (sensitivity) against the false positive rate (1-specificty) for several choices of the positivity criterion.

  19. Positivity criterion Disease-free Diseased 2 1 xi FN TP FP TN Test negative Test positive

  20. Example: One of the parameters which are evaluated for the diagnosis of CHD, is the value of “HDL/Total Cholesterol”. Consider a population consisting of 67 patients with CHD, 93 patients without CHD. The result of HDL/Total Cholesterol values of these two groups of patients are as follows.

  21. Descriptive Statistics HDL/Total Cholestrol Min Max GROUP Mean SD CHD- ,2926 ,066 ,16 ,52 CHD+ ,2301 ,048 ,06 ,34 • To construct the ROC Curve, we should find sensitivity and specificity for each cut off point. We have two alternatives to find these characteristics. • Cross tables • Normal Curve

  22. If HDL/Total Cholestrol is less than or equal to 0,26, we classify this group into diseased. Specificity Sensitivity

  23. Cutoff TPR FPR 0,000 0,000 0,000 0,093 0,015 0,000 0,129 0,030 0,000 0,142 0,045 0,000 0,156 0,060 0,000 0,158 0,075 0,000 0,162 0,075 0,011 0,168 0,104 0,011 0,171 0,119 0,011 0,173 0,119 0,022 0,175 0,119 0,032 . . . . . . . . . 0.26 0.78 0.31 . . . . . . 0,393 1,000 0,935 0,402 1,000 0,946 0,407 1,000 0,957 0,420 1,000 0,968 0,446 1,000 0,978 0,493 1,000 0,989 1,000 1,000 1,000 Let cutoff=0,171 Best cutoff point

  24. ROC Curve 1,0 ,9 Cutoff=0.26 TPR=0.78 FPR=0.31 TNR=0.69 FNR=0.22 ,8 ,7 ,6 ,5 Sensitivity ,4 ,3 ,2 ,1 1 - Seçicilik 0,0 0,0 ,1 ,2 ,3 ,4 ,5 ,6 ,7 ,8 ,9 1,0 1-Specificity

  25. FN TP FP TN If Patients with CHD and without CHD are normally distributed, we can easily find sensitivity and specificity from the area under these normal curves. Sensitivity and specificity are calculated for each different cotoff points CHD+ CHD- 0,23 0,29 Cutoff=0,28

  26. CHD+ CHD- 0,23 0,29 Cutoff=0,28 If we take cut off point=0.28, the characteristics of test are: ZCHD+=(0.28-0.23)/0.048=1.04 TPR=0.5+0.3508=0.8508 FNR=1-TPR=0.1492 ZCHD-=(0.28-0,29)/0.066=-0.15 TNR=0.5+0.0596=0.5596 FPR=1-TNR=0.4404

  27. Cutoff TPR FNR TNR FPR 0,10 0,00 1,00 1,00 0,00 0,15 0,05 0,95 0,98 0,02 0,20 0,27 0,73 0,91 0,09 0,25 0,66 0,34 0,73 0,27 0,28 0,85 0,15 0,56 0,44 0,30 0,93 0,07 0,44 0,56 0,35 0,99 0,01 0,18 0,82 0,45 1,00 0,00 0,00 1,00 1,00 0,90 0,80 0,70 0,60 TPR 0,50 0,40 0,30 0,20 0,10 0,00 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 FPR

More Related