1 / 49

Performance of a diagnostic test

Performance of a diagnostic test. Steen Ethelberg 17th EPIET Introductory Course Lazareto, Menorca, Spain September 2011. Thierry Ancelle Marta Valenciano. Outline. Performance characteristics of a test Sensitivity Specificity Choice of a threshold.

romaine
Download Presentation

Performance of a diagnostic test

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance of a diagnostic test Steen Ethelberg 17th EPIET Introductory Course Lazareto, Menorca, Spain September 2011 • Thierry Ancelle • Marta Valenciano

  2. Outline • Performance characteristics of a test • Sensitivity • Specificity • Choice of a threshold. • Performance of a test in a population • Positive predictive value of a test (PPV) • Negative predictive value of a test (NPV) • Impact of disease prevalence, sensitivity and specificity on predictive values.

  3. 1. Performance of a test in an experimental setting

  4. Population with ill and non-ill individuals

  5. Test should identify the ill only

  6. In reality, tests are not perfect

  7. Sensitivity & specificity Identified patients All patients Se = = 90% 18 78 Identified non-patients 20 80 All non-patients Sp = = 97.5% Se = Sp =

  8. Sensitivity of a test • Ability of a test to correctly identify affected individuals • Proportion of people testing positive among affected individuals Patients + － True positive (TP) Test False negative (FN) Sensitivity (Se) = TP / ( TP + FN )

  9. Sensitivity of a PCR for congenital toxoplasmosis Patients with toxoplasmosis Sensitivity = 54 / 58 = 0.931= 93.1 %

  10. Specificity of a test • Ability of test to identify correctly non-affected individuals • Proportion of people testing negative among non-affected individuals Non-affected people + － False positive (FP) Test True negative (TN) Specificity (Sp) = TN / ( TN + FP )

  11. Specificity of a PCR for congenital toxoplasmosis Individuals without toxoplasmosis Specificity = 114 / 125 =0.912 = 91.2 %

  12. Performance of a test Disease Yes No + TP FP Test ­ FN TN TP Se = TP + FN TN Sp = TN + FP

  13. Distribution of quantitative test results among affected and non-affected people Threshold for positive result Ideal situation Non affected: Affected: Number of people tested TN TP 0 5 10 15 20 Quantitative result of the test

  14. Distribution of quantitative results among affected and non-affected people Threshold for positive result More realistic situation Non-affected: Affected: TN TP Number of people tested FN FP 0 5 10 15 20 Quantitative result of the test

  15. Effect of Decreasing the Threshold Non affected: Threshold for positive result Affected: FP Number of people tested TP TN FN 0 5 10 15 20 Quantitative result of the test

  16. Effect of Decreasing the Threshold Disease Yes No + TP FP Test ­ FN TN TP Se = TP + FN TN Sp = TN + FP

  17. Effect of Increasing the Threshold Non-affected: Threshold for positive result Affected: TN Number of people tested TP FN FP 0 5 10 15 20 Quantitative result of the test

  18. Effect of Increasing the Threshold Disease Yes No + TP FP Test ­ FN TN TP Se = TP + FN TN Sp = TN + FP

  19. Performance of a test and threshold • Sensitivity and specificity vary in opposite directions when changing the threshold (e.g. the cut-off in an ELISA) • The choice of a threshold is a compromise to best reach the objectives of the test • consequences of having false positives? • consequences of having false negatives?

  20. When false diagnosis is worse than missed diagnosis • Example: Screening for congenital toxoplasmosis • One should minimise false positives • Prioritise SPECIFICITY

  21. When missed diagnosis is worse than false diagnosis • Example: Testing for Helicobacter pylori infection • One should minimise the false negatives • Prioritise SENSITIVITY

  22. Using several tests • One way out of the dilemma is to use several tests that complement each other • First use test with a high sensitivity • Second use test with a high specificity

  23. ROC curves • Representation of relationship between sensitivity and specificity for a test • Receiver Operating Characteristics curve • Simple tool to: • Help define best cut-off value of a test • Compare performance of two tests.

  24. Prevention of Blood Transfusion Malaria:Choice of an Indirect IF Threshold Sensitivity (%) 100 1/10 1/20 1/40 80 1/80 1/160 60 IIF Dilutions 1/320 40 1/640 20 0 0 20 40 60 80 100 100% - Specificity (%)

  25. Comparison of Performance of ELISA and CATT Test for Screening of Human Trypanosomiasis Sensitivity (%) 100 80 ELISA CATT 60 40 20 0 0 25 50 75 100 100 - Specificity (%)

  26. Comparison of Performance of ELISA and CATT Test for Screening of Human Trypanosomiasis Sensitivity (%) 100 80 ELISA CATT 60 Area under the ROC curve (AUC) 40 20 0 0 25 50 75 100 100 - Specificity (%)

  27. Performance of a test • Validity • Sensitivity • Specificity • Reproducibility • Concepts may also used more broadly • Exposure status • Case definitions

  28. 2. Performance of a test in a population

  29. Would also like to know… • As a clinician • probability that a individual with a positive test is really sick? • probability that a individual with a negative test is really healthy? • As an epidemiologist • proportion of positive tests corresponding to true patients? • proportion of negative tests corresponding to healthy subjects?

  30. Predictive values Real patients Positive patients PPV= = 86% 15 Real non-patients 22 Negative patients 75 NPV= = 94% 80 PPV= NPV=

  31. Positive Predictive Value • Probability that an individual testing positive is truly affected • proportion of affected people among those testing positive Disease Yes No + Test TP FP PPV = TP/(TP+FP)

  32. Negative Predictive Value • Probability that an individual testing negative is truly non-affected • proportion of non affected among those testing negative Disease Yes No ­ Test FN TN NPV = TN/(TN+FN)

  33. TP FP Predictive value of a positive and a negative test Disease Yes No + PPV = TP/(TP+FP) Test ­ NPV = TN/(TN+FN) FN TN PPV = VPP = PV+ NPV = VPN = PV-

  34. Predicted values are not constants • The predicted values depend on the sensitivity and on the specificity of the test as well as on the prevalence of the disease • Will be different in different populations.

  35. TP FP Relation between predictive values and sensitivity/ specificity Disease Yes No + PPV = TP/(TP+FP) Test ­ NPV = TN/(TN+FN) FN TN

  36. Step 1: Specify the prevalence (Pr) of disease Disease Yes No + Test ­ Pr 1-Pr

  37. Step 2: Use sensitivity (Se) to distribute test results among the diseased Disease Yes No + Se Pr Test ­ (1-Se)Pr Pr 1-Pr

  38. Step 3: Use specificity (Sp) to distribute test results among the non-diseased Disease Yes No + (1-Sp)(1-Pr) Se Pr Test ­ (1-Se)Pr Sp(1-Pr) Pr 1-Pr

  39. Step 4: Determine the proportion testing positive and the proportion testing negative Disease Yes No + (1-Sp)(1-Pr) Se Pr + (1-Sp)(1-Pr) Se Pr Test ­ (1-Se)Pr Sp(1-Pr) (1-Se)Pr+ Sp(1-Pr) Pr 1-Pr

  40. Se Pr = PPV + - - Se Pr (1 Sp)(1 Pr) Sp(1 - Pr) = NPV + - Sp(1 - Pr) (1 Se) Pr Step 5: Calculate PPV and NPV with appropriate expressions from Step 4

  41. Relation between predictive values and sensitivity/ specificity Se Pr = PPV + - - Se Pr (1 Sp)(1 Pr) Sp(1 - Pr) = NPV + - Sp(1 - Pr) (1 Se) Pr Increasing specificity increasing PPV Increasing sensitivity increasing NPV

  42. Relation between predictive values and prevalence Se Pr = PPV + - - Se Pr (1 Sp)(1 Pr) Sp(1 - Pr) = NPV + - Sp(1 - Pr) (1 Se) Pr Increasing prevalence  increasing PPV Decreasingprevalence increasingNPV

  43. PPV and NPV of a test according to the prevalence (80% sensitivity and specificity) 100 80 NPV 60 Predictive value (%) 40 20 PPV 0 0 25 50 75 100 Prevalence (%)

  44. Example: Two different populations, Se=Sp=90% Prevalence: 50%  PPV = 90% Ill Not ill TP FP + Test FN TN ­ Prevalence: 10%  PPV = 50%

  45. Example: Screening for human trypanosomiasis in two settings • CATT test • Sensitivity = 95% • Specificity = 75% • Endemic area • Prevalence = 20% • Low endemic area • Prevalence = 0.5% • 100,000 tests performed in each area

  46. Example: Screening for human trypanosomiasis in two settings CATT test sensitivity = 95% CATT test specificity = 75% Prevalence = 20% PPV = 48.7% NPV = 98.4%

  47. Example: Screening for human trypanosomiasis in two settings CATT test sensitivity = 95% CATT test specificity = 75% Prevalence = 0.5% PPV = 1.90% NPV = 98.97%

  48. To sum up… • Sensitivity and specificity • intrinsic characteristics of a test • capacity to identify the affected • capacity to identify the non-affected • independent from the disease prevalence • Predictive values • performance of a test in real life • how to interpret a positive test • how to interpret a negative test • dependent on the disease prevalence

  49. Thank you!QUESTIONS?

More Related