1 / 29

Diagnostic Testing Brian Gage, MD October. 26, 2004 DOC Research Acknowledgment: http://www.cebm.utoronto.ca/ glossary/s

Diagnostic Testing Brian Gage, MD October. 26, 2004 DOC Research Acknowledgment: http://www.cebm.utoronto.ca/ glossary/spsncriteria.htm. Goals. Define terms used to describe diagnostic tests Understand concepts pertaining to Dx tests Set up a 2 x 2 table in Excel to do the math

megan
Download Presentation

Diagnostic Testing Brian Gage, MD October. 26, 2004 DOC Research Acknowledgment: http://www.cebm.utoronto.ca/ glossary/s

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diagnostic Testing Brian Gage, MD October. 26, 2004 DOC Research Acknowledgment: http://www.cebm.utoronto.ca/ glossary/spsncriteria.htm

  2. Goals • Define terms used to describe diagnostic tests • Understand concepts pertaining to Dx tests • Set up a 2 x 2 table in Excel to do the math • Use the table to evaluate V/Q scan for Dx of PE

  3. Uncertainty Prevails in Medicine “Physicians can do more to admit the existence of uncertainty, both to themselves and to their patients. Although this will undoubtedly be unsettling, it is honest, and it opens the way for a more intensive search for ways to reduce uncertainty.” DAVID M. EDDY, MD, PhD

  4. Example: You are contacted to help design a study of a D-Dimer test • Background: Plasma D-Dimer is a fibrin degradation product resulting from activation of coagulation and fibrinolysis • Accuracy of Roche Cardiac D-Dimer: • Area under the ROC curve (sens. vs. 1-spec): 0.89 • Intra-assay reproducibility, CV ~12% • Coefficient of Variation = SD/mean • Sensitivity (89%-100%); specificity 50%. • FDA approved for measurement of D-Dimer

  5. THE CLASSIC 2x2 TABLE TEST + TEST -

  6. Accuracy • Sensitivity • the proportion of diseased persons who have a positive test • also called the true positive rate and can be calculated from a/(a+c) • Specificity • the proportion of non-diseased persons who have a negative test • also called the true negative rate and can be calculated from d/(b+d)

  7. Technical Accuracy • Specificity: Remember SpPin When a test has a high Specificity, a Positive test rules IN the disorder. • Sensitivity: Remember SnNout When a test has a high Sensitivity, a Negative result rules OUT the disorder.

  8. Why aren’t Sensitivity & Specificity Affected by Disease Prevalence?

  9. Predictive Values • Positive Predictive Value • The proportion of patients with a positive test who have the disease • Also known as “post-test probability” or “posterior probability” following a positive test. • Negative Predictive Value • The proportion of patients with a negative test who don’t have the disease. • Are predictive values affected by prevalence?

  10. Likelihood Ratio and Odds Ratios • Similar to the concepts of “ruling in” and “ruling out” disease • Pre Test Odds x LR = Post Test Odds • Odds = P/(1-P) where P is the probability • E.g. If the probability is 25% what are the odds? 25%/(1-25%)= 1:3

  11. Likelihood Ratios • Allow many levels of interpretation for a “positive” test LR Meaning >10 Strong evidence to rule in a disease 5-10 Moderate evidence to rule in 0.5-2 Indeterminate 0.2-0.5 Weak evidence to rule out 0.1-0.2 Moderate evidence to rule out <0.1 Strong evidence to rule-out disease

  12. Odds Form of Bayes’ Theorem • Uses likelihood ratio (LR) • LR+ = sensitivity / (1-specificity) • LR - = (1-sensitivity)/specificity • Post-test odds = Pre-test odds x LR+ • Example of 2 x 2 table in Excel

  13. So, . . .the importance of “Bayes’ Theorem” • At low prevalence (e.g. primary care), even great tests can have significant false positives

  14. Valid Dx Tests • Measure test reproducibility, both inter- and intra-observer variability • A prerequisite for accuracy • Quantified with the Kappa statistic or correlation coefficient (r). • Include patients with and without the target disorder • Avoid spectrum bias • Interpret the new diagnostic test without knowledge of the gold standard • Double blinded if possible

  15. Example: an excellent test and a rare disease • The case of screening for muscular dystrophy (MD) at HH • Technical Precision of CPK test: • Sensitivity (ability to rule out disease): 100% • Specificity (ability to identify disease): 99.98% • But, the prevalence of MD is 1 in 5000 (0.02%)

  16. Does Baby J. have M.D.? Of 100,000 males, 20 will have M.D. (1 in 5,000, or 0.02% prevalence) • The test will correctly identify all 20 who have the disease (sensitivity = 100%)

  17. Does Baby J. have M.D.? • Of the 99,980 without M.D. • Specificity = 99.98% • 99,980 x 0.9998 = 99,960 will be negative • Therefore, false positives = 20

  18. The post test odds are 1:1 • Therefore, • Out of 100,000 infants, 20 will be truly positive and 20 will be false positive • Positive predictive value = 50% • An unselected baby boy with a positive screening test only has a 50/50 chance of having MD!

  19. Down’s Normal Karyotype Down’s Risk Normal Risk AFP Choosing the Threshold of a + Test Probability density function

  20. Another Example: Lyme Disease • Antibody assay • Sensitivity= 95%; specificity= 95% • High Lyme Disease prevalence (20%) • Positive predictive value = 83% • Low Lyme Disease prevalence (2%) • Positive predictive value = 28% Brown SL. JAMA 1999;282:62-6.

  21. Another Example: Mammography • Mammography in women between 40-50 yrs • If 100,000 low-risk women are screened: 6,034 mammograms will be abnormal 5,998 (99.4%) will be false-positive 36 will actually have breast cancer Why? Prevalence = 0.036% Hamm RM. J Fam Pract 1998;47:44-52.

  22. Summary • Accuracy of a diagnostic test can be summarized by two measures: sensitivity & specificity • These numbers determine likelihood ratios, LR • LR can be used on the odds form of Bayes’ formula to design a study or evaluate the post-test odds of disease • The threshold to call a test +, is a trade-off between sensitivity & specificity

  23. Example: You are contacted to help design a study of a D-Dimer test • Background: Plasma D-Dimer is a fibrin degradation product resulting from activation of coagulation and fibrinolysis • Accuracy of Roche Cardiac D-Dimer: • Area under the ROC curve (sens. vs. 1-spec): 0.89 • Intra-assay reproducibility, CV ~12% • Coefficient of Variation = SD/mean • Sensitivity (89%-100%); specificity 50%. • FDA approved for measurement of D-Dimer

  24. Questions to Ponder During Break: • If using the D-Dimer test to Dx a DVT or PE, should you target patients with a low, medium, or high pretest probability of disease? • Hints: • Use the pretest probabilities of PE from PIOPED, Table 6: 9%, 30%, 68% • D-Dimer is less accurate, but faster and cheaper than Doppler LE, spiral CT, V/Q scan, or angiogram • Or use the pretest probabilities from Wells clinical prediction rule for DVT: 3%, 17%, 75% • Wells PS et al. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet 1997;350:1796

  25. Question 2, to Ponder: • Question: How would you design the study to determine whether availability of the D-Dimer test in the ER or outpatient setting reduced the use of Doppler LE, V/Q scans, and/or spiral CT?

  26. Example 2: Use of D-Dimer post Joint Replacement • Suppose you’d like to use the D-Dimer test to determine how long to prescribe an anticoagulant after orthopedic surgery, a controversial area • You’d like to test the hypothesis that anticoagulant therapy can be stopped once the D-Dimer falls to a certain level. • You’re not sure what that level is because surgery can elevate the D-Dimer. • How would you design the study?

More Related