1 / 76

Medical Epidemiology

Medical Epidemiology. Interpreting Medical Tests and Other Evidence. Interpreting Medical Tests and Other Evidence. Dichotomous model Developmental characteristics Test parameters Cut-points and Receiver Operating Characteristic (ROC) Clinical Interpretation

deyoung
Download Presentation

Medical Epidemiology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Medical Epidemiology Interpreting Medical Tests and Other Evidence

  2. Interpreting Medical Tests and Other Evidence • Dichotomous model • Developmental characteristics • Test parameters • Cut-points and Receiver Operating Characteristic (ROC) • Clinical Interpretation • Predictive values: keys to clinical practice • Bayes’ Theorem and likelihood ratios • Pre- and post-test probabilities and odds of disease • Test interpretation in context • True vs. test prevalence • Combination tests: serial and parallel testing • Disease Screening • Why everything is a test!

  3. Dichotomous model Simplification of Scale • Test usually results in continuous or complex measurement • Often summarized by simpler scale -- reductionist, e.g. • ordinal grading, e.g. cancer staging • dichotomization -- yes or no, go or stop

  4. Dichotomous model • Test Errors from Dichotomization • Types of errors • False Positives = positive tests that are wrong = b • False Negatives = negative tests that are wrong = c

  5. Developmental characteristics: test parameters Error rates as conditional probabilities • Pr(T+|D-) = False Positive Rate (FP rate) = b/(b+d) • Pr(T-|D+) = False Negative Rate (FN rate) = c/(a+c)

  6. Developmental characteristics: test parameters Complements of error rates as desirable test properties • Sensitivity = Pr(T+|D+) = 1 - FN rate = a/(a+c) Sensitivity is PID (Positive In Disease) [pelvic inflammatory disease] • Specificity = Pr(T-|D-) = 1 - FP rate = d/(b+d) Specificity is NIH (Negative In Health) [national institutes of health]

  7. Typical setting for finding Sensitivity and Specificity • Best if everyone who gets the new test also gets “gold standard” • Doesn’t happen • Even reverse doesn’t happen • Not even a sample of each (case-control type) • Case series of patients who had both tests

  8. Setting for finding Sensitivity and Specificity • Sensitivity should not be tested in “sickest of sick” • Should include spectrum of disease • Specificity should not be tested in “healthiest of healthy” • Should include similar conditions.

  9. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy

  10. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Healthy Sick

  11. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 20%True pos=82%

  12. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Fals pos= 9%True pos=70%

  13. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) F pos= 100% T pos=100%

  14. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) F pos= 50% T pos=90%

  15. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Receiver Operating Characteristic (ROC)

  16. Developmental characteristics: Cut-points and Receiver Operating Characteristic (ROC) Receiver Operating Characteristic (ROC)

  17. Receiver Operating Characteristic (ROC) • ROC Curve allows comparison of different tests for the same condition without (before) specifying a cut-off point. • The test with the largest AUC (Area under the curve) is the best.

  18. Developmental characteristics: test parameters Problems in Assessing Test Parameters • Lack of objective "gold standard" for testing, because • unavailable, except e.g. at autopsy • too expensive, invasive, risky or unpleasant • Paucity of information on tests in healthy • too expense, invasive, unpleasant, risky, and possibly unethical for use in healthy • Since test negatives are usually not pursued with more extensive work-ups, lack of information on false negatives

  19. Clinical Interpretation: Predictive Values Most test positives below are sick. But this is because there are as many sick as healthy people overall. What if fewer people were sick, relative to the healthy?

  20. Clinical Interpretation: Predictive Values Now most test positives below are healthy. This is because the number of false positives from the larger healthy group outweighs the true positives from the sick group. Thus, the chance that a test positive is sick depends on the prevalence of the disease in the group tested!

  21. Clinical Interpretation: Predictive Values • But • the prevalence of the disease in the group tested depends on whom you choose to test • the chance that a test positive is sick, as well as the chance that a test negative is healthy, are what a physician needs to know. • These are not sensitivity and specificity! • The numbers a physician needs to know are the predictive values of the test.

  22. Clinical Interpretation: Predictive Values Sensitivity (Se) Pr{T+|D+} true positives total with the disease Positive Predictive Value (PV+, PPV) Pr{D+|T+} true positives total positive on the test

  23. Positive Predictive Value • Predictive value positive • The predictive value of a positive test. • If I have a positive test, does that mean I have the disease? • Then, what does it mean? • If I have a positive test what is the chance (probability) that I have the disease? • Probability of having the disease “after” you have a positive test (posttest probability) • (Watch for “OF”. It usually precedes the denominator Numerator is always PART of the denominator)

  24. Clinical Interpretation: Predictive Values D+ T+ T+ and D+

  25. Clinical Interpretation: Predictive Value Specificity (Sp) Pr{T-|D-} true negatives total without the disease Negative Predictive Value (PV-, NPV) Pr{D-|T-} true negatives total negative on the test

  26. Negative Predictive Value • Predictive value negative • If I have a negative test, does that mean I don’t have the disease? • What does it mean? • If I have a negative test what is the chance I don’t have the disease? • The predictive value of a negative test.

  27. Mathematicians don’t Like PV- • PV- “probability of no disease given a negative test result” • They prefer (1-PV-) “probability of disease given a negative test result” • Also referred to as “post-test probability” (of a negative test) • Ex: PV- = 0.95 “post-test probability for a negative test result = 0.05” • Ex: PV+ = 0.90 “post-test probability for a positive test result = 0.90”

  28. Mathematicians don’t Like Specificity either • They prefer false positive rate, which is 1 – specificity.

  29. Where do you find PPV? • Table? • NO • Make new table • Switch to odds

  30. Use This Table ? NO

  31. Make a New Table

  32. Make a New Table

  33. Switch to Odds • 1000 patients. 100 have disease. 900 healthy. Who will test positive? • Diseased 100__X.95 =_95 Healthy 900 X.08 = 72 • We will end with 95+72= 167 positive tests of which 95 will have the disease • PPV = 95/167

  34. From pretest to posttest odds • Diseased 100 X.95 =_95 Healthy 900 X.08 = 72 • 100 = Pretest odds 900 • .95 = Sensitivity__ = prob. Of pos test in dis .08 1-Specificity prob. Of pos test in hlth • 95 =Posttest odds. Probability is 95/(95+72) 72

  35. Remember to switch back to probability

  36. What is this second fraction? • Likelihood Ratio Positive • Multiplied by any patient’s pretest odds gives you their posttest odds. • Comparing LR+ of different tests is comparing their ability to “rule in” a diagnosis. • As specificity increases LR+ increases and PPV increases (Sp P In)

  37. Clinical Interpretation: likelihood ratios • Likelihood ratio = Pr{test result|disease present} Pr{test result|disease absent} • LR+ = Pr{T+|D+}/Pr{T+|D-} = Sensitivity/(1-Specificity) • LR- = Pr{T-|D+}/Pr{T-|D-} = (1-Sensitivity)/Specificity

  38. Clinical Interpretation: Positive Likelihood Ratio and PV+ O = PRE-TEST ODDS OF DISEASE POST-ODDS (+) = O x LR+ =

  39. Likelihood Ratio Negative • Diseased 100_ X.05 =_5__ Healthy 900 X.92 = 828 • 100 = Pretest odds 900 • .05 = 1-sensitivity = prob. Of neg test in dis .92 Specificity prob. Of neg test in hlth (LR-) • Posttest odds= 5/828. Probability=5/833=0.6% • As sensitivity increases LR- decreases and NPV increases (Sn N Out)

  40. Clinical Interpretation: Negative Likelihood Ratio and PV- POST-ODDS (-) = O x LR- =

  41. Remember to switch to probability and also to use 1 minus

  42. Post test probability given a negative test = Post odds (-)/ 1- post odds (-)

  43. Prevalence (Probability) = 5% Sensitivity = 90% Specificity = 85% PV+ = 24% PV- = 99% Test not as useful when disease unlikely Prevalence (Probability) = 90% Sensitivity = 90% Specificity = 85% PV+ = 98% PV- = 49% Test not as useful when disease likely Value of a diagnostic test depends on the prior probability of disease

  44. Clinical interpretation of post-test probability Disease ruled out Disease ruled in

  45. Advantages of LRs • The higher or lower the LR, the higher or lower the post-test disease probability • Which test will result in the highest post-test probability in a given patient? • The test with the largest LR+ • Which test will result in the lowest post-test probability in a given patient? • The test with the smallest LR-

  46. Advantages of LRs • Clear separation of test characteristics from disease probability.

  47. Likelihood Ratios - Advantage • Provide a measure of a test’s ability to rule in or rule out disease independent of disease probability • Test A LR+ > Test B LR+ • Test A PV+ > Test B PV+ always! • Test A LR- < Test B LR- • Test A PV- > Test B PV- always!

  48. Using Likelihood Ratios to Determine Post-Test Disease Probability

More Related