1 / 111

Assessing Information from Multilevel and Continuous Tests

Assessing Information from Multilevel and Continuous Tests . Likelihood Ratios for results other than “+” or “-”. Michael A. Kohn, MD, MPP 10/7/2010. Four Main Points. 1) Making a multi-level test dichotomous by choosing a fixed cutpoint reduces the value of the test.

beth
Download Presentation

Assessing Information from Multilevel and Continuous Tests

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Information from Multilevel and Continuous Tests Likelihood Ratios for results other than “+” or “-” Michael A. Kohn, MD, MPP 10/7/2010

  2. Four Main Points 1) Making a multi-level test dichotomous by choosing a fixed cutpoint reduces the value of the test. 2) The ROC curve summarizes the ability of the test to differentiate between D+ and D- individuals. 3) LR(result) = P(result|D+)/P(result|D-) = slope of ROC curve. (NOTE: Do not calculate an LR(+) or LR(-) for a multilevel test.) 4) Pre-Test Odds x LR(result) = Post-Test Odds

  3. NOTE Do not calculate an LR(+) or LR(-) for a test with more than two possible results.

  4. Additional Topics • Walking Man • Weird ROC Curves • Optimal Cutoffs • C Statistic

  5. Example from Chapter 3 65-year-old woman with mammogram suspicious for malignancy Pre-test probability ≈ 0.015 LR(“suspicious for malignancy”) ≈ 100 Post-test probability = ?

  6. Update Pre-Test Probability Using LR(test result) • Convert pre-test probability (P) to pre-test odds. Pre-Test Odds = P/(1-P) • Calculate LR. P(result|D+)/P(result|D-). • Post-Test Odds = Pre-Test Odds × LR • Convert post-test odds to post-test probability. Prob = Odds/(1+Odds)

  7. Update Pre-Test Probability Using LR(test result) 1) Pre-test probability P = 0.015 Pre-test odds = P/(1-P) ≈ 0.015 2) LR(“Suspicious for Malignancy”) = 100 3) Post-Test Odds = 0.015 × 100 = 1.5 4) Post-test probability = Odds/(1+Odds) = 1.5/2.5 = 0.60

  8. Can Use Slide Rule

  9. Can Use Excel

  10. Can Use Web-Based Calculator • We will come back to this (This ends the example for Chapter 3.)

  11. Evaluating the Test--Test Characteristics • For dichotomous tests, we discussed sensitivity P(+|D+) and specificity P(-|D-) • For multi-level and continuous tests, we will discuss the Receiver Operating Characteristic (ROC) curve

  12. Using the Test Result to Make Decisions about a Patient • For dichotomous tests, we use the LR(+) if the test is positive and the LR(-) if the test is negative • For multilevel and continuous tests, we use the LR(r), where r is the result of the test

  13. Septic Arthritis Bacterial infection in a joint.

  14. Clinical ScenarioDoes this Adult Patient Have Septic Arthritis?

  15. Clinical ScenarioDoes this Adult Patient Have Septic Arthritis? A 48-year-old woman with a history of rheumatoid arthritis who has been treated with long-term, low-dose prednisone presents to the emergency department with a 2-day history of a red, swollen right knee that is painful to touch. She reports no prior knee swelling and no recent trauma or knee surgery, illegal drug use, rash, uveitis, or risky sexual behavior. On examination, she is afebrile and has a right knee effusion. Her peripheral white blood cell (WBC) count is 11 000/µL and her erythrocyte sedimentation rate (ESR) is 55 mm/h. An arthrocentesis is performed, and the initial Gram stain is negative. You have the synovial white blood cell (WBC) count. Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  16. Clinical ScenarioDoes this Adult Patient Have Septic Arthritis? Assume pre-test probability of septic arthritis is 0.38. How do you use the synovial WBC result to determine the likelihood of septic arthritis? Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  17. Why Not Make It a Dichotomous Test? Synovial Septic Arthritis WBC Count Yes No >25,000 77% 27% ≤ 25,000 23% 73% TOTAL* 100% 100% *Note that these could have come from a study where the patients with septic arthritis (D+ patients) were sampled separately from those without (D- patients). Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  18. Why Not Make It a Dichotomous Test? Sensitivity = 77% Specificity = 73% LR(+) = 0.77/(1 - 0.73) = 2.9 LR(-) = (1 - 0.77)/0.73 = 0.32 “+” = > 25,000/uL “-” = ≤ 25,000/uL

  19. Clinical ScenarioSynovial WBC = 48,000/mL Pre-test prob: 0.38 LR(+) = 2.9 Post-Test prob = ?

  20. Clinical ScenarioSynovial WBC = 48,000/mL Pre-test prob: 0.38 Pre-test odds: 0.38/0.62 = 0.61 LR(+) = 2.9 Post-Test Odds = Pre-Test Odds x LR(+) = 0.61 x 2.9 = 1.75 Post-Test prob = 1.75/(1.75+1) = 0.64

  21. Clinical ScenarioSynovial WBC = 48,000/mLSlide Rule Pre-test prob: 0.38 LR(+) = 2.9 Post-Test prob = (Demonstrate Slide Rule)

  22. Can Use Excel Pre-test prob: 0.38 LR(+) = 2.9 Post-Test prob =

  23. Can Use Web-Based Calculator http://www.quesgen.com/PostProbofDisease.php P(+|D+) = Sensitivity = 77% P(+|D-) = 1 - Specificity = 1 - 73% = 27%

  24. Clinical ScenarioSynovial WBC = 128,000/mL Pre-test prob: 0.38 LR = ? Post-Test prob =?

  25. Clinical Scenario Synovial WBC = 128,000/mL Pre-test prob: 0.38 Pre-test odds: 0.38/0.62 = 0.61 LR = 2.9 (same as for WBC=48,000!) Post-Test Odds = Pre-Test Odds x LR(+) = 0.61 x 2.9 = 1.75 Post-Test prob = 1.75/(1.75+1) = .64

  26. Why Not Make It a Dichotomous Test? Because you lose information. The risk associated with a synovial WBC=48,000 is equated with the risk associated with WBC=128,000. Choosing a fixed cutpoint to dichotomize a multi-level or continuous test throws away information and reduces the value of the test.

  27. Main Point 1: Avoid Making Multilevel Tests Dichotomous Dichotomizing a multi-level or continuous test by choosing a fixed cutpoint reduces the value of the test

  28. “LR Table” Used to create a histogram like the next slide.

  29. Synovial Fluid WBC Count

  30. Histogram • Does not reflect prevalence of D+ (Dark D+ columns add to 100%, Open D- columns add to 100%) • Sensitivity and specificity depend on the cutpoint chosen to separate “positives” from “negatives” • The ROC curve is drawn by serially lowering the cutpoint from highest (most abnormal) to lowest (least abnormal).* * Just said that choosing a fixed cutpoint reduces the value of the test. The key issues are 1) the ROC curve is for evaluating the test, not the patient, and 2) drawing the ROC curve requires varying the cutpoint, not choosing a fixed cutpoint.

  31. ROC Table Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  32. ROC Table

  33. ROC Table

  34. ROC Table ROC Table -- thresholds not results; sensitivity and 1 - specificity not proportions; columns don’t sum to 100%. Compare to LR Table on next slide.

  35. LR Table -- test results not thresholds; proportions not sensitivities or 1 - specificity; columns total to 100%)

  36. ROC Table used to create an ROC curve like the one on the next slide.

  37. Cutoff ≥0 Cutoff > 25k Cutoff > 50k Cutoff > 100k Cutoff > ∞

  38. Test Discriminates Fairly Well Between D+ and D- D+ D- Test Result

  39. Test Discriminates Well Between D+ and D-

  40. Test Discriminates Poorly Between D+ and D- D+ D- Test Result

More Related