1 / 102

Assessing Information from Multilevel and Continuous Tests

Assessing Information from Multilevel and Continuous Tests . Likelihood Ratios for results other than “+” or “-”. Michael A. Kohn, MD, MPP 10/2/2008. Four Main Points. 1) Dichotomizing a multi-level test by choosing a fixed cutpoint reduces the value of the test.

marrim
Download Presentation

Assessing Information from Multilevel and Continuous Tests

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Information from Multilevel and Continuous Tests Likelihood Ratios for results other than “+” or “-” Michael A. Kohn, MD, MPP 10/2/2008

  2. Four Main Points 1) Dichotomizing a multi-level test by choosing a fixed cutpoint reduces the value of the test. 2) The ROC curve summarizes the ability of the test to differentiate between D+ and D- individuals. 3) LR(result) = P(result|D+)/P(result|D-) = slope of ROC curve. (NOTE: Do not calculate an LR(+) or LR(-) for a multilevel test.) 4) Pre-Test Odds x LR(result) = Post-Test Odds

  3. NOTE Do not calculate an LR(+) or LR(-) for a test with more than two possible results.

  4. Additional Topics • Optimal Cutoffs • Walking Man • C Statistic

  5. Example from Chapter 3 65-year-old woman with mammogram suspicious for malignancy Pre-test probability ≈ 0.015 LR(“suspicious for malignancy”) ≈ 100 Post-test probability = ?

  6. Update Pre-Test Probability Using LR(test result) • Convert pre-test probability (P) to pre-test odds. Pre-Test Odds = P/(1-P) • Calculate LR. P(result|D+)/P(result|D-). • Post-Test Odds = Pre-Test Odds × LR • Convert post-test odds to post-test probability. Prob = Odds/(1+Odds)

  7. Update Pre-Test Probability Using LR(test result) 1) Pre-test probability P = 0.015 Pre-test odds = P/(1-P) ≈ 0.015 2) LR(“Suspicious for Malignancy”) = 100 3) Post-Test Odds = 0.015 × 100 = 1.5 4) Post-test probability = Odds/(1+Odds) = 1.5/2.5 = 0.60

  8. Can Use Slide Rule

  9. Can Use Excel

  10. Can Use Web-Based Calculator • We will come back to this (This ends the example for Chapter 3.)

  11. Evaluating the Test--Test Characteristics • For dichotomous tests, we discussed sensitivity P(+|D+) and specificity P(-|D-) • For multi-level and continuous tests, we will discuss the Receiver Operating Characteristic (ROC) curve

  12. Using the Test Result to Make Decisions about a Patient • For dichotomous tests, we use the LR(+) if the test is positive and the LR(-) if the test is negative • For multilevel and continuous tests, we use the LR(r), where r is the result of the test

  13. Septic Arthritis Bacterial infection in a joint.

  14. Clinical ScenarioDoes this Adult Patient Have Septic Arthritis?

  15. Clinical ScenarioDoes this Adult Patient Have Septic Arthritis? A 48-year-old woman with a history of rheumatoid arthritis who has been treated with long-term, low-dose prednisone presents to the emergency department with a 2-day history of a red, swollen right knee that is painful to touch. She reports no prior knee swelling and no recent trauma or knee surgery, illegal drug use, rash, uveitis, or risky sexual behavior. On examination, she is afebrile and has a right knee effusion. Her peripheral white blood cell (WBC) count is 11 000/µL and her erythrocyte sedimentation rate (ESR) is 55 mm/h. An arthrocentesis is performed, and the initial Gram stain is negative. You have the synovial white blood cell (WBC) count. Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  16. Clinical ScenarioDoes this Adult Patient Have Septic Arthritis? Assume pre-test probability of septic arthritis is 0.38. How do you use the synovial WBC result to determine the likelihood of septic arthritis? Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  17. Why Not Make It a Dichotomous Test? Synovial Septic Arthritis WBC Count Yes No >25,000 77% 27% ≤ 25,000 23% 73% TOTAL* 100% 100% *Note that these could have come from a study where the patients with septic arthritis (D+ patients) were sampled separately from those without (D- patients). Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  18. Why Not Make It a Dichotomous Test? Sensitivity = 77% Specificity = 73% LR(+) = 0.77/(1 - 0.73) = 2.9 LR(-) = (1 - 0.77)/0.73 = 0.32 “+” = > 25,000/uL “-” = ≤ 25,000/uL

  19. Clinical ScenarioSynovial WBC = 48,000/mL Pre-test prob: 0.38 LR(+) = 2.9 Post-Test prob = ?

  20. Clinical ScenarioSynovial WBC = 48,000/mL Pre-test prob: 0.38 Pre-test odds: 0.38/0.62 = 0.61 LR(+) = 2.9 Post-Test Odds = Pre-Test Odds x LR(+) = 0.61 x 2.9 = 1.75 Post-Test prob = 1.75/(1.75+1) = 0.64

  21. Clinical ScenarioSynovial WBC = 48,000/mLSlide Rule Pre-test prob: 0.38 LR(+) = 2.9 Post-Test prob = (Demonstrate Slide Rule)

  22. Can Use Excel Pre-test prob: 0.38 LR(+) = 2.9 Post-Test prob =

  23. Can Use Web-Based Calculator http://www.quesgen.com/Calculators/PostProdOfDisease/PostProdOfDisease.html P(+|D+) = Sensitivity = 77% P(+|D-) = 1 - Specificity = 1 - 73% = 27%

  24. Clinical ScenarioSynovial WBC = 128,000/mL Pre-test prob: 0.38 LR = ? Post-Test prob =?

  25. Clinical Scenario Synovial WBC = 128,000/mL Pre-test prob: 0.38 Pre-test odds: 0.38/0.62 = 0.61 LR = 2.9 (same as for WBC=48,000!) Post-Test Odds = Pre-Test Odds x LR(+) = 0.61 x 2.9 = 1.75 Post-Test prob = 1.75/(1.75+1) = .64

  26. Why Not Make It a Dichotomous Test? Because you lose information. The risk associated with a synovial WBC=48,000 is equated with the risk associated with WBC=128,000. Choosing a fixed cutpoint to dichotomize a multi-level or continuous test throws away information and reduces the value of the test.

  27. Main Point 1: Avoid Making Multilevel Tests Dichotomous Dichotomizing a multi-level or continuous test by choosing a fixed cutpoint reduces the value of the test

  28. Synovial Fluid WBC Count

  29. Histogram • Does not reflect prevalence of D+ (Dark D+ columns add to 100%, Open D- columns add to 100%) • Sensitivity and specificity depend on the cutpoint chosen to separate “positives” from “negatives” • The ROC curve is drawn by serially lowering the cutpoint from highest (most abnormal) to lowest (least abnormal).* * Just said that choosing a fixed cutpoint reduces the value of the test. The key issues are 1) the ROC curve is for evaluating the test, not the patient, and 2) drawing the ROC curve requires varying the cutpoint, not choosing a fixed cutpoint.

  30. Margaretten, M. E., J. Kohlwes, et al. (2007). Jama297(13): 1478-88.

  31. Cutoff ≥0 Cutoff > 25k Cutoff > 50k Cutoff > 100k Cutoff > ∞

  32. Test Discriminates Fairly Well Between D+ and D- D+ D- Test Result

  33. Test Discriminates Well Between D+ and D-

  34. Test Discriminates Poorly Between D+ and D- D+ D- Test Result

  35. Test Discriminates Poorly Between D+ and D-

  36. Area Under ROC Curve Cutoff ≥0 Cutoff > 25k Cutoff > 50k Area Under Curve = 0.8114 Cutoff > 100k Cutoff > ∞

  37. Area Under ROC Curve Summary measure of test’s discriminatory ability Probability that a randomly chosen D+ individual will have a more positive test result than a randomly chosen D- individual

  38. Area Under ROC Curve • Corresponds to the Mann-Whitney U Test Statistic (= Wilcoxon Rank Sum), which is the non-parametric equivalent of Student’s t test. • Also corresponds to the “c statistic” reported in logistic regression models

  39. Main Point 2ROC Curve Describes the Test, Not the Patient • Describes the test’s ability to discriminate between D+ and D- individuals • Not particularly useful in interpreting a test result for a given patient

  40. ROC Curve Describes the Test, Not the Patient Clinical Scenario Synovial WBC count = 48,000 Synovial WBC count = 128,000

More Related