1 / 66

Evidence Based Diagnosis Mark J. Pletcher, MD MPH 6/28/2012

Combining Tests. Evidence Based Diagnosis Mark J. Pletcher, MD MPH 6/28/2012. Acknowledgements. For this lecture I’ve adapted a slide set from Mike Kohn. Combining Tests. Overview A case with 2 simple tests Test non-independence Approaches to combining tests

chava-mendoza
Download Presentation

Evidence Based Diagnosis Mark J. Pletcher, MD MPH 6/28/2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Combining Tests Evidence Based Diagnosis Mark J. Pletcher, MD MPH 6/28/2012

  2. Acknowledgements For this lecture I’ve adapted a slide set from Mike Kohn

  3. Combining Tests Overview A case with 2 simple tests Test non-independence Approaches to combining tests Looking at all possible combinations of results Recursive partitioning, logistic regression, other Overfitting and validation in multitest panels

  4. Combining Tests – A Case Case Pregnant woman getting prenatal care, worried about Down’s Syndrome (Trisomy 21) Chorionic Villus Samping (CVS) is a definitive test, but there is a risk of miscarriage Should she get this procedure?

  5. Combining Tests – A Case Age helps… Risk goes up with age Our patient is 41, so pretest risk is ~2%...

  6. Combining Tests – A Case • Ultrasound can help even more • It’s harmless • Several features predict Trisomy 21 (Down’s) at 11-14 weeks* • Nuchal translucency • Nasal bone absence How do we use these two features together? *Cicero, S., G. Rembouskos, et al. (2004). "Likelihood ratio for trisomy 21 in fetuses with absent nasal bone at the 11-14-week scan." Ultrasound Obstet Gynecol23(3): 218-23.

  7. Combining Tests – A Case • First, nuchal translucency (NT)

  8. Wider translucent “gap” here is predictive of Down’s

  9. Nuchal Translucency Data Cross-sectional study 5556 Pregnant Women undergoing CVS 333 (6%) with Trisomy 21 fetus All had ultrasound at 11-14 weeks

  10. Dichotomize here for purposes of illustration

  11. Nuchal Translucency Data Trisomy 21 Nuchal D+ D- Translucency ≥ 3.5 mm (+) 212 478 690 < 3.5 mm (-) 121 4745 4866 Total 333 5223 Sensitivity and Specificity? PPV and NPV?

  12. Nuchal Translucency • Sensitivity = 212/333 = 64% • Specificity = 4745/5223 = 91% and IF we assume that this cross-sectional sample represents our population of interest, then: • Prevalence = 333/(333+5223) = 6% • PPV = 212/(212 + 478) = 31% • NPV = 4745/(121 + 4745) = 97.5%

  13. Nuchal Translucency Data Trisomy 21 Nuchal D+ D- Translucency ≥ 3.5 mm 212 478 690 < 3.5 mm 121 4745 4866 Total 333 5223 LR+ and LR-?

  14. Nuchal Translucency Data Trisomy 21 Nuchal D+ D- LR Translucency ≥ 3.5 mm 212 478 < 3.5 mm 121 4745 Total 333 5223 LR+ = P(T+|D+)/P(T+|D-) LR- = P(T-|D+)/P(T-|D-)

  15. Nuchal Translucency Data Trisomy 21 Nuchal D+ D- LR Translucency ≥ 3.5 mm 212 478 7.0 < 3.5 mm 121 4745 0.4 Total 333 5223 LR+ = (212/333)/(478/5223) = 7.0 LR- = (121/333)/(4745/5223) = 0.4

  16. Back to the case… • Let’s apply this data to our case, with pre-test probability of 2%

  17. Post-test risk using NT only Pre-test prob: 0.02 at age 41 Pre-test odds: 0.02/0.98 = 0.0204 IF TEST IS POSITIVE - LR = 7.0 Post-Test Odds = Pre-Test Odds x LR(+) = 0.0204 x 7.0 = 0.143 Post-Test prob = 0.143/(0.143 + 1) = 12.5%

  18. Post-test risk using NT only Pre-test prob: 0.02 at age 41 Pre-test odds: 0.02/0.98 = 0.0204 IF TEST IS NEGATIVE - LR = 0.4 Post-Test Odds = Pre-Test Odds x LR(+) = 0.0204 x 0.4 = 0.0082 Post-Test prob = 0.0082/(0.0082 + 1) = .8%

  19. Back to the case… • Is .8% risk low enough to not get CVS? • Is 12.5% risk high enough to risk CVS? • OTHER Ultrasound features are also predictive • Nasal bone absence

  20. Nasal Bone Seen NBA=“No” Neg for Trisomy 21 Nasal Bone Absent NBA=“Yes” Pos for Trisomy 21

  21. Nasal Bone Absence Test Data Nasal Bone Tri21+ Tri21- LR Absent Yes 229 129 27.8 No 104 5094 0.32 Total 333 5223

  22. Post-test risk using NBA only Pre-test prob: 0.02 at age 41 Pre-test odds: 0.02/0.98 = 0.0204 IF TEST IS POSITIVE - LR = 27.8 Post-Test Odds = Pre-Test Odds x LR(+) = 0.0204 x 27.8 = .567 Post-Test prob = .567/(.567 + 1) = 36%

  23. Post-test risk using NBA only Pre-test prob: 0.02 at age 41 Pre-test odds: 0.02/0.98 = 0.0204 IF TEST IS NEGATIVE - LR = 0.32 Post-Test Odds = Pre-Test Odds x LR(+) = 0.0204 x 0.32 = 0.0065 Post-Test prob = 0.0065/(0.0065 + 1) = .6%

  24. Back to the case… • NBA is a bit better than NT, but still important uncertainty… • Can we combine our NT results with NBA results and do even better? • How do we combine test results?

  25. Combining tests • Approach #1 – Assume independence • Knowing results of one test doesn’t influence how you interpret the next test • We usually assume LR is independent of pre-test probability • This is what we did when we used a pre-test risk of 2% instead of 6% in our calculations • If so, we can just do the calculations sequentially

  26. Assuming test independence First do NT, assume it’s positive (LR = 7) Pre-test risk Post-test risk 2%  12.5% Then do NBA, assume it’s also positive (LR = 23.7) Pre-test risk Post-test risk 12.5%  77%

  27. Assuming test independence What’s the mathematical shortcut? LR(1) * LR(2) = LR(1&2) 7 * 27.8 = 195

  28. Assuming test independence What’s the mathematical shortcut? LR(1) * LR(2) = LR(1&2) NT NBA LR + + 195 + - 2.2 - + 11.2 - - 0.13

  29. Assuming test independence Slide rule approach (pre-test prob = 6%) Line arrows up without shrinkage

  30. Combining tests • Is it reasonable to assume independence? • Does nasal bone absence tell you as much if you already know that the nuchal translucency is >3.5 mm? • What can we do to figure this out?

  31. Combining tests • Approach #2 – evaluate all possible test result combinations

  32. Joint eval of 4 test result combinations Vs. 195 2.2 11.2 .13 If tests were independent…

  33. Combining tests The Answer – the tests are NOT completely independent So we CANNOT just multiply LR’s What should we do in this case? Use LR’s from the combination table

  34. Joint eval of 4 test result combinations Use these!

  35. Create ROC Table

  36. AUROC = 0.896

  37. Optimal Cutoff Analysis • If we assume: • Pre-test probability = 6% • Threshold for CVS = 2% • Optimal algorithm is “any positive test  CVS”

  38. Non-independence • What does non-independence mean?

  39. Non-independence The total arrow length is NOT equal to the sum of its parts! Slide rule approach (pre-test prob = 6%)

  40. Non-independence • Technical definition of independence - must condition on disease status: • If this stringent definition is not met, the tests are non-independent In patients with disease, a false negative on Test 1 does not affect the probability of a false negative on Test 2. In patients without disease, a false positive on Test 1 does not affect the probability of a false positive on Test 2.

  41. Non-independence • Reasons for non-independence? • Tests measure the same aspect of disease. • Simple example: predicting pneumonia • Cyanosis: LR = 5 • O2 sat 85%-90%: LR = 6 • Can’t just multiply these LR’s because they really just reflect the same physiologic state!

  42. Non-independence • Reasons for non-independence? • Tests measure the same aspect of disease. • In our example: • One aspect of Down’s syndrome is slower fetal development; the NT decreases more slowly AND the nasal bone ossifies later. Chromosomally NORMAL fetuses that develop slowly will tend to have false positives on BOTH the NT Exam and the Nasal Bone Exam.

  43. Non-independence • Other reasons for non-independence? • Disease is heterogeneous • In severe pneumonia, all tests tend to be abnormal, so each individual test tells you less • O2 sat and respiratory rate • Non-disease is heterogeneous • In patients with cough but no pneumonia, abnormal tests may still track together • 02 sat and respiratory rate also both abnormal with PE; and both are normal with viral URI See EBD page 158

  44. Back to the case… • Remember that we actually simplified the case: Nuchal translucency is really a continuous test. • How do we take into account actual continuous NT measurement and NBA (and age, race, fetal crown-rump length, etc)?

  45. Back to the case… • Can’t do combination table for all possible combinations! • 2 dichotomous tests = 4 combinations • 4 dichotomous tests = 16 combinations • 3 3-level tests = 27 combinations • How do we deal with continuous tests?

  46. Combining tests • Approach #3: Recursive partitioning • Repeatedly split the data to find optimal testing/decision algorithm • “prune” the tree

  47. Combining tests • Approach #3: Recursive partitioning

  48. Combining tests • Approach #3: Recursive partitioning Non-optimal test ordering

  49. Combining tests • Approach #3: Recursive partitioning You might do nasal bone test first, then “prune”

  50. Combining tests • Approach #3: Recursive partitioning Final algorithm: do Nasal Bone exam first, stop if absence and do CVS…

More Related