1 / 67

Diagnostic research

Diagnostic research. Delivered by Nia Kurniati. Lecture Contents. I. Diagnostics in practice - Explained with a case Scientific development of diagnostic research Design Data-analysis Reporting Exercises Summary. Diagnostics in practice.

caron
Download Presentation

Diagnostic research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Diagnostic research Delivered by Nia Kurniati

  2. Lecture Contents I. Diagnostics in practice - Explained with a case • Scientific development of diagnostic research • Design • Data-analysis • Reporting • Exercises • Summary

  3. Diagnosticsin practice Diagnostics always start with a patient with a complaint/symptom Case: neck stiffness • Child, 2 years-old, comes to ER with parents • Child turns out to have a very stiff neck What is the physician’s aim?

  4. Diagnostics in practice Aim of the physician • Quickly and efficiently determine the correct diagnosis Why diagnose? • Basis medical handling • Determines treatment choice • Gives information about prognosis What are possible diagnoses for neck stiffness?

  5. Diagnostics in practice Differential diagnosis (DD) • Bacterial meningitis • Viral meningitis • Pneumonia • ENT infection • Other (e.g. myalgia) What is the most important diagnosis? Which one does the physician not want to miss?

  6. Diagnostics in practice Most important diagnosis • Bacterial meningitis (BM) • If missed: often fatal

  7. Diagnostics in practice Suppose: 20% of all children on the ER with neck stiffness has BM 20% with disease in that population = prevalence ≈ Prior-probability What is your decision for the child in this case?

  8. Diagnostics in practice Decision for child in case • Prior-probability too low to treat • Prior-probability too high to send home Decision: reduce uncertainty diagnostics What is the best test?

  9. Diagnostics in practice Best test Lumbal punction (liquor culture)

  10. Diagnostics in practice Gold standard • True disease status; ‘truth’ • Never 24 karat • Reference standard/test • Decisive test with doubt Perform reference test for everybody (=every child on ER with neck stiffness)?

  11. Diagnostics in practice Reference test for everybody? • Unethical  too invasive/risky • Inefficient  too expensive • Do not perform unnecessarily How should we then determine the probability of disease presence and what would be ideal?

  12. Diagnostics in practice How then? • Simpler diagnostics: • Usually history taking, physical exam, simple lab tests, imaging, etc. • Ideal: diagnosis without reference test • Diagnostic process in practice: • Stepwise process: less  more invasive • Not one diagnosis based on 1 test • Each item: separate test

  13. Diagnostics in practice Suppose: after anamnesis & PE + 10% probability of BM • Probability of disease given test results = posterior-probability • The bigger the difference between prior and posterior probability, the better the diagnostic value of the tests Our decision for child in case: probability is too high to send home --> next step?

  14. Diagnostics in practice Next step • Additional research, e.g. blood tests (leucocytes, CRP, sedimentation, etc.)

  15. Diagnostics in practice Suppose: + 1% posterior-probability after anamnesis, PE+ simple lab testsposterior probability low enough to send home • Ideal diagnostic process:simple tests reduce posterior probability to 0 or 100% (without reference) • Most often physician continues testing until sufficiently sure (approximation of 0 or 100%) • Choose when sufficiently sure: depends on prognosis of disease if untreated + risks/costs treatment

  16. Diagnostics in practice Summarizing • What does diagnosing involve in practice? • Estimation of probability of disease presence based on test(s) results of the patient When is the probability of disease best estimated? Why is this usually not done?

  17. Diagnostics in practice Why not all possible tests? • Invasive (for patient and budget) • Unnecessary: different test results give same info • However: In practice often more tested than necessary! What diagnostics truly necessary scientific diagnostic research

  18. break

  19. Scientific development of diagnostic research

  20. Study design Scientific diagnostic research • What tests truly contribute to probability estimation? • Has to serve practice follow practice

  21. Study design • Research question • Domain • Study population • Determinant(s): test(s) to study • Endpoint: presence/absence disease (outcome) • Study design: design • Data analysis, interpretation + reporting

  22. Research question With as few as possible simple, safe, and cheap tests estimate the probability of the presence/absence of disease. Determinant-outcome relation: probability of disease as a function of test results outcome = probability of disease = % = prevalence test results = determinants

  23. Research question Case What tests contribute to probability estimation of presence or absence of BM in children with neck stiffness at the ER? Or: Determinants of presence/absence disease (BM)? %BM = ƒ(age, gender, fever, blood leucocytes, blood CRP, etc)

  24. Research population Case: All children with neck stiffness in 2002 at ER Utrecht

  25. Domain • For whom domain, generalisation = type of patient with certain symptom /complaint + setting • Research population = 1 sample from domain Case: All children (e.g. in Western world) suspected of disease (BM) based on neck stiffness (characteristic) in secondary care (setting)

  26. Determinants = Tests to study Diagnostic determinants All possible important tests (in domain) Case Items anamnesis, PE and lab (blood and urine) tests

  27. Endpoint ‘True’ presence/absence disease = Diagnostic outcome = Results reference test NB: reference = not infallible but always best available test in practice at that moment Case Positive liquor culture

  28. PICO EBM • Population/ problem • Intervention • Comparison/ control • Outcome • Domain • Determinant • Reference test • Outcome

  29. Measure determinants/endpoint Determinants Without knowledge (blinded) of the outcome Same method in study and practice never measure more precisely than in practice (overestimation information yield) Endpoint Assessment blind for determinants With the best possible test known in practice

  30. Study design Observational and descriptive Observational = no manipulation of determinants Descriptive = not causal if the determinant only predicts no hypothesis functional mechanism determinant-outcome >1 determinant

  31. Study design Cross-sectional = Simultaneously measure determinants and outcome

  32. Data-analysis After data collection, per patient Value determinants (test results) Diagnostic outcome (reference test)

  33. Data-analysis • Data analysis: 3 steps 1) Estimate prior probability (before additional test results) 2) Compare each test result separately with reference = univariate 3) Compare combination of test results with reference = multivariate (via model) - Following order in practice - Determine added value test result to already collected (previous) test results

  34. Data-analysis Case Data scientific research available: 200 patients with neck stiffness at ER Liquor culture positive (BM+) n=40 Liquor culture negative (BM-) n=160 Step 1: Prior probability (prevalence) of BM? = % BM+ = 40/ 200 patients = 20%

  35. Data-analysisreading 2 by 2 table • Step 2: Analysis per determinant (univariate) • Use 2 by 2 table

  36. Data-analysisreading 2 by 2 table Horizontally Positive predictive value (PV+) = probability Disease (+) if test (+) PV(+) = A / A + B Negative predictive value (PV-) = probability disease (-) if test (-) PV(-) = D / C + D Vertically Sensitivity (SE) = probability test (+) if disease (+) SE = A / A + C Specificity (SP) = probability test (-) if disease (-) SP = D / B + D What numbers do you think are most useful in practice (PV+ and PV- or SE and SP)? Gold standard Disease + Disease – TP A B FP Test + Test – FN C D TN

  37. Data-analysis BM+ BM- tot. 20 90 110 Perfect diagnostic test False Positive = 0 False Negative = 0 e.g. Fever > 380C as predictor for BM Yes (+) Fever > 380C No (-) 20 70 90 40 160 200

  38. Data-analysisreading 2 by 2 table Gold standard BM+ BM– 20 TP A 90 B FP Fever + Fever – FN C 20 D TN 70 Horizontally probability BM+ if fever+ = 20/110 = 18% PV+ = A / A + B probability BM - if fever- = 70/90 = 78% PV- = D / C + D Vertically probability fever+ if BM+ = 20/40 = 50% SE = A / A + C probability fever- if BM- = 70/160 = 44% SP = D / B + D What numbers do you think are most useful in practice (PV+ and PV- or SE and SP)?

  39. Data-analysis: combination of determinants In practice not one single diagnosis based on 1 test Tests together distinguish ill/non-ill Method: statistical model Moreover: diagnostic process is hierarchical (simple to invasive/expensive) therefore always start with anamnesis model --> see case

  40. Data-analysis Case: model with all anamnestic tests (gender + age + fever + pain) %BM = ƒ(gender, age, fever, pain) Statistical model can be seen as 1 (composed) test Quantify diagnostic value model with area under ROC curve (Receiver Operating Characteristic =Area Under Curve (AUC))

  41. Data-analysis

  42. Data-analysis Case: AUC model = 0,71 Informal interpretation AUC = % correctly diagnosed The larger the ROC area  the better the model AUC range: 0,5 1,0 AUC = 0,5  bad (Se = 1- Sp  diagonal [coin]) AUC > 0,7  reasonable AUC > 0,8  good AUC > 0,9  excellent AUC = 1,0  perfect (Se=100% & 1-Sp=0%)

  43. Data-analysis Quantify added value additional tests to previous tests Extend previous model (follow order practice) Quantify change in AUC Case Model 1 anamnesis model + physical exam (5 extra tests) --> AUC = 0,72  interpretation? Model 2 anamnesis model + 3 blood tests ---> AUC = 0,90  interpretation?

  44. Data-analysis

  45. Data-analysis • The AUC does not directly say anything about individual patients and is therefore not directly applicable

  46. Reporting Research question Study set-up Research population, setting, determinants, outcome, design Results Predictive values (new) test and/or ROC curve ROC curve combination of tests Added value new test --> ROC curve

  47. break

  48. Exercise 1 Mercury thermometer or timpanic membrane infrared meter to use for temperature measurement

  49. Exercise 1 Ad question 1 Research question: Can fever be determined with the TIM? Determinant: test under study = timpanic membrane infrared meter Outcome: fever determined with rectal mercury thermometer (RMT) Domain: Children in secondary/tertiary care (ER hospital)

  50. Exercise 1 Ad question 2 GS RMT Fever+ Fever– Se = probability TIM+ if RMT+ = 77/96 = 80 % SP = probability TIM- if RMT- = 108/117 = 92% 77 TP A 9 B FP TIM >38° TIM 38° FN C 19 D TN 108

More Related