1 / 55

How to Use an Article About a Diagnostic Test

How to Use an Article About a Diagnostic Test. Akbar S oltani. MD , Endocrinologist Tehran University of Medical Sciences (TUMS) Endocrine and Metabolism Research Center (EMRC) Evidence-Based Medicine Research Center (EBMRC) Shariati Hospita l www.soltaniebm.com www.ebm.ir

cale
Download Presentation

How to Use an Article About a Diagnostic Test

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How to Use an Article About a Diagnostic Test Akbar Soltani. MD, Endocrinologist Tehran University of Medical Sciences (TUMS) Endocrine and Metabolism Research Center (EMRC) Evidence-Based Medicine Research Center (EBMRC) Shariati Hospital www.soltaniebm.com www.ebm.ir www.avincennact.ir

  2. OBJECTIVES • Objectives of testing • Diagnostic research • Critical appraisal • Summary

  3. 2/3 legal claims against GPs in UK 40,000-80,000 US hospital deaths from misdiagnosis per year Adverse events, negligence cases, serious disability more likely to be related to misdiagnosis than drug errors Diagnosis uses <5% of hospital costs, but influences 60% of decision making

  4. Objectives of testing • Increasing certainty of the presence or absence of disease • This requires sufficient discriminative power. • 2×2 table relating test outcome to a reference standard.

  5. Objectives of testing • Supporting clinical management • Determining localization, and shape of arterial lesions is necessary for treatment decisions

  6. Objectives of testing • Assessing prognosis As the starting point for clinical follow up and informing patients. • Monitoring clinical course When a disease is untreated, or during or after treatment. • Measuring fitness For example, for sporting activity or for employment.

  7. OBJECTIVES • Objectives of testing • Diagnostic research • Critical appraisal • Summary

  8. Evidence Based Approach How to Use an Article About a Diagnostic Test

  9. Appraising diagnostic tests: 3 easy steps 1. Are the results valid? 2. What are the results? 3. How Can the Results be Applied to Patient Care?

  10. Some definitions Disease Present Absent Positive Test Result Negative Sensitivity = A / (A+C) Specificity = D / (B+D)

  11. Tip….. • Sensitivity is useful to me • ‘The new rapid chlamydia test was positive in 47 out of 56 women with chlamydia (sensitivity =83.9%)’ • Specificity seems a bit confusing! • ‘The new rapid chlamydia test was negative in 600 of the 607 women who did not have chlamydia (specificity = 98.8%)’ • So…the false positive rate is sometimes easier • False positive rate = 1 – specificity • So a specificity of 98.8% means that the new rapid test is wrong (or falsely positive) in 1.2% of women

  12. Some definitions Disease Present Absent Positive Test Result Negative Sen*prevalence Sen*prev+(1-Spec) *(1-prev) PPV = A / (A+B) Spec*(1-prev) (1-Sen)*prev+ Spec*(1-prev) NPV = D / (C+D)

  13. Basic structure of diagnostic studies Series of patients Index test Reference (“gold”) standard Compare the results of the index test with the reference standard, blinded

  14. Appraising diagnostic tests: 3 easy steps 1. Are the results valid? 2. What are the results? 3. How Can the Results be Applied to Patient Care?

  15. What were the key selection (inclusion & exclusion) criteria? • Were they replicable? • List important selection criteria; e.g. age group, gender, risk profile, medical history. • There should be sufficient information in the paper to allow the reader to theoretically select a similar population

  16. Did selection lead to an appropriate spectrum of participants (like those assessed in practice) • Participants with the range of common presentations of the target disorder and with commonly confused diagnosis

  17. 1. Was an appropriate spectrum of patients included? Spectrum bias • You want to find out how good chest X rays are for diagnosing pneumonia in the Emergency Department • Best = all patients presenting with difficulty breathing get a chest X-ray • Spectrum bias = only those patients in whom you really suspect pneumonia get a chest X ray

  18. Spectrum bias • Studies from referral centers • Patients with negative results are less likely to referred

  19. 2) Did the results of the test being evaluated influence the decision to perform the reference standard? 2. Were all patients subjected to the gold standard? Verification (work-up) bias • What was the gold standard of diagnosis? • The validity of the study requires that there is an accepted, valid and replicable reference standard of diagnosis.

  20. gold standard 1.Laboratory tests (Infectious & endocrine diseases) 2.Imaging (DVT, PTE ) 3.Biopsy (Cancer , vasculitis) 4.Autopsy (neurologic diseases) 5.long-term follow-up (SLE , MS)

  21. 2) Did the results of the test being evaluated influence the decision to perform the reference standard? 2. Were all patients subjected to the gold standard? Verification (work-up) bias • You want to find out how good is exercise ECG (“treadmill test”) for identifying patients with angina • The gold standard is angiography • Best = all patients get angiography • Verification (work-up bias) = only patients who have a positive exercise ECG get angiography

  22. 3. Was there an independent, blind or objective comparison with the gold standard? Observer bias • You want to find out how good is exercise ECG (“treadmill test”) for identifying patients with angina • All patients get the gold standard (angiography) • Observer bias = the Cardiologist who does the angiography knows what the exercise ECG showed (not blinded) • Another example: The pulmonary nodule on CT, and comparison to CXR

  23. 4) Were the methods for performing the test described in sufficient detail to permit replication? • This description should cover all issues that are important in the preparation of the patient (diet, drugs to be avoided, precautions after the test), the performance of the test (technique, possibility of pain), and the analysis and interpretation of its results.

  24. Basic structure of diagnostic studies Series of patients Index test Reference (“gold”) standard Compare the results of the index test with the reference standard, blinded

  25. Read this abstract Scan in UTI abstract

  26. Scan in UTI abstract Gold standard Accuracy Series of patients Index test

  27. Women presenting with history of recurrent UTIs Series of patients Self diagnosis based on symptoms Index test Positive urine culture Reference (“gold”) standard Compare the results of the index test with the reference standard 172 episodes: Positive urine culture in 144 (84%)

  28. Do reports meet the standards? • Between 1978 and 1993 the authors found 112 articles, predominantly in radiological tests and immunoassays. • Few of the standards were met consistently - ranging from 46% avoiding workup bias down to 9% reporting accuracy in subgroups.

  29. Appraising diagnostic tests: 3 easy steps 1. Are the results valid? 2. What are the results? 3. How Can the Results be Applied to Patient Care?

  30. What are the Results? Biostatistics Review • Sensitivity : Of all the people with a particular disease, the proportion who will test positive for it (PID) • Specificity : Of all the people without the disease, the proportion who will test negative for it (NIH) A gold standard diagnosis is already known, (presumably without error). The above terms are describing the properties of a particular TEST.

  31. What are the Results? Biostatistics Review Example: Suppose the Prevalence of a particular disease in a population is 10%. (of 1000 people, 100 have the disease) Sensitivity: 90/100 = 90% Specificity: 810/900 = 90%

  32. What are the Results? Biostatistics Review • Positive Predictive Value : Of all the people who tested positive for a disease, the proportion that actually has it • Negative Predictive Value : Of all the people who tested negative for a disease, the proportion that actually does not have it In these patients, what you know are their test results, from which you are trying to determine whether they actually have the disease.

  33. What are the Results? Biostatistics Review Same example: Prevalence of a particular disease in the population is 10%. PPV: 90/180 = 50% NPV: 810/820 = 99% Sens: 540/600 = 90% Spec: 360/400 = 90%

  34. What are the Results? Note: The PPV and NPV of a test are DEPENDENT on prevalence. Suppose the Disease Prevalence is 60% Sens: 540/600 = 90% Spec: 360/400 = 90% PPV: 540/580 = 93% NPV: 360/420 = 86%

  35. What are the Results? The use of PPV and NPV when describing a diagnostic test has several drawbacks: • They are dependent on the prevalence of disease • The prevalence of disease in the general population is not the same as that of the patients you see in clinic/ER. • Not all tests have results that can be categorized as “+” or “-”. For these reasons, PPV and NPV may soon be considered “Old School”.

  36. In My Experience! Likelihood of disease and the PPV of diagnostic tests in specialty setting A test with 98% sen &sp Primary care setting p=2% ppv=50% Specialty setting p=30% ppv=95%

  37. Try it • A disease with a prevalence of 4% must be diagnosed. • It has a sensitivity of 50% and a specificity of 90%. • If the patient tests positive, what is the chance they have the disease?

  38. Prevalence of 4%, Sensitivity of 50%, Specificity of 90% 11.6 people test positive………. of whom 2 have the disease So, chance of disease is 2/11.6 about 17% Sensitivity = 50% Disease +ve 2 4 100 Testing +ve 9.6 96 Disease -ve False positive rate = 10%

  39. Prevalence of 4%, Sensitivity of 50%, Specificity of 90% • Doctors with an average of 14 yrs experience ….answers ranged from 1% to 99% ….half of them estimating the probability as 50% Gigerenzer G BMJ 2003;327:741-744

  40. What are the Results? Likelihood Ratios Simply stated, the Likelihood Ratio is how much more likely someone is to get a positive test result if they have the disease, compared to someone who doesn’t. In fact, the LR is how much more likely someone is to get any particular test result (positive, negative, “intermediate probability, etc) if they have disease, compared to someone who doesn’t.

  41. What are the Results? From our previous example: The LR is (90/100) / (90/900) = 9.0 This is the LR for a (+) result. Someone with disease is 9 times as likely to test positive than someone without it. Note also that : LR = SENS / 1 - SPEC WOWO

  42. What are the Results? The LR for a negative test is: (10/100) / (810/900) = 0.11 Someone with disease is 0.11 times as likely (1/9) to test negative than someone without it.

  43. Likelihood Ratios Post-Test Probability LR = 18 High Pre-Test Probability Post-Test Prob. 90% Mr. A Pre-Test Prob. 15% Mrs. B Pre-Test Prob. 40% Post-Test Prob. 20% LR = 1.2 V/Q Scan Results

  44. Application: Using Post-test Probability Above this point, treat Below this point, no further testing Disease ruled IN Determined by: Complications of untreated disease Risks of therapy Complications of tests Cost Disease not ruled in or out Disease ruled OUT

  45. What do likelihood ratios mean? LR>10 = strong positive test result LR=1 No diagnostic value LR<0.1 = strong negative test result

  46. What do likelihood ratios mean? McGee: Evidence based Physical Diagnosis (Saunders Elsevier)

  47. % Bayesian reasoning Pre test 5% Post test 20% ? Appendicitis: McBurney tenderness LR+ = 3.4 Fagan nomogram %

More Related