1 / 71

Critiquing for Evidence-based Practice: Diagnostic and Screening Tests

Critiquing for Evidence-based Practice: Diagnostic and Screening Tests. M8120 Columbia University Fall 2001 Suzanne Bakken, RN, DNSc. Overview. Probability fundamentals Clinical role of probability revision Characterizing new information (test performance) Probability revision methods

roman
Download Presentation

Critiquing for Evidence-based Practice: Diagnostic and Screening Tests

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Critiquing for Evidence-based Practice: Diagnostic and Screening Tests M8120 Columbia University Fall 2001 Suzanne Bakken, RN, DNSc

  2. Overview • Probability fundamentals • Clinical role of probability revision • Characterizing new information (test performance) • Probability revision methods • Bayes’ formula • Contingency table • In class exercises

  3. Probability Fundamentals • Strength of belief • A number between 0 and1 that expresses an opinion about the likelihood of an event • Probability of an event that is certain to occur is 1 • Probability of an event that is certain to NOT occur is 0

  4. Summation Principle • Probability that an event will occur plus probability that it will not occur equals 1 • Probability of all possible outcomes of a chance event is always equal to 1 • Blood type: What is p[AB] given p[O]=0.46, p[A]=.40, and p[B]=.10? • Fraternal triplets: What is the probability of at least one boy and one girl?

  5. Type O (0.46) Type A (0.40) Type B (0.10) Type AB (0.04) Diagramming Probabilities Chance node Sum of probabilities at chance node = 1

  6. Conditional Probability • Probability that event A occurs given that event B is known to occur • p[AlB] • p[A,B]=p[AlB] X p[B] • Examples in health care

  7. Components of Probability Estimates • Personal experience • Published experience - evidence • Attributes of the patient

  8. Role of Probability Revision Techniques Prior Probability Posterior Probability Before Finding After Finding Abnormal Finding 1 0 Probability of Disease Diagnosis

  9. Role of Probability Revision Techniques Posterior Probability Prior Probability After Finding Before Finding Negative Finding 1 0 Probability of Disease Diagnosis

  10. Findings • Signs • Symptoms • Diagnostic tests • Probabilistic relationships between findings and disease basis of diagnostic decision support systems • Dxplain • QMR • Iliad

  11. Definitions • Prior probability - the probability of an event before new information (finding) is acquired; pretest probability or risk • Posterior probability - the probability of an event after new information (finding) is acquired; posttest probability or risk • Probability revision - taking new information into account by converting prior probability to posterior probability

  12. Review of Conditional Probability Probability that an event is true, given that another event is true p[AlB] p[A and B] = p[AlB] p[B] What is the probability that someone has HIV antibody given a positive HIV test? What is the probability that someone has venothrombosis given a swollen calf? What is the probability that someone has dyspnea given the nurse says dyspnea is present?

  13. Characterizing “Test” Performance • Compare test against gold standard (e.g., presence of disease; established test) • Ideal test - no values at which the distribution of those with the disease and without the disease overlap • Few tests ideal so … • TP • TN • FP • FN

  14. Test Performance • True positive rate (TPR) p[+lD] = probability of an abnormal test result given that the disease is present; the number of persons WITH the disease who have an abnormal test result divided by the number of persons WITH the disease; sensitivity

  15. Contingency Table View • Disease present = TPR + FNR = 1 • Disease absent = FPR + TNR = 1

  16. Test Performance • False positive rate (FPR) p[+lno D] = probability of an abnormal test result given that the disease is absent: the number of persons WITHOUT the disease who have an abnormal test result divided by the number of persons WITHOUT the disease

  17. Contingency Table View • Disease present = TPR + FNR = 1 • Disease absent = FPR + TNR = 1

  18. Test Performance • True negative rate (TNR) p[-lno D] = probability of a normal test result given that the disease is absent; number of persons WITHOUT the disease who have a normal test result divided by number of persons WITHOUT the disease; 1 - FPR or specificity; 100% specificity = pathognomonic

  19. Contingency Table View • Disease present = TPR + FNR = 1 • Disease absent = FPR + TNR = 1

  20. Test Performance • False negative rate (FNR) p[-l D] = probability of a normal test result given that the disease is present; number of persons WITH the disease who have a normal test result divided by number of persons WITH the disease; 1 - TPR

  21. Contingency Table View • Disease present = TPR + FNR = 1 • Disease absent = FPR + TNR = 1

  22. Sensitivity vs. Specificity • Weighing sensitivity Vs. specificity in setting cutoff level for abnormality in a test • Consequences of FPR vs. FNR • Severity of disease • Availability of treatment • Risk of treatment • Sensitivity and specificity are characteristics of a test and a criterion for abnormality

  23. 1.0 0.5 TPR ( sensitivity) 0 0.5 1.0 FPR (1 - specificity) Receiver Operating Characteristic (ROC) Curves Increased p[D] Decreased p[D]

  24. Contingency Table View • Disease present = TPR + FNR = 1 • Disease absent = FPR + TNR = 1

  25. Example of HIV What is the TPR? What is the FPR? What is specificity?

  26. TP TPR = TP + FN Example of HIV TPR = sensitivity p[+lD]

  27. Example of HIV FPR = p[+lno D] FP FPR = FP + TN

  28. Example of HIV TNR = specificity p[-lno D] TN TNR = TN + FP

  29. Nurse and Patient Rating of Symptoms • By definition, patient is gold standard for symptom rating • RN as “test” for presence or absence of symptom • Fatigue • RN yes/Pt yes = 50 • RN yes/Pt no = 15 • RN no/Pt yes = 20 • RN no/Pt no = 15 • What is sensitivity? What is specificity?

  30. Nurse and Patient Ratings of Symptoms: Sensitivity TP TPR = TP + FN

  31. Nurse and Patient Ratings of Symptoms: Specificity TN TNR = TN + FP

  32. Moving from Test Characteristics to Predictive Value and Posterior Probabilities • Predictive value • Forms of Bayes’ • Bayes’ formula • Contingency table view • Likelihood ratio

  33. Prevalence • Frequency of disease in the population of interest at a given point in time

  34. Predictive Value • Sensitivity, specificity, and their complements (FNR & FPR) focus on probability of findings given presence or absence of disease so not in a clinically useful form • Predictive value focuses on probability of disease given findings • Predictive value takes prevalence of disease in study population into account

  35. Positive Predictive Value The fraction of persons with an abnormal finding who have the disease number of persons with disease with abnormal finding PV+ = number of persons with abnormal finding TP PV+ = TP + FP

  36. Negative Predictive Value The fraction of persons with an normal finding who DO NOT have the disease number of persons with normal finding WITHOUT disease PV- = number of persons with normal finding TN PV+ = TN + FN

  37. Nurse and Patient Rating of Symptoms • By definition, patient is gold standard for symptom rating • RN as “test” for presence or absence of symptom • Shortness of Breath • RN yes/Pt yes = 25 • RN yes/Pt no = 10 • RN no/Pt yes = 15 • RN no/Pt no = 50 • What is PV+? What is PV-?

  38. Nurse and Patient Ratings of Symptoms: PV+ TP PV+ = TP + FP

  39. Nurse and Patient Ratings of Symptoms: PV- TN PV- = TN + FN

  40. Role of Probability Revision Techniques Prior Probability Posterior Probability Before Finding After Finding Abnormal Finding 1 0 Probability of Disease Diagnosis

  41. Calculating Posterior Probability with Bayes’ • What is the probability that someone has HIV antibody given a positive HIV test? • Can calculate with Bayes’ if you know: • Prior probability of the disease • Probability of an abnormal test (+) result conditional upon the presence of the disease (TPR) • Probability of an abnormal test (+) result conditional upon the absence of the disease (FPR)

  42. Bayes’ Theorem p[D] x p[+lD] p[Dl+] = {p[D] x p[+lD]} + {p[no D] x p[+lno D]} Not a clinically useful form!

  43. p[Dl+] p[+,D] = p[+] p[+,D] p[Dl+] = p[+,D] + p[+, no D] Deriving Bayes’ Theorem Given definition of conditional probability Given definition of conditional probability: 4 p[+,D] 1 p[+lD] = p[D] and p[+,no D] Summation principle: p[+lno D] = p[no D] p[+] = p[+,D] + p[+,no D] Given principle of conditional independence, rearrange expressions above: 2 p[+,D] = p[D] x p[+lD] p[+,no D] = p[no D] x p[+lnoD] 5 Thus: Substitute into 1: 3 p[D] x p[+lD] 6 p[Dl+] = p[D] x p[+lD] +p[no D] x p[+lnoD]

More Related