Challenges for Evidence-based Diagnosis & Course Review

1. Challenges for Evidence-based Diagnosis&Course Review Thomas B. Newman, MD, MPH Professor of Epidemiology and Biostatistics and Pediatrics, UCSF Challenges for EBD 3Dec08

2. Lecture Outline • Announcements • Criticisms of EBM • Challenges for EBD & rational decision making • Course review

3. Announcements • We hope you read chapter 12. • Optional reading on the website • Exam question contest winner • Take-home final will be handed out in section • Due Thursday, 12/11/08 at 8:45 AM, no exceptions • Your own work only, no collaboration • How many do not want paper copies? • We hope you enjoy it! • Next week: exam review and course eval. • Biostatistics final timing and section today

4. Criticisms of Evidence-Based Medicine -1 • EBM over-values randomized blinded trials and denigrates other forms of evidence, including clinical experience. • No systematic reviews of parachutes* • Historical background: EBM as a revolutionary questioning of authority • Oversimplification of problems, "evidence hierarchy," reliance on checklists *Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials BMJ  2003;327:1459-1461

5. Criticisms of Evidence-Based Medicine -2 • Evidence-based treatment recommendations tend towards the nihilistic • May not recommend tests or treatments that some clinicians, professional societies or disease-specific advocacy groups believe are effective. • Mammography screening; Bernadine Healy piece

6. Criticisms of Evidence-Based Medicine -3 • EBM has been or might be used by payers as an excuse to deny payment and limit clinician autonomy. • Perceived need to recommend tests and treatments for fear they will not be reimbursed if optional • Prostate cancer screening: Winners and Losers • Medicare cannot legally consider costs in coverage decisions • NICE controversy (NYT 12/3/08) • What is the alternative to EBM?

7. Stories and Statistics • People are more moved by individual stories than by statistics • "A single death is a tragedy; a million deaths is a statistic." * • Infant safety seats on airplanes • Requirement recommended by the AAP • FAA projected net increase in deaths due to diversion to cars • TBN et al estimated if this did NOT occur, cost to save 1 life ~$1.3 billion (@ $200/ticket) *Joseph Stalin

8. "The question, I think, Mr. Chairman, comes down to how many more children must die, how many more have to be hurt before we reach the threshold of FAA's ghoulish cost/benefit ratio?" --Congressman Jim Lightfoot, Iowa Congressional Testimony

9. “Real” vs “Theoretical” Children “The argument in support of the FAA’s resistance to the NTSB...is unreasonable on its face and ridiculous in its justification. It protects theoretical children driving in cars at the expense of real flesh-and blood infants whose safety is unquestionably compromised when flown as a lap-baby.” Nader R, Smith WJ. Collision course: the truth about airline safety. Blue Ridge Summit, PA: TAB Books, 1994. Cited by Beshai D. Arch Ped Adol med 2003;157:953-4

10. Challenges for evidence-based diagnosis • Difficulty estimating pre-test probabilities • Difficulty finding applicable likelihood ratios • Uncertainty about treatment thresholds • Clinicians are better at making decisions than at estimating any of the above

11. Heuristics Used in Probability Estimation • Representativeness • Availability • Adjustment from an anchor These heuristics can lead to biased estimates. See Chapter 12 for details.

12. Representativeness • If patient has typical features of a disease, we assign the disease high probability, even if prior probability was very low • Examples: • Chest pain radiating to the back  aortic dissection • “Worst headache of my life”  subarachnoid hemorrhage

13. Availability • The more easily you can imagine something happening, the higher the probability you assign it • Most recent and worst cases stand out • Examples: • Distorted view of risk of rare, serious illnesses from training in tertiary settings • TN distorted view of risk of kernicterus • (Consequences of error/regret factor into probability estimation)

14. Anchoring • People tend to estimate probabilities by starting some place and revising probability up or down • This happens even if anchor is irrelevant • Probability of pulmonary embolism study* • “Do you think it is > or <1%?” OR “Do you think it is > or < 90%?” • What do you think it is? • Average estimate 23% vs 53% *Brewer et al. Med Decis Making 2007;27:203-11

15. Another bias • Example: I have a rule that allows me to say whether any sequence of 3 numbers is golden • The series 2, 4, 6 is golden • Class suggest other series of 3 numbers and I’ll tell you if they are golden • See if you can guess my rule. * Mlodinow L. The Drunkard's Walk. Random House, 2008

17. Future of EBM • More tests and treatments • Greater competition for resources (potentially ugly) • More guidelines, algorithms, computer-aided decision making • Need for leaders to understand how these work, evaluate them critically, and help produce them!

18. Course Review (interactive)