Teaching Clinical Effectiveness

1. Teaching Clinical Effectiveness Robert Baldor, M.D. Frank J. Domino, M.D. Family Medicine and Community Health University of Massachusetts Medical School Worcester, MA

2. Clinical Effectiveness… …. is the extent to which specific clinical interventions do what they are intended to do, i.e. maintain and improve the health of patients, securing the greatest possible health gain from the available resources. B

3. To do…. • the right thing • at the right time • in the right place • with the right result B

4. Our Goal To appreciate the shift to ‘Comparative Effectiveness’ as it relates to teaching ‘Clinical Effectiveness’. B

5. By the end of this session, you will.. • Gain an appreciation for the recent emphasis on ‘Comparative Effectiveness‘  • Understand the basic concepts of comparative effectiveness and the relationship to our current understanding of evidenced based medicine • Discover where to find on-line resources to assist with teaching • Discuss methods for incorporating comparative effectiveness into the curriculum for your students and residents B

6. HMO’s Managed care Capitation Evidenced-based medicine Medical Homes Care management Global fees Comparative effectiveness? The more things change, the more they stay the same… B

7. Soaring health care costs…. • $2.2 trillion – 16% of GDP in 2007 • Without any changes in federal law, it will rise to 25 % of GDP in 2025. B

8. The American Recovery and Reinvestment Act (ARRA) of 2009 • Growing concern that we have little solid evidence of the value of many treatments • Hope to save money by discouraging the use of costly, ineffective treatments B

9. Comparative Effectiveness Research • Congress tasked the IOM to recommend priorities for research to be addressed by CER and supported by ARRA funds B

10. Perspective research… • Is it better to treat neck pain with surgery or PT & meds? • What is the best combination of counseling & meds to treat depression? • How do drugs compare with surgery to treat intermittent claudication? • Is it better to treat CHF by meds alone or by including home monitoring of BP and weight? B

11. A Teaching Perspective • Your MS III has just seen a patient and believes they have new onset depression. “I think he is pretty depressed and needs medication. On my Psychiatry rotation last month, citalopram was the SSRI of choice; they thought it worked better.” • What would you use for an SSRI and why? B

12. CER is done elsewhere… • Britian, France and others have official bodies that assess health technologies and compare the effectiveness, and sometimes the cost, of different treatments. B

13. The CER legislation became a lightning rod … • Republicans complained that the government would intrude in a person’s health by enforcing clinical guidelines. • Pharmaceutical and medical-device lobbyists fear the findings will be used to deny coverage for more expensive treatments and, thus, to ration care. B

14. Rationing ??? • Agreement that researchers should compare the clinical merits of different treatments. • Whether they should also consider cost is hotly debated. • The House Appropriations Committee inadvertently stoked rationing concerns in a report that stated ‘research comparing different treatments could yield significant payoffs because less effective, more expensive treatments will no longer be prescribed’ Sounds like you’re trying to kill Granny to me! B

15. Then of course there is Rush… • Bureaucrats “will monitor treatments to make sure your doctor is doing what the federal government deems appropriate and cost-effective”. What he really meant was to keep your hands off my drugs! B

16. The plan • The FDA regulates drugs and devices, but the goal is to establish if a treatment is safe and effective, not if it was better than the alternatives. The plan is for • systematic reviews of published scientific studies • clinical trials of head-to-head comparisons F

17. IOM definition • CER is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. How is this different from EBM? F

18. Your MS III has just seen a patient… ...and believes they have new onset depression “I think he is pretty depressed and needs medication. On my Psychiatry rotation last month, citalopram was the SSRI of choice; they thought it worked better.” • What would you use for an SSRI and why? F

19. How do you teach “what works better? Where would YOU look to find this data? • Pubmed/Clinical Queries? • In Cochrane? • In UpToDate? F

20. Will CER help us to answer those questions? B

21. IOM prioritization – a political process • Consulting stakeholders… • A public session - consumers, advocacy groups, provider groups, insurers, manufacturers, and academia. • A web-based questionnaire received more than 2,600 topics. B

22. The selection process:How an Idea becomes a CER No Yes No Yes B

23. Anybody in this room provide input? • What’s on your top 10 list? • For your learners? Is this different than your list • as practitioners? • as patients? F

24. Research methodology • RCTs felt to be the most appropriate methodology for ½ of the topics • Prospective observational studies, database reviews, and systematic literature reviews for the rest F

25. EBM Informational Pyramid Systematic Review Evidence Based Review Randomized Clinical Trial Cohort Study Prospective Case Control Study Retrospective Textbook (Harrison, Uptodate, etc.) F

26. CER USE many forms…. • Systematic literature reviews to summarize evidence, identify gaps, and generate new ideas for research • Randomized controlled trials - considered the gold standard of evidence, they also have shortcomings • Observational research using prospective registries and large established databases, including EHRs, are used for cohort studies to understand the natural progression of disease and the factors that influence clinical outcomes. F

27. The Forest Plot Estimates with 95% confidence intervals Line of no effect Kennedy 1997 Locke 1952A Estimate and confidence interval for each study Lopes 1997 Reynolds 1998 Estimate and confidence for the meta-analysis Seiberth 1994 Scale (effect measure) 0.2 1.0 5 Risk ratio Favours LR Favours control Direction of effect Forest Plot: If <> to your Left, Intervention was Effective at Lowering Risk of Outcome. F

28. FOREST PLOT Meta Analysis: Vitamin D on Falls 400-800 IU per day F

29. 100 topics, ranked by quartile….. • Collectively address broad societal needs. • Conditions with the greatest aggregate effect on the health of the U.S. population • Less common conditions that severely affect individuals in vulnerable subgroups of the population. (24 of the 100 topics) B

30. 1st Quartile medical examples…… • Medical Treatments • low back • ADHD treatment strategies • a-fib (surgery v catheter ablation v drugs) • ACEi/ARBs for CAD • Medical Diagnostics • Use of PET, MRI, CT for cancer diagnosing, staging • Use of genetic and biomarker testing B

31. 1st Quartile prevention home, community, delivery • Unintended pregnancies • Obesity, hypertension, diabetes in at-risk populations • Primary prevention (exercise and balance training) v clinical treatments in preventing falls • Management strategies (e.g., pharmacologic treatment, social/family support) for dementia in home and institutional settings. • Compare the medical home v. usual care in managing those with chronic disease. B

32. 4th quartile examples…. • Compare the effectiveness of: • long-term treatments for acne. • diagnostic imaging performed by non-radiologists and radiologists. B

33. Back to our student… • Your MS III has just seen a patient and believes they have new onset depression. “I think he is pretty depressed and needs medication. On my Psychiatry rotation last month, citalopram was the SSRI of choice; they thought it worked better.” • What would you use for an SSRI and why? • How do you teach “what works better”??? F

34. Lets answer that question together • What resources would your learners use… • What resources would YOU use? • What kind of a study is found (ie RCT, SR?) • So what’s the answer? F

35. OK lets take a break • When we come back – you’ll pick a clinical question related to clinical effectiveness and search for an answer

36. Appreciating EBM & CER • Pick a clinical question; either one you have on your mind or one from this list: • Treatment for Localized Prostate Cancer • Best Oral Rx for Type II DM • Choosing An Anti Depressant • ACEi or ARB for Hypertension • Best Pain Control in Osteoarthritis • Fracture Prevention in Osteoporosis • Best Management of Gestational Diabetes

37. Teaching Method for use of CER and EBM to young (and old) Learners • Pick a topic of their interest • Search using conventional methods of information acquisition • Find a high level of evidence Systematic Review on the topic • Look for a Comparative Effectiveness Review on the topic

38. Appreciating EBM & CER • Read a summary or review article on your topic (AFP, eMedicine, etc.) • Find a Randomized Controlled Trial using PubMed Clinical Queries Filterswww.pubmed.gov use CQ* • Find a Systematic Review on your question • Try to find a Comparative Effectiveness Review (AHRQ.gov**)

39. Using Pubmed Clinical Querieswww.pubmed.gov

40. Pubmed Clinical Queries Prostate cancer AND local

41. Appreciating EBM & CER • Read a summary or review article on your topic (AFP, eMedicine, etc.) • Find a Randomized Controlled Trial using PubMed Clinical Queries Filterswww.pubmed.gov use CQ* • Find a Systematic Review on your question • Try to find a Comparative Effectiveness Review (AHRQ.gov**)

42. Comparative Effectiveness Reviewswww.ahrq.gov

44. After our exercise… • What’s the curriculum for our students and residents? • What are the best methods to teach about clinical effectiveness? • Lecture? • On-line curriculum? • Workshops? • Others?