Introduction to Teaching Evidence-based Health Care

1. Introduction to Teaching Evidence-based Health Care Sharon E. Straus MD MSc FRCPC Associate Professor, University of Toronto Knowledge Translation Program

2. Objectives • To outline a potential framework for teaching EBM • To describe how this framework can be used for evaluating our EBM educational initiatives • To discuss some of your objectives for this workshop

3. What is EBHC? • EBHC requires the integration of the best available research evidence with • our clinical expertise and • our patient’s unique values and circumstances

4. Its practice requires: • Asking • Acquiring • Appraising • Applying • Assessing

5. A framework for teaching EBHC and evaluating our efforts • Who is the learner? • What is the intervention? • What are the outcomes?

6. Who is the learner? • We must identify our learners, their needs and their learning styles • Learners include clinicians who want to practise EBHC and the patients they care for • Do all clinicians want or need to learn how to practise all 5 steps?

7. Who is the learner? • Targeted Clinicians: • EBHC Doers • EBHC Users • EBHC Replicators • The extent to which each of the 5 steps is performed is determined by: • The nature of the encountered condition • Time constraints • Level of expertise with each of the 5 skills

8. What is the intervention? • The 5 steps of practising EBHC – but what is the appropriate dose, formulation and method of delivery? • 1 minute or 60 hours • Journal clubs and/or freestanding courses • At the bedside, in the classroom or online

9. What is the intervention? • If our learners are interested in the ‘using’ mode, the intervention should focus on formulation of questions, searching for preappraised evidence and applying that evidence • If the learners are interested in the ‘doing’ mode, they should receive training in all 5 skills • The intervention should match the clinical setting, available time and other circumstances

10. What is the intervention? One approach doesn’t meet all our learners’ needs • Some studies use an approach to clinical practice and others use training in discrete microskills of EBHC • Review of graduate medical education found 18 reports of curricula and most commonly focused on critical appraisal • Some courses last 90 minutes, others weeks to months • Acad Med 1999;74:686-94 • Depending on the targeted learner, different skills emphasized

11. What are the relevant outcomes? • Attitudes • Knowledge • Skills • Behaviours • Clinical outcomes

12. What are the relevant outcomes? • Attitudes • There are several studies that have looked at attitudes towards EBM but little psychometric data available • Self-Directed Learning Readiness Scale can be used to assess readiness and is defined as the ‘degree to which the individual possesses the attitudes, abilities, and personality characteristics necessary for SDL’

13. What are the relevant outcomes? • Knowledge and Skills • Changes in clinicians’ knowledge and skills are relatively easy to detect and demonstrate • Several instruments developed to evaluate these • However, these instruments primarily focus on evaluating skills of clinicians who want to practise in the ‘doing’ mode rather than the ‘using’ mode

14. Effect of teaching strategies on critical appraisal skills • Review of 7 studies showed gain in knowledge (assessed by written test) in undergrads • Cochrane review identified 1 study that met inclusion criteria: • Critical appraisal course increased knowledge of critical appraisal • No studies found increased use of medical literature or change in other behaviours • CMAJ 1998;158:177-81; Cochrane Library; Update Software, Issue 3, 2005 (review updated, 2001 )

15. What are the relevant outcomes? • Behaviours • More difficult to measure because they require assessment in the practice setting • One study included videotaping of resident-patient interactions and analysing them for EBHC content • A recent before and after study found that a multi-component EBHC intervention significantly improved evidence-based practice patterns (JGIM, 2005) • Clinical Outcomes • The most difficult to measure

16. Consider your most recent EBM teaching experience: • Who was the learner, what was the intervention, what was the outcome • What worked during this session? • What didn’t work during this session?

17. The top 10 successes that we’ve had or seen in teaching EBM • Teaching EBM succeeds: • When it centers around real clinical decisions • When it focuses on learners’ actual learning needs • When it balances passive with active learning • When it connects new knowledge to old • When it involves everyone on the team

18. Top 10 successes • Teaching EBM succeeds: • When it matches and takes advantage of, the clinical setting, available time, and other circumstances • When it balances preparedness with opportunism • When it makes explicit how to make judgments, whether about the evidence itself or how to integrate evidence with other knowledge, clinical expertise and patient preferences • When it builds learners’ lifelong learning abilities

19. Top 10 mistakes we’ve made or see when teaching EBM • Teaching EBM fails: • When learning how to do research is emphasised over how to use it • When learning how to do statistics is emphasised over how to interpret them • When teaching EBM is limited to finding flaws in published research • When teaching portrays EBM as substituting research evidence for, rather than adding it to clinical expertise, patient values and circumstances

20. Top 10 mistakes we’ve made or see when teaching EBM • Teaching EBM fails: • When teaching with or about evidence is disconnected from the team’s learning needs about the patient’s illness or their own clinical skills • When teaching occurs at the speed of the teacher’s speech or mouse clicks rather than the pace of the learner’s understanding • When the teacher strives for full educational closure by the end of each session rather than leaving plenty to think about and learn between sessions

21. Top 10 mistakes we’ve made or see when teaching EBM • Teaching EBM fails: • When it humiliates learners for not already knowing the ‘right’ fact or answer • When it bullies learners to decide to act based on fear of others’ authority or power, rather than on authoritative evidence and rational argument • When the amount of teaching exceeds the available time or the learner’s attention

22. Have fun!

23. What are some barriers to teaching EBHC? • Time constraints – for teachers and learners • Lack of resources • Paucity of evidence that EBHC works

24. What can we do in 1 minute?

27. What can we do in 5 minutes?

31. Time constraints • Post-call rounds: • Learners: all members of the medical team • Objectives: decide on working diagnosis and initial therapy of newly admitted patients • Evidence of highest relevance: accuracy and precision of the clinical examination and other diagnostic tests; effectiveness and safety of therapy • Strategies/Intervention: demonstrate e-b exam, carry a PDA with synopses of evidence, write educational prescriptions, add a clinical librarian to the team

32. Morning Report • Learners: all members of the medical teams • Objectives: briefly review new patient(s) and discuss/debate diagnostic and management strategies • Evidence of highest relevance: accuracy and precision of diagnostic tests, effectiveness and safety of therapy • Strategies: educational prescriptions for foreground questions (CQ log), fact follow-ups for background questions, 1-2 minute summaries of critically appraised topics

33. Limited time and resources for EBHC Teachers • Educational sessions can target the different modes of practising EBHC • We can • Share educational materials • Share teaching tips (www.cma.ca/cmaj) • Share evaluation instruments • Development of evaluation clearinghouse/database • www.sgim.org/ebm.cfm

34. Paucity of Evidence that EBHC works • No evidence from RCTs showing impact on clinical outcomes • Evidence from process studies • Evidence from outcomes research

35. What’s the ‘E’ for EBHC? • Are we asking the right question? • Providing evidence from clinical research is necessary but not sufficient for the provision of optimal care • Changing behaviour is a complex process requiring comprehensive approaches directed towards patients, physicians, managers and policy makers • Provision of evidence is but one component • BMJ 2003;327:33-5

36. Outcomes research When cared for by evidence-based neurologists: • Patients with stroke 44% more likely to receive warfarin and more likely to be placed in a stroke unit • Patients were 22% less likely to die in the next 90 days • Stroke 1996;27:1937-43.

37. In a city-wide study of E-B practice vs. outcome in carotid stenosis: • Generated E-B indications for endarterectomy and reviewed 291 patients • Found the surgical indications • Appropriate in 33% • Questionable in 49% • Inappropriate in 18%

38. Stroke or expected death within the next 30 days: • Expected (if left alone) 0.5% • Expected (if appropriate selection) 1.5% • Observed among operated patients >5% Stroke 1997;28:891-8.