A Study of the Gatekeepers to Improve Impact of CPD/CME Programs

1. A Study of the Gatekeepers to Improve Impact of CPD/CME Programs Society for Academic Continuing Medical Education New York April 9, 2011 Elizabeth A. Lindsay PhD Scientist , Department of Family Medicine, Cross-appointed, Associate Professor Epidemiology and Community Medicine Academy for Innovation in Medical Education and The Office of CME University of Ottawa & Paul Hendry MD Assistant Dean, Continuing Medical Education University of Ottawa

2. Disclosure I have no actual or potential conflict of interest in relation to this workshop.

3. Session Learning Objectives Through attending this session, participants will be able to: • Identify the components of the CIHR “Knowledge to Action Framework”. • Name specific elements of behavior change theory that can support a study of program development competencies. • Identify gaps that may need to be addressed to improve the quality of program planning that occurs for traditional group learning

4. Objectives of the Study To gather information regarding the knowledge, beliefs and intentions of CPD program planners related to recommended best practices for developing and implementing CPD programs. Through a study of physicians who plan and lead “group learning” programs to assess their: Knowledge of recommended best practices for accredited group learning programs. Perceived importance of the “best practices” recommended changes Belief in the potential impact of making the changes Perception of support for the changes from peers Perception of the personal, professional and environmental factors that can influence the changes.

5. Study Participants • Approximately 45 physicians participated in planning committees over an 18 month period. • Family physicians and other specialists Usual Practice for Planning One-day Programs • 2 face-to- face meetings before event • E-mail communication to finalize faculty, titles & objectives • 1 debriefing meeting

6. What do they presently do?

7. Step 5 Execute sessions according to plan and evaluate sessions, report to planning committee Step 4 Invite faculty and share learning objectives, recommendations for learning formats Traditional Steps for Building a CPD Program Step 3 Outline content area to reflect learning objectives and choose format/methods/faculty Step 6 Feedback and other learning from past meetings Synthesize results Step 2 Identify subject area for content and write learning objectives Step 1 Clarify target audience and carry out needs assessment process

8. Impact of Accredited Group Learning on Clinical Practices • Multiple reviews show us the challenges of demonstrating impact on competency and performance • However, group learning is the predominant CPD activity for most physicians as reported by the MainCert and MainPro programs in Canada • Recent study in the Ottawa region demonstrated that physicians are quite satisfied with the CPD options available to them and believe that group learning does have impact on their clinical practices. • KT research and CPD delivery have not been effectively integrated or coordinated. Perhaps the failure to do so, is one important reason why there are major gaps in evidence based care delivery

9. Seems appropriate that we try to improve impact How? • Identify ingredients that increase impact • Transfer this knowledge and skill to those who build the programs and tools

10. What do planning committees need to be able to do? From the AAMC/SACME Harrison survey*…. Toward a more effective product: Enhancing didactic activities • Practice enablers (patient material, flow-sheets, algorithms etc at or after course) • Follow-up method post-course to reinforce learning Examples used in the survey • Pre-course planning ( needs assessment, objectives, choose methods, evaluate) • In-course enhancement • Meaningful interactivity • Simulations • Tools for patient management • Follow-up methods post-course to reinforce learning • Sequential learning * Association of American Medical Colleges. Academic CME in the US and Canada: The 2010 AAMC/SACME Harrison survey p. 13

11. Canadian Institutes for Health Research(CIHR): “Knowledge to Action” Cycle • Knowledge Translation and Implementation Science can contribute to identifying approaches and providing a framework for the design of this project. • Build on Ian Graham’s presentation to SACME last Fall – “Knowledge to action cycle meets CME/CPD”

12. Knowledge Inquiry Tailoring Knowledge Synthesis Products/ Tools Monitor Knowledge Use Select, Tailor, Implement Interventions from: Graham et al: Lost in Knowledge Translation: Time for a Map? Evaluate Outcomes KNOWLEDGE CREATION Assess Barriers/ Supports to Knowledge Use Sustain Knowledge Use Adapt Knowledge to Local Context http://www.jcehp.com/vol26/2601graham2006.pdf Identify Problem Identify, Review, Select Knowledge

13. Monitor Knowledge Use Step 6 Feedback and other learning from past meetings Synthesize results Step 5 Execute sessions according to plan and evaluate sessions, report to planning committee Select, Tailor, Implement Interventions Evaluate Outcomes Step 4 Invite faculty and share learning objectives, recommendations for learning formats Assess Barriers/ Supports to Knowledge Use Step 3 Outline content area to reflect learning objectives and choose format/methods/faculty Sustain Knowledge Use Step 2 Identify subject area for content and write learning objectives Adapt Knowledge to Local Context Step 1 Clarify target audience and carry out needs assessment process Identify Problem Identify, Review, Select Knowledge Tools Products Tailoring Knowledge Synthesis Knowledge Inquiry

14. What do planning committees need to be able to do?This study will focus on planning process as suggested by “K to A” cycle. Toward a more effective product: Enhancing didactic activities* • Practice enablers (patient material, flow-sheets, algorithms etc at or after course) • Follow-up method post-course to reinforce learning Examples used in the survey • Pre-course planning (needs assessment, objectives, choose methods, evaluate) • In-course enhancement • Meaningful interactivity • Simulations • Tools for patient management • Follow-up methods post-course to reinforce learning • Sequential learning * Association of American Medical Colleges. Academic CME in the US and Canada: The 2010 AAMC/SACME Harrison survey p. 13

15. Creating an effective combination of strategies is both art and science. • “The choice of KT interventions remains an “art” informed by science, meaning that practice-based experience and creativity are important in selecting KT interventions. We suggest that the use of a stepwise approach and structured methods helps take a comprehensive and balanced approach.” • Wensing M, Bosch M, Grol R. Selecting, tailoring and implementing knowledge translation interventions in Strauss S, Tetroe J, Graham ID(editors) Knowledge Translation in Health Care: Moving from Evidence to Practice. 2006 p.110

16. How does thinking within the “Knowledge to Action Cycle” affect how we design an intervention to change physician practices? Four areas that go beyond usual program planning process: • Clear statement of goal performance – note how this relates to design of evaluation plan • Change occurs within an organizational and cultural context • Use behaviour change theory to guide analysis of determinants of desired performance • Choose intervention vehicles based on evidence of what works best given learning objectives.

17. Monitor Knowledge Use Step 6 Feedback and other learning from past meetings Synthesize results Step 5 Execute sessions according to plan and evaluate sessions, report to planning committee Select, Tailor, Implement Interventions Evaluate Outcomes Step 4 Invite faculty and share learning objectives, recommendations for learning formats Assess Barriers/ Supports to Knowledge Use Step 3 Outline content area to reflect learning objectives and choose format/methods/faculty Sustain Knowledge Use Step 2 Identify subject area for content and write learning objectives Adapt Knowledge to Local Context Step 1 Clarify target audience and carry out needs assessment process Identify Problem Identify, Review, Select Knowledge Tools Products Tailoring Knowledge Synthesis Knowledge Inquiry

18. Use Behavior Change Frameworks to guide study questions. PRECEDE (Green) • Predisposing factors • Enabling factors • Reinforcing factors Theory of Planned Behaviour (Ajien) • Perceived severity • Can I make a difference • Do others want me to do it • Do I care whether others want me to do it • Does the environment (time, equipment patient issues etc) make it possible to do it

19. Objectives of the Study To gather information regarding the knowledge, beliefs and intentions of CPD program planners related to recommended best practices for developing and implementing CPD programs. Through a study of physicians who plan and lead “group learning” programs assess their: Knowledge of recommended best practices for accredited group learning programs. Perceived importance of the “best practices” recommended changes Belief in the potential impact of making the changes Perception of support for the changes from peers Perception of the personal, professional and environmental factors that can influence the changes.

20. Application of PRECEDE/PROCEED to design of study questions Predisposing factors – beliefs, attitudes, previous experiencePerception of severity of the problemIf I intervene - will it make a difference? Example Questions • Do planners believe there is a problem with present practices? • Do they believe they can do what needs to be done? • How much do their personal learning styles influence what they plan for others?

21. Enabling and Reinforcing factors – variables that affect ease of complying and support it continuing Perceived ease of engagement in the behaviourPerceived access to resources and opportunities to engage in the behaviourIs high performance rewarded? Example Questions Do planners have the resources they need to do the recommended best practice? Simulation, break out rooms, time, people who can lead Are there payoffs for doing things according to the new model?

22. DiscussionObjectives of the Study To gather information regarding the knowledge, beliefs and intentions of CPD program planners related to recommended best practices for developing and implementing CPD programs. Through a study of physicians who plan and lead “group learning” programs to assess their: Knowledge of recommended best practices for accredited group learning programs. Perceived importance of the “best practices” recommended changes Belief in the potential impact of making the changes Perception of support for the changes from peers Perception of the personal, professional and environmental factors that can influence the changes.

23. Resources Strauss S, Tetroe, J, Graham ID. Knowledge Translation in Health Care: Moving from evidence to practice. 2009. Wiley-Blackwell. Oxford. Green LW, & Kreuter MW. Health Program Planning: An educational and ecological. 4th edition 2005. McGraw Hill. Toronto. Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes, 1991;50:179-211 Michie S, Johnson M, Francis J, Hardeman W, Eccles M. From theory to intervention: Mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology:An International Review, 2008, 57(4):660-680. Michie S, Johnson M, Abrahm C, Lawton R. Parker D, Walker A, on behalf of the “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005;14:26-33. Moore, DE. How physicians learn and how to design learning experiences for them: An approach based on an interpretive review of evidence. In:Hager, M, Russell S & Fletcher SW (Eds.). Continuing education in the health professions: Proceedings of a conference. A Report for the Josiah Macy Foundation, 2008; http://www.josiahmacyfoundation.org/index.