Elizabeth A. Lindsay MS, PhD

1. Program Planning Committees: Tools for Building Capacity 4th National Accreditation Conference Toronto, Ontario September 21, 2012 Elizabeth A. Lindsay MS, PhD

2. Disclosure I have no actual or potential conflict of interest in relation to this workshop.

3. Learning Objectives After active engagement in this workshop, participants will be better able to: Identify recommended competencies for planning committee members Discuss planning committee roles and responsibilities with a focus on building capacity and meeting standards Identify and develop practice planning committee solutions for use in faculty development

4. Agenda A. Background for the workshop – 20 min Why are planning committees important? Is there currently a problem with planning processes? What do committee members need to be able to do – what is the scope of their roles? Overview of the “Gatekeeper Study” results B. Small group discussion – 20 min Objectives - to discuss specific competencies and tools directed at selected aspects of the planning process for large or small group learning C. Full group discussion - 35 min Presentation of examples of tools to support planning processes Examples of tools recommended in small group discussion D. Evaluation/Feedback – 5 min.

5. What does a high functioning program planning committee look like?

6. Is our present system working well?

7. What do they presently do? Usual Practice for Planning One-day Programs 2-3 face-to- face meetings before event E-mail communication to finalize faculty, titles & objectives Occasionally, 1 debriefing meeting

8. Impact of Group Learning on Clinical Practices Multiple reviews show us the challenges of demonstrating impact of educational programs on competency and performance and there are pressures to improve the impact of our programs as well as to expand the type of activities we offer. However, group learning continues to be the predominant CME/CPD activity for most physicians, as reported by the MainCert and MainPro programs in Canada Recent study of family physicians in the Ottawa region demonstrated that they are quite satisfied with the CME/CPD options available to them and believe that group learning does impact their clinical practices….little pressure to change. Expectations for the planning processes for accredited group learning and on-line learning are clearly defined – how well are we doing them? Those who evaluate applications for accreditation can advise us.

9. Seems appropriate that we try to improve impact How? Identify ingredients that can increase impact Transfer this knowledge and skill to those who build or lead the programs

10. Step 5 Execute sessions according to plan and evaluate sessions, report to planning committee Step 4 Invite faculty and share learning objectives, recommendations for learning formats Steps for Building an Accreditable Program Step 3 Outline content area to reflect learning objectives and choose format/methods/faculty Step 6 Feedback and other learning from past meetings Synthesize results Step 2 Identify subject area for content and write learning objectives Step 1 Clarify target audience and carry out needs assessment process

11. What do we know about impact of group learning? Group learning ( lectures, workshops, small groups, rounds) Effect size average - 6-10% * Key factors that determine impact - amount and type of interaction; less complex behaviours and more serious outcomes* Previously noted factors that enhance impact: needs based; relevance to practice reflection before, during and after meetings commitment to change at educational events combined with other interventions sequential repetition *Forsetlund et al 2009

12. What do planning committees need to be able to do? From the AAMC/SACME Harrison survey*…. Toward a more effective product: Enhancing didactic activities Practice enablers (patient material, flow-sheets, algorithms etc at or after course) Follow-up method post-course to reinforce learning Examples used in the survey Pre-course planning ( needs assessment, objectives, choose methods, evaluate) In-course enhancement Meaningful interactivity Simulations Tools for patient management Follow-up methods post-course to reinforce learning Sequential learning * Association of American Medical Colleges. Academic CME in the US and Canada: The 2010 AAMC/SACME Harrison survey p. 13

13. Basic Assumptions for the Gatekeeper Study Investigators:Elizabeth Lindsay, Paul Hendry, Michael Allen, Heather Armson , Jatinder Takhar Health care providers who serve on planning committees and as presenters are the “gatekeepers” to improving effectiveness of group learning programs There is a relationship between program design, implementation processes and potential impact of group learning programs on participants

14. We Applied Behavior Change Frameworks* to Guide our Study Questions Are physicians who serve on planning committees Aware of best practices in program design? Willing to apply new planning processes? Do physicians who serve on planning committees believe that selected planning processes are: Important? Feasible? Will make affect impact of the program? *PRECEDE (Green) and Theory of Planned Behavior (Azjen)

15. Study Participants 52 physicians who had participated in planning committees at the University of Ottawa over an 2.5 yr period. Respondents Family physicians - 17 Other specialists - 15 Total 32/62%

16. Competency areas selected for study** Assessed information that describes the learning needs of potential participants in the program eg. gaps between current and recommended clinical practices Created learning objectives for the program that state what the learner will be able to do as a result of participating in the specific sessions. Recommended types of learning formats that work best for different types of objectives eg for knowledge based or skills based outcomes Discuss objectives and recommended learning formats with faculty/teachers to ensure mutual understanding Encourage faculty/teachers to present the level of evidence supporting educational messages (eg. Meta-analysis, RCT, expert opinion) Identify strategies that will facilitate practice integration of recommended best practices or address barriers to implementation Audit programs to assess existence of bias or compliance with accreditation standards **Selected from Alliance for CME list of competencies for CME/CPD professionals

18. Conclusions High levels of willingness to improve Strong belief in the importance of all factors but Not as strong regarding belief of potential impact Lots of variability regarding feasibility

19. Purpose and process of small group discussion Focus on one element of the planning process Considering our expectations regarding planning committees – what can we do to support them? Describe and create a list of specific processes and tools you use or think we should use to support planning committees

20. Topics for small group discussion Composition of the Planning Committee – The Chair and Preparation • Composition of the Planning Committee – The Committee Members • Competencies and Pre-work Structure and Process of Planning Meetings – During meetings • Structure and Process of Planning Meetings – Between Meetings Choose effective presenters and helping them prepare Choose educational strategies that go beyond didactic presentations withQ&A

21. Full Group Discussion With the package of resources as a basis --- add the ideas raised by each of the small groups

22. Topics for small group discussion Composition of the Planning Committee – The Chair and Preparation Qualities of a high functioning chair Preparation Ideas – evidence for importance of improving impact • Composition of the Planning Committee – The Committee Members • Competencies and Pre-work • Content Experts • Audience Representatives • Education Designer • Event Planner

23. Topics for small group discussion Structure and Process of Planning Meetings – During meetings Agenda should reflect time for different steps in the process in order to set up expectations for the meeting process eg. Time for looking at needs assessment information Use a planning template that can be completed during discussion at the meeting, columns for Gap and Objective, Title, Presenters – event planner can do some of this with help from the chair or another committee member • Structure and Process of Planning Meetings – Between Meetings • The event planner, education designer and chair can work on planning template between meetings to ensure the process moves forward in a timely manner • Individual members of the committee may have tasks that they have agreed to complete eg. Check on availability of presenter or find a particular resource – ideally meeting planner should follow-up to gather this information and apply it to the planning template.

24. Topics for small group discussion • Choose effective presenters and helping them prepare • Presenters strengths – ideas for how to find them • Knowledgeable and creditable regarding content • Demonstrates good understanding of the clinical setting in which learner works and the challenges faced there • Clarity regarding content and presentation format • Sufficient interaction to ensure understanding eg. Able to answer questions appropriately • Create template letters of invitation to potential presenters to ensure draft objectives and title are included and invite feedback Choose educational strategies that go beyond didactic presentations withQ&A Match objective with learning format – see Michie What makes a workshop, a workshop? Improving interaction – Tip sheets

25. Resources • Strauss S, Tetroe, J, Graham ID. Knowledge Translation in Health Care: Moving from evidence to practice. 2009. Wiley-Blackwell. Oxford. • Green LW, & Kreuter MW. Health Program Planning: An educational and ecological. 4th edition 2005. McGraw Hill. Toronto. • Ajzen I. The theory of planned behavior. Organizational Behavior and Human Decision Processes, 1991;50:179-211 • Michie S, Johnson M, Francis J, Hardeman W, Eccles M. From theory to intervention: Mapping theoretically derived behavioural determinants to behaviour change techniques. Applied Psychology:An International Review, 2008, 57(4):660-680. • Michie S, Johnson M, Abrahm C, Lawton R. Parker D, Walker A, on behalf of the “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005;14:26-33. • Moore, DE. How physicians learn and how to design learning experiences for them: An approach based on an interpretive review of evidence. In:Hager, M, Russell S & Fletcher SW (Eds.). Continuing education in the health professions: Proceedings of a conference. A Report for the Josiah Macy Foundation, 2008; http://www.josiahmacyfoundation.org/index.

26. Contents of Handout Title page - University of Ottawa Office of Continuing Medical Education (OCME) Handbook for Planning Committees A How‐to‐Guide for the development of an education program that may be accredited for Family Physicians (MainPro 1) and Specialists (MOC) Examples of slides for discussion of KT, evidence of impact of group learning. Excel planning template for listing of topics, gaps and objectives Tip Sheets – Example – Interactivity Invitation template to participate on a planning committee, including roles, responsibilities, time commitment, timelines and expectations regarding response to communication. K to A framework Michie matching of objectives and methods