Creative Commons License

1. Creative Commons License Attribution-NonCommercial-ShareAlike 2.0 • You are free: • to copy, distribute, and display this presentation, and/or • to make derivative works • Under the following conditions: • Attribution. You must give the original authors credit. • Noncommercial. You may not use this work for commercial purposes. • Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. • See http://creativecommons.org/licenses/by-nc-sa/2.0/ for full license.

2. Teaching and Assessing CompetenciesFaculty Development WorkshopUCSF School of MedicineMay 22, 2007 Lee A. Learman, M.D., Ph.D.Professor and Residency Program Director Dept. of Obstetrics, Gynecology & Repro. Sciences Director of Curricular Affairs, UCSF Office of GME

3. Objectives • Understand the status of your education program’s evaluation system in context of others’ experiences • Identify promising teaching and assessment practices and understand what has led to their successful implementation • Understand how competency outcomes should inform curricular quality improvement (Phase 3 of the the ACGME Outcomes Project)

4. Schedule 1:00 – 1:15 Welcome, Overview and Introductions __________________________________________________ 1:15 – 2:00 ACGME Update 2:00 – 2:45 Teaching Strategies 2:45 – 3:45 Assessment Tools 3:45 – 4:45 Curricular CQI __________________________________________________ NB – There will be refreshments available and a “working break” in the middle of the workshop.

5. Resources from the ACGME Website “Educating Physicians for the 21st Century,” a series of five PowerPoint presentations with a Facilitator’s Manual, is a new educational resource from the Research and Education Department. The Facilitator’s Manual contains speaker notes and discussion questions to help Program Directors give these presentations during faculty meetings or educational retreats. If you wish to provide feedback, click here. Educating Physicians for the 21st Century - Introduction (web-based learning module) Educating Physicians for the 21st Century

6. Why assess? • Whatever we measure, we tend to improve • D. Leach

7. Why assess? • The competencies are organizing principles to frame our conversations [curriculum]. M. Dunn

8. The Six Competencies • Medical Knowledge • Patient Care • Practice Based Learning and Improvement • Systems Based Practice • Professionalism • Interpersonal and Communication Skills

9. What does the ACGME expect? • Your program should document and demonstrate: • Learning opportunities in each competency domain • Evidence of multiple assessment methods • Use of aggregate data to improve the educational program

10. Common Program Requirements • Common Program Requirements • core accreditation requirements for all specialties • revision effective July 1, 2007 • Specialty Program Requirements • additional accreditation requirements specific to a specialty • Go to www.acgme.org

11. Curriculum Overall educational goals for the program distributed annually Competency-based goals and objectives for each assignment and each educational level, distributed annually and reviewed with residents prior to rotations Integration into the curriculum of the six ACGME competencies Resident Evaluation - Formative An evaluation must occur at completion of each rotation Objective assessments of competency in the 6 areas Multiple evaluators (faculty, peers, patients, self, other professional staff) Progressive improvement by PGY Documented semiannual evaluation Common Program Requirements

13. Resident Evaluation - Summative Upon completion of the program Part of the permanent record Documents performance during the final period of evaluation Verifies that the resident ‘has demonstrated sufficient competence to enter practice without direct supervision.’ Program Improvement Formal, systematic evaluation at least annually Must monitor and track: Resident performance Faculty development Graduate performance Program evaluations by residents and faculty Must use results to improve the program Written plan of action Reviewed and approved by teaching faculty Common Program Requirements

14. Program Focus Provide learning opportunities in all six competency domains Improve evaluation processes as needed Provide aggregate resident performance data for Internal Review Accreditation Focus Review evidence that programs are teaching and addressing the competencies Provide constructive citations early and more constructive citations later in phase Internal Review includes aggregate resident performance data TimelinesPhase 2 - 7/02-06

15. Program Focus Use resident performance data as basis for improvement Begin to use external measure (patient surveys, clinical quality measure) to verify resident and program performance Accreditation Focus Evidence that programs are making data driven improvements Review the external program performance indicators and input from GMEC to make sure your program is achieving its educational objectives TimelinesPhase 3 - 7/06-6/11

16. Medical Knowledge • Residents must demonstrate knowledge about established and evolving biomedical, clinical, and cognate (e.g. epidemiological and social-behavioral) sciences and how to apply this knowledge to patient care. • Acquisition • Analysis • Application

17. Content Specialty specific curriculum that addresses key skill sets RRC Specialty specific requirements Setting Clinical Teaching Lectures/Seminars/Conferences Journal Club Procedural workshops Board review courses or formalized group study experiences Simulations Self directed learning through case based modules Teaching Medical Knowledge

18. Patient Care • Residents must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health. • Gathering information • Synthesis • Partnering with patients/families

19. Content Specialty specific skills that address key skill sets Specialty specific procedural knowledge Knowledge about information technology Setting Clinical Teaching Lectures/Seminars/Conferences Role Modeling Workshops Simulations Self directed learning through case based modules Teaching Patient Care

20. Practice Based Learning and Improvement • Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices. • Life-long learning • Evidence based medicine • Quality improvement • Teaching skills

21. Content Self Reflection Quality or Practice Improvement Setting Self reflect on practice and determine improvement Lectures/Seminars/Conferences Quality Improvement Project Teaching Practice Based Learning and Improvement

22. Content Evidence Based Medicine Teaching Skills Setting Lectures/Seminars/Conferences Journal Club Research Project Clinical Teaching Clinical Teaching Role Modeling Interactive Workshop Teaching Practice Based Learning and Improvement (cont.)

23. Systems-based Practice • Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide optimal health care • Health care delivery system • Cost effective practice • Patient safety and advocacy/Systems causes of error

24. Content Health care system Different types of medical practice and delivery systems System resources System issues and the reduction of errors Conducting a root case analysis Setting Clinical teaching Patient Safety projects Systems based approach to M&M Lectures/Seminars/Conferences Interdisciplinary Teams Teaching Systems Based Practice

25. Content Cost Effective Practice Tools and techniques for controlling costs and allocating resources Understanding of financing/insurance structures Mock practice experiences/mock financials Evaluating risk and benefit of costly prescribing Setting Clinical Teaching Practice management curricula or projects Teaching Systems Based Practice (cont.)

26. Content Patient Safety and Advocacy Setting Lecture/Seminar/Conference Clinical teaching Individual or Group Projects Conducting a root cause analysis on near miss or sentinel event Systems based approach to M & M Teaching Systems Based Practice (cont.)

27. Professionalism • Residents must demonstrate professionalism, as manifested through a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population • Professional behavior • Ethical principles • Cultural competence

28. Content Respect, altruism, integrity, honesty, compassion, empathy Setting Clinical teaching Case based teaching Mentoring Role modeling Role plays and clinical vignettes Teaching Professionalism

29. Content Ethics Setting Clinical teaching Case based teaching Ethics Committee Mentoring Role modeling Teaching Professionalism (cont.)

30. Content Cultural Competence Setting Clinical teaching Case based teaching Interactive Workshops Lecture/Conference/Seminar Institutional Initiatives Role modeling Mentoring Teaching Professionalism (cont.)

31. Interpersonal and Communication Skills • Residents must be able to demonstrate interpersonal and • communication skills that result in effective information • exchange and teaming with patients, their patients families, • and professional associates. • Communicating with patients and families • Communicating with team members • Scholarly Communication

32. Content Communication with Patients and Families Setting Clinical teaching Role modeling Case based teaching Interactive workshops or seminars using role-plays Teaching Interpersonal and Communication Skills

33. Content Communication with colleagues Setting Standardized communication around handoff Clinical teaching Role modeling Interactive workshops or seminars Teaching Interpersonal and Communication Skills (cont.)

34. Content Scholarly communication Setting Grand Rounds Presenting lectures, seminars, conferences Writing abstracts Presenting a poster Scholarly articles Teaching Interpersonal and Communication Skills (cont.)

35. Summary • Competency based residency education focuses on a resident’s performance (learning outcomes) • The major goals of the Outcome Project are: • Develop competence as a physician • Improve patient care • Review Common Program Requirements • Review the ACGME Phase 3 Expectations

36. Teaching Strategies 2:00 – 2:30 Group Activity – Curricular Threads 2:30 – 2:45 Share Best Practices __________________________________________________ For the competency your group selected: • Identify/share the methods your program is currently using to teach this competency • Can you trace the teaching of this competency as a ‘curricular thread’ through the years of your education program? • How might you improve the coherence of the learning experience for this competency? • Nominate one or two “best practices” from the group

37. Assessment Tools 2:45 – 3:30 Group Activity – Looking Glass 3:30 – 3:45 Share Best Practices __________________________________________________ For the competency your group selected: • Identify/share the methods your program is currently using to assess this competency (in addition to any global assessments you conduct after a rotation) • What specific tools do you use, how frequently, and how you prepare the learners and evaluators for the assessment? • Discuss the challenges you have faced and surmounted in implementing these assessments • Nominate one or two “best practices” from the group

38. Curricular CQI 3:45 – 4:30 Group Activity – Closing the Loop 4:30 – 4:45 Share Strategies 4:45 – 5:00 Wrap-up and Evaluation __________________________________________________ How does your program use resident performance data as a basis for improvements in the curriculum? • Describe one change you have implemented in your curriculum based on resident performance data. • Which resident performance outcome(s) did you use? • In what setting did you review the data? Who were the stakeholders? How was the change implemented? • Looking ahead, how will you organize and interpret data from the multitude of evaluation tools you are, or will be, using?