Presenters: Dr. Rae Wright , Family Medicine of Southwest Washington

1. CCC: Lessons Learned – Two Programs, Two Case Examples Presenters: Dr. Rae Wright, Family Medicine of Southwest Washington Dr. Zinna Johns, East Pierce Family Medicine Hosted by: Family Medicine Residency Network Webinar: October 1, 2014

2. Structure • Program 1: Family Medicine of Southwest Washington • Developing the CCC Team • Developing and Determining Evaluation Tools • Case: Resident Profile • Program 2: East Pierce Family Medicine • CCC Background • Case: Resident Profiles – CCC Discussion and Response • Outcome • Discussion

3. CCC and Case 1 Family Medicine of Southwest Washington Presented By: Dr. Rae Wright

4. Strategically Select CCC Members Start with a champion Respected and trusted by faculty and residents Active in teaching and evaluating residents in a variety of settings Interested in learning Milestones lingo History of being collaborative in meetings, etc. FMSW has 6 members, 5 full spectrum FM faculty and one BH faculty

5. Develop a Milestones Based Evaluation System • Collect aggregate date on Milestones over time • Tools should be easy to interpret • New Innovations has built-in tools • Direct vs. indirect evaluations • Shift cards • Milestones reports • Gradually integrate new evaluations

6. Periodic Meetings for CCC • Discussion of residents of concern • Use Competencies and Milestones based language for discussion • FMSW Style • CCC meets 1-2 times per month • Must have at least 3 members present for interim meeting. Usually 4-5. • Pre-biannual meeting with other faculty including advisors

7. Case 1: Concern • “Working to improve documentation – some uncertainty about what is needed.” • “Still struggling on nights to get work done by [themself], as well as learning about all the small extra tasks that are required, but once [resident] is shown will then consistently perform them. Visible improvement over the few days I was with [resident].” • “Needs some improvements in organizational skills to prioritize and perform duties as needed for care of patients.” • “Presentations are a work in progress. I encourage [resident] to avoid extraneous comments and questions during presentations. Presentations were initially difficulty to follow due to the lack of structure, but they improved in the week we had together.” • “Presentations not yet polished, can be scattered.” • “Does not consistently carry pager when on call.”

8. Discussion • Competencies/Milestones of Concern • Patient Care (PC-1) • Professionalism (Prof-1,2) • Communication (C-3,4) • Plan for Improvement • Seek out feedback to improve performance real time. • Carry pager as required. • Focus on task at hand before moving to next tasks. • Practice oral presentations as part of active precepting and with senior residents.

9. Semi-Annual Meetings • Preparation • Use support staff to gather all data beforehand • Pre-meet with advisors and other available faculty for Resident Review • Meeting • Consider splitting into 2 groups • Use a time keeper

10. Case 1: Resident Profile • “Enthusiastic, energetic, and always eager to learn.” • “Actively seeks out feedback and takes suggestions well.”

17. After the CCC Semi-Annual Review Milestones information completed in NI Email sent to advisors with instructions and meeting time Advisors review all information with advisees, including Milestones info on NI Informal vs. formal feedback to CCC after advisors meet with advisees

18. What did we do with our resident? Interim meetings with advisor Active precepting in clinic Fine tune presentations when in clinic Shadow senior residents in the inpatient setting to see the other side Resident received all feedback well and has made some progress

19. CCC and Case 2 East Pierce Family Medicine Presented By: Dr. Zinna Johns

20. CCC Background CCC EPFM Style • 3 CCC meetings per class broken up into specific teams: Pine, Oak, and Maple. • Thus a total of 9 CCC meetings biannually at EPFM. • Advisor(s) for each team must be at the CCC for their advisees. Other faculty members may attend if schedule allows. • Program Director, Program Coordinator and Behavioral Health Specialist present for all CCC meetings.

21. Case 2: Resident Profile: Above or Below the Bar? • SA is one of 6 residents in the 1st class of residents at EPFM • At time of CCC, is half way through her 2nd year of residency • In general, performance is “meets” or “exceeds expectations” • Had a reputation as a resident that “sets the bar”

22. Case 2: Resident Profile: Above or Below the Bar? • Spring 2014 CCC for R2s on average took about 35-42 minutes. • The outlier was the CCC evaluation for SA, which took 75 minutes. • Areas of concern were: SBP4, Prof1-4, and Com3. • Tools used for evaluation include: Faculty observation, 360 evaluations, ITEs, Rotation evaluations that were mapped to Milestones

29. Case 2: Resident Profile

30. Case 2: Concern • Through out residency, SA has had cyclical episodes of interactions that were concerning for lack professional conduct. • Behaviors such as explosive response to changes to a previously established policy; inappropriate selection of time and modality of giving negative feedback (to med students, peers, and faculty); repeated inflexibility with changes that are perceived as unfair • Resident is effectively isolating self from fellow residents because of lack of willingness to be a team player.

31. Case 2: Mock CCC Discussion

32. Case 2: Outcomes • Resident was given Milestones feedback, after completing self assessment with the Milestones packet. • On average scored 2.5. except for the areas of concern. • On self assessment, scored self at 3.5. • Areas of weakness were reviewed with resident. • Resident was informed that the behavior was problematic and needed to change.

33. Case 2: Outcomes • Reviewed the cyclic pattern with resident and outlined correlation with stressful schedules such as night float. • Resident was directed to Behavioral Health Specialist for tools and/or reading a book about professionalism and communication. • 2 follow-up meetings have occurred since. • SA never met with BH (now 4-5 months later). Resident chose a book about spirituality at the workplace and felt overall improvement. • Planned pre-CCC meeting with SA to revisit areas of weakness, which persist with clarification that persistence these behaviors could lead to formal process.

34. Lessons Learned • CCC Meetings increase in value with higher number of faculty members. • The more faculty members, the longer the CCC meetings. • If there is a prolonged discussion on a specific Milestone for a certain resident, that person is possibly struggling in that area. • CCC’s task is simply to evaluate the data and assign the resident’s progress for the Milestones. • This must be separated from identification of whether a resident is in difficulty. • CCC is not intended for problem solving and the tendency to do so will limit efficiency.

35. Questions & Discussions