University of Manitoba Pre-Survey Meeting with Program Administrators

1. University of ManitobaPre-Survey Meeting withProgram Administrators Date: July 3, 2013 Time: 2:00 to 4:00 p.m. Room: Pharmacy Apotex, Theatre # 264

2. Objectives of the Meeting • Provide an overview of • Accreditation Process • Questionnaire and Appendices • Schedule • Standards • Opportunity to network with colleagues

3. Who has been involved in a survey process? • Last regular survey in 2008 • External review • Internal review

4. Accreditation • Is a process to: • Improve quality of postgraduate medical education • Provides a means for assessment of residency programs • Assists program director in reviewing conduct of program • Based on Standards

5. Six Year Survey Cycle 1 6 Monitoring 5 2 3 4 Internal Reviews

6. Pre-Survey Process University Questionnaires Specialty Committee Questionnaires Royal College Comments Questionnaires & Comments Comments Program Director Surveyor

7. Questionnaire and Appendices • First impression of the program by the surveyor is the questionnaire and appendices • All sections of the questionnaire are complete • Error-free vs typos • Neat vs sloppy • Clear descriptions vs confusing information • Spell out abbreviations

8. Questionnaire and Appendices When submitting your questionnaire and appendices please format and name them as follows: Questionnaire • Each individual section (i.e. each standard) is submitted as a separate PDF file • The files are named as: • General_Information.pdf, Standard_B1.pdf, Standard_B2.pdf, etc.

9. Questionnaire and Appendices Appendices • For each appendix, the name of the appendix (e.g. Appendix A) is clearly indicated on the first page. • Each individual appendix is submitted as a separate PDF file. • The files are named as: Appendix_A.pdf, Appendix_B.pdf, etc. • Any additional attachments are labelled/formatted as above with the name corresponding to the appendix name as listed in the General Information section of the PSQ (e.g. Appendix_K.pdf, Appendix_L.pdf, Appendix_M.pdf, etc.)

10. The Survey Schedule • Review – Length & Day • Based on • Number of residents in program • Traveling to different sites • Last schedule in 2008 • Coordinated by the Royal College and PGME office • Inform PGME office if you foresee a scheduling conflict • PGME office will notify the program of the day of the review • Surgical Foundations will be reviewed on Monday morning.

11. The Survey Schedule • Transportation • Arranged by program • Include details • Name & contact info for person responsible • Morning • Pick-up at 07:45 a.m. • At the Fairmont Winnipeg • To and from different sites • When required during review • Afternoon • Survey must end no later than 4:00 p.m. • At the Fairmont Winnipeg

12. The Survey Schedule • Sequence to follow • Document review (30 min) • Program director (75 min) • Department head * (30 min) • Residents * (per group of 20 - 60 min) • Teaching faculty * (60 min) • Residency Program Committee (60 min) • Exit Meeting (15 min) • Morning after the review concluded • 07:30-07:45 a.m. at the Fairmont Winnipeg * Scheduled anytime after program director but before RPC

13. The Survey Schedule Document Review (30 min) • Scheduled at beginning of schedule • Available in meeting room • Residency Program Committee minutes - last 6 years • Resident assessment files • If automated, arrange access to a computer, make sure that it is easy to navigate and retrieve • Provide assistance/demo • Remember surveyors have limited time to review the documents on-site

14. The Survey Schedule • Program director (75 min) • Scheduled after document review • To discuss • Overall view of program • Evaluation of Standards • Strengths, challenges and weaknesses • Specialty Committee Comments • Sent by email to program director week prior to on-site visit

15. The Survey Schedule • Department head (30 min) • Scheduled anytime after program director but before RPC • To discuss • Support for program director and program • Resources available to program

16. The Survey Schedule • Residents (60 min) • Scheduled anytime after program director but before RPC • Groups of 20 residents • Can be organized in various groups • By postgrad year • By Junior & Senior years • Encourage residents to: • Complete the ‘confidential’ CAIR questionnaire • Sent by CAIR • Meet as a group to discuss strengths & challenges prior to visit • Send reminders – date, time, locations

17. The Survey Schedule • ALL residents invited to attend/participate in their resident meeting(s)? • Residents who cannot attend • Arrange link by video- or tele- conferencing • Are visa trainees or (clinical) fellows invited to attend? • Different funding / title across Canada • For accreditation purposes only, a resident is a person who is following the same academic program/training as a resident eligible to write College exams – funding doesn’t matter !

18. The Survey Schedule • Teaching faculty (60 min) • Scheduled anytime after program director but before RPC • To discuss • Involvement with residents • Communication with program director • Teaching faculty who sit on RPC need only attend RPC meeting

19. The Survey Schedule • Tours (15 min) • Optional • Tours of wards/clinics not necessary • Is there something special to showcase? • MUST be scheduled before RPC

20. The Survey Schedule • Residency Program Committee (60 min) • Program director attends first half of meeting • Resident representative(s) attend the meeting • Smaller programs • Often faculty all sit on RPC • Can combine faculty & RPC together into one meeting • MUST be scheduled as LAST meeting of review

21. The Survey Schedule • Survey Team Discussion • Your program will be discussed the evening of the day that your review ends • Identify the strengths & weaknesses • Provide a recommendation

22. The Survey Team • Chair - Dr. Sarkis Meterissian • Responsible for general conduct of survey • Deputy chair – Dr. Maureen Topps • Visits teaching sites / hospitals • Surveyors • Resident representatives – CAIR • Regulatory authorities representative – FMRAC • Teaching hospital representative– ACAHO

23. The Survey Schedule New terminology – June 2012 • Approved by the Royal College, CFPC and CMQ. • Recommendation • Accredited program • Follow-up: • Next regular survey • Progress report (Accreditation Committee) • Internal review • External review • Accredited program on notice of intent to withdraw accreditation • Follow-up: • External review

24. The Survey Schedule • Exit Meeting (15 min) • Morning after review • 07:30 – 07:45 a.m. • At the Fairmont Winnipeg • Program director will be informed • Survey team recommendation • Strengths & weaknesses of program

25. The Survey Schedule BE PROACTIVE • Start early • Share the information with your program director • Make a plan and set timelines • Create and use your own checklist • Consult with the PGME office • In doubt: ask questions

26. Day of the Survey • Be available to assist the surveyors as needed. • ‘Do Not Disturb – Meeting in Progress’ sign on door. • Ensure that the surveyors are comfortable (i.e.: private room, water, a layout of the area, etc.) • Arrange for beverages, healthy snacks for breaks and a lunch, when appropriate. • Organize all the required documents in the room and ensure that there is a computer with internet access available to the reviewers. • Retrieve all confidential documents from the room(s) at the end of the day.

27. After the Survey • Final exit with University • Friday, February 28, 2014 between 9:00 and 10:00 a.m. • CFPC/Royal College Chairs present a summary of survey week • Survey reports sent to PGME • 6 to 8 weeks post-survey • Program response sent to College • Report any errors of fact only • Accreditation Committee Decisions • May/June 2014 • Dean & postgraduate dean attend • Appeal process is available

28. General Standards of Accreditation “A” Standards • Apply to University, specifically the PGME office “B” Standards • Apply to EACH residency program • Updated January 2011 “C” Standards • Apply to Areas of Focused Competence (AFC) programs

29. “A” Standards • Applicable to the University • Reviewed by chair’s team (Team A) • Schedule organized by PGME office • Looking at • Structure in place • Training sites • Liaison with hospitals

30. “B” Standards • Applicable to ALL programs • Reviewed by surveyors • Looking at B1 Administrative Structure B2 Goals & Objectives B3 Structure and Organization B4 Resources B5 Clinical, Academic & Scholarly Content B6 Assessment of Resident http://www.royalcollege.ca/portal/page/portal/rc/common/documents/accreditation/accreditation_blue_book_b_standards_e.pdf

31. B1 – Administrative Structure There must be an appropriate administrative structure for each residency program. • Program director • Time & support • Residency Program Committee • Operation of the program • Program & resident evaluations • Appeal process • Selection & promotions of residents • Process for teaching & assessment of competencies • Research • Regular review of program • Faculty assessments

32. B1 – Administrative Structure“Pitfalls” • Program director autocratic • Residency Program Committee dysfunctional • Unclear Terms of Reference (membership, tasks and responsibilities) • Agenda and minutes poorly structured • Poor attendance • Department head unduly influential • RPC is conducted as part of a Dept/Div meeting • No resident voice

33. B2 – Goals and Objectives There must be a clearly worded statement outlining the Goals & Objectives of the residency program. • Rotation-specific • Address all CanMEDS Roles • Functional / used in: • Planning • Resident assessment • Distributed to residents & faculty • Reviewed regularly • At least every 2 years

34. B2 – Goals & Objectives“Pitfalls” • Missing CanMEDS roles in overall structure • Okay to have rotations in which all CanMEDS roles may not apply (research, certain electives) • Goals and objectives not used by faculty/residents • Goals and objectives dysfunctional – does not inform assessment • Goals and objectives not reviewed regularly

35. B3 – Structure & Organization There must be an organized program of rotations and other educational experiences to cover the educational requirements of the specialty. • Increasing professional responsibility • Senior residency • Service / education balance • Resident supervision • Clearly defined role of each site / rotation • Educational environment

36. B3 – Structure & Organization “Pitfalls” • Graded responsibility absent • Service/education imbalance • Service provision by residents should have a defined educational component including evaluation • Educational environment poor

37. B4 - Resources There must be sufficient resources – Specialty-specific components as identified by the Specialty Committee. • Number of teaching faculty • Number of variety of patients, specimens and procedures • Technical resources • Ambulatory/ emergency /community resources/experiences • Educational

38. B4 – Resources “Pitfalls” • Insufficient faculty for teaching/ supervision • Insufficient clinical/technical resources • Infrastructure inadequate

39. B5 – Clinical, Academic & Scholarly Content of Program The clinical, academic and scholarly content of the program must prepare residents to fulfill all the Roles of the specialist. • Educational program • Organized curriculum • Content specific areas defined by Specialty Committee • CanMEDS Roles • Teaching of the individual competencies

40. B5 – Clinical, Academic & Scholarly Content of Program “Pitfalls” • Organized academic curriculum lacking or entirely resident driven • Poor attendance by residents and faculty • Teaching of essential CanMEDS roles missing • Role modelling is the only teaching modality

41. B5 – Clinical, Academic & Scholarly Content of Program Appendix ‘H’Formal Academic Curriculum • The surveyor will be verifying that the content of the academic curriculum covers all the of the CanMEDS Roles.

42. B6 – Assessment of Resident Performance There must be mechanisms in place to ensure the systematic collection and interpretation of assessment data on each resident. • Based on objectives • Include multiple assessment techniques • Regular, timely, formal • Face-to-face

43. B6 – Assessment of Resident Performance “Pitfalls” • Mechanism to monitor, promote, remediate residents lacking • Formative feedback not provided and/or documented • Assessments not timely, not face to face • Summative evaluation (ITER) inconsistent with formative feedback, unclearly documents concerns/ challenges

44. University of ManitobaRegular on-site Survey February 23 to 28, 2014

45. Contact Information at theRoyal College accred@royalcollege.ca 613-730-6202 Office of Education Sarah Taber Assistant Director Education Strategy & Accreditation Educational Standards Unit Sylvie Lavoie Survey Coordinator