IIME Workshop on Assessment

1. IIME Workshop on Assessment Meeting Summary Chengdu, China February 27, 2003

2. MCQ Summary • Hardest working group!! • 150 Item test translated into Chinese • Translation of each question reviewed by the entire group for clarity • Examination details reviewed • Location • Receiving and returning examination materials • Rules for how to proctor the examination • Leader of MCQ team to recruit assistant proctor

3. OSCE Summary • 15 Station (10 room) OSCE • 5 Long SP, 5 Short SP, 5 Non-SP Cases • Blueprint of exam finalized • 1-day examination (3 sessions of 2 ½ hours) • Schools with 60 or 90 students can share patients and/or repeat day 1 examination • CMB support provided through Association for Chinese University Medical Colleges

4. 2003 OSCE Day 1 Long SP cases Short SP cases Non-SP cases Med 0201: Coronary heart OBG 0501: Ectopic preg Medicine paper cases, Xray, ECG,mcq OBG 0502: High risk preg Med 0701: Asthma Medicine paper case, lab report, mcq, auscultion Surg 0401: Acute choly Peds 0301: Fever + cough Literature Search: computer ER 0101: Low abdo pain Peds 0801: Malnutrition Surgery gowning or sterile technique, tube, or CPR Psych 0604: Panic disorder Psych 0601: Depression OBG paper case, pelvic model, mcq

5. Observation/Logbook Summary • Scales finalized for all 17 GMER • Faculty at this meeting trained to use the observation form • IIME to create evaluation form and send final version before April 30, 2003 • Faculty development plan workshops at your home schools • Observations before the examination • Submit forms back to IIME for scoring and analysis

6. Tasks for March 2003 • Leader for IIME project from each school will be primary contact for all information • IIME will create spreadsheet for student registration process • Leader registers students for examination • Spreadsheet returned to IIME with student identification numbers • Additional data on students for comparison to your school-based examinations and grades • IIME creates draft of faculty observation forms

7. Tasks for April-June 2003:Faculty Observations • IIME creates final evaluation forms and sends to each school • Schools begin faculty development workshops to teach how to use the evaluation forms • Begin faculty observations in June or July for 3 months • At least 3 evaluations for each student

8. Tasks for April – June 2003:Information for Students and Teachers • IIME will develop a brochure in Mandarin to educate students about the IIME project: • MCQ • OSCE • Faculty Observations • Also provide brochure to teachers • Draft score report to deans and presidents

9. Tasks for July:Standardized Patient Development • Possible meeting in July (Shenyang and/or Chengdu) • 1-day workshop • OSCE Coordinator, Standardized Patient Trainer, OSCE team (1-2 people/school) • Details of how to hire, train, and work with standardized patients.

10. Tasks for August – September 2003 • Final room arrangements for MCQ, receive test books • Final preparations for OSCE, complete training of SPs, recruit faculty • Complete faculty observations in clerkships and prepare forms to return to IIME

11. October 2003!!! • MCQ Administration • Secure receipt, administration, and return of test materials • OSCE Administration • Score reports created on-site and email to IIME • Observations • Forms sent to IIME by secure mail

12. Timeline: 2003

13. Timeline: 2003

14. Quality Assurance Program • MCQ examination • Statistical review for quality of questions • Comparison with your home school examinations • OSCE examination • Anonymous videotapes sent to review and score at other schools • Observations • Statistical review for quality of scores reported by faculty • Student and faculty assessment of examination • How realistic? • Fair coverage of domains?

15. Score Reports • Data security in the IIME office • Score reports created by GMER item, by discipline, and by student to return to IIME leader at your school. • Comparison of scores with other Chinese students WILL BE anonymous and combined ONLY. • Areas of success and improvement will be identified in this report • Comparison of these tests with your school examinations can be provided

16. Score Report: Major Domains • Professional Values, Attitudes, Behaviors and Ethics • Scientific Foundation of Medicine • Clinical Skills • Communication Skills • Population Health and Health Systems • Management of Information • Critical Thinking and Research

17. Score Report: Professionalism · Professional values which include excellence, altruism, responsibility, compassion, empathy, accountability, honesty and integrity, and a commitment to scientific methods; ·Self-regulation and a recognition of the need for continuous self-improvement with an awareness of personal limitations including limitations of one’s medical knowledge;  Recognition of the moral obligation to provide end of life care, including palliation of symptoms; ·Ability to effectively plan and efficiently manage one’s own time and activities to cope with uncertainty, and the ability to adapt to change; ·Personal responsibility for the care of individual patient.

18. Score Report: Scientific Foundations

19. Score Report: Pediatrics MCQ OSCE

20. Score Reports: Correlation of Overall MCQ Examination

21. IIME 2003 • Security and confidentiality • Pilot Study

22. Why We Will Succeed • International Standard Outcomes • International Standard Assessment Tools • International Quality Assessment Process • International Quality Leaders, Faculty and Assistants working on the IIME project at EVERY school!