Meaningful Evaluation : Framework, Process, Impact

1. Meaningful Evaluation:Framework, Process, Impact Inis Jane Bardella, M.D., FAAFP Associate Dean for Faculty Development and Global Health Initiatives Department of Family and Preventive Medicine Lecia M. Apantaku, M.D., FACS Assistant Dean for Clinical Sciences Education Associate Professor, Department of Surgery Chicago Medical School Rosalind Franklin University of Medicine and Science

2. Objectives • Discuss the relationship between evaluation and learning. • Discuss a framework [RIME, Reporter Interpreter Manager Educator] for evaluation. • Discuss the process of evaluation. • Apply the RIME framework to achieve meaningful [accurate, useful, efficient] evaluation of students and residents.

3. Vision Effectively prepare medical students and residents to meet the health needs of the people of the United States [especially the poor] with compassion, integrity, high ethical standards and a high level of competence.

4. Responsibility • Patients, always first • Students and Residents • Faculty • Clinical teachers – you • Clerkship director • Program director • Education system • Centegra • CMS

5. Definition Process of making judgments based on factual information and observations in order to rate, rank or assess an individual’s status at a given point in time.

6. Background – Adult Learning and Teaching What is different about adults? • Self-directed • Goal-directed • Application-oriented • Competency concerns • Shared responsibility • Moral responsibility • Teacher is: Resource, Facilitator, Mentor

7. Background – Adult Learning and Teaching How does this influence evaluation? How should this influence evaluation?

8. Bottom Line of Evaluation • Evaluation drives learning. • Objectives determine evaluation. • Evaluation is the second greatest fear of preceptors/clinical faculty. • Evaluation is absolutely necessary to determine progression toward independent practice.

9. So, why formal evaluation? • Summarize performance • Determine competencies (behaviors) • Guide future learning • Communicate summary information • Determine a grade

10. Framework – RIME • Reporter • Interpreter • Manager • Educator

11. Framework – RIME • Reporter Consistently professional, good interpersonal skills Reliably obtains and communicates clinical findings Ownership of clinical findings • Interpreter Consistently able to analyze and prioritize patient problems Ownership of explaining things

12. Framework – RIME • Manager Consistently proposes reasonable options incorporating patient preferences Ownership of developing action plan with patient • Educator Consistent level of knowledge of current medical evidence Can critically apply knowledge to specific patients Ownership of evidence for action and sharing

13. RIME Caveat Students must be reasonable, residents must be right.

14. Application – RIME • What is the RIME? • How did you determine the RIME?

15. Framework – RIME • Synthetic model – synthesis • What we do all day long in patient care! • Focus is patient – performance in relationship to patient care • Professionalism required • High level of interpersonal skills required • Descriptive • Progressive, building model • Competencies (KSA, behaviors) integral • Not relative – Expectations clear

16. A Few More RIME Caveats • Program must determine common problems and uncommon problems. • Program must determine core or essential areas and non-essential areas

17. Break TIME

18. Application • What is the RIME? • What if: • Beginning 3rd year? • Middle 3rd year? • End 3rd year? • Mid 4th year? • Beginning intern? • Mid intern? • 2nd year resident?

19. Process – PIE Cycle

20. Process - PIE Planning • Planning • Clerkship goals and objectives • Requirements/projects • Student self-evaluate • Review the form

21. Process - PIE • Instruction • Observation • Feedback • Action plans • Gather and record data • Patient and staff perceptions • Chart documentation • Observation • Student progress notes Instruction

22. Process - PIE Evaluation • Evaluation • Form • Student • You • Unrushed • Emphasize objectives, improvement • Specific comments • Meeting

23. Comments about Comments Examples? • Demonstration • Consistency • Progression • Improvement • Recommendations • RIME it

24. Application - Forms • RIME the elements • RIME the learner • Complete the form using RIME as the framework to assess behavior, i.g. competency. • Comments

25. Summary • Evaluation is a continuous process. • Evaluation drives learning. • Goals & objectives determine evaluation. • RIME it. • Determine the student’s starting point. • Develop a plan for progression. • Determine the final evaluation. • Include comments • To teach is to learn again!

26. References Bardella IJ, ed. Essentials of precepting 2002-2003. 2002, revised 2006. Ende, J. Feedback in clinical medical education. JAMA. 1983;250:777-781. Massachusetts statewide area health education center program. Faculty development workbook. 1995. Pangaro LN. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med. 1999;74:1203-1207. Pangaro LN. Evaluating trainees in the clinical setting: what does competency in patient care look like? Presented 25 Jan 2012 at RFUMS, North Chicago, IL. Society of teachers of family medicine pep2 committee. Preceptor education project, second edition: facilitators guide. Society of teachers of family medicine. Kansas City. 1999.

27. Thank youinis.bardella@rosalindfranklin.edu