Recognizing, Evaluating, and Rewarding Educators and Educational Scholarship

1. Recognizing, Evaluating, and Rewarding Educators andEducational Scholarship Health Professions Educational Research Symposium January 14, 2007 Fort Lauderdale, FL Ruth-Marie E. Fincher, MD Vice Dean for Academic Affairs Medical College of Georgia School of Medicine

2. “Education Is Our Core Mission” Education Research Service Patient Care From Michael Wilkes, 2003

3. How It Seems Research Service Education Patient Care Modified by R Fincher, 2004

4. Making Us Feel Like. . .

5. Agenda for Presentation • Background • Teaching & educational scholarship • Framework • Assessing teaching & learning • GEA scholarship project • 5 categories of educational activities • Assessment of each category • Quantity • Quality • Engagement with educational community • Scholarly approach • Scholarship

6. Changing Institutional Culture May Be More Difficult than Moving a Graveyard The Scholarship Culture is Changing Slide adapted from Malcolm Cox Swanson G. In Tosteson et al. New Pathways to Medical Education. Cambridge MA, Harvard University Press, 1994

7. Teaching and Its Scholarship Are Often. . . • Overshadowed by • Research • Clinical care • Undervalued in reward system • Promotion • Tenure • Salary

8. Institutions Reward What They Value • Teaching is highlighted in most mission statements • Teaching and related scholarship must be rewarded if they are truly important to an institution

9. Recent Evolution of Medical School Environment Caused a“Crisis of Mission” Concept from Cuban: How Scholars Trumped Teachers, 2001

10. Think outside the box! Think outside the box!

11. Medical Schools Only Unique Mission = Teaching • . . . .“widespread agreement that those members of the faculty who are most committed to. . . the education of medical students must be supported and rewarded. . . . Faculty appointments, promotion and tenure policies must reflect the changing roles and responsibilities of medical school faculty.” Whitcomb M: Acad Med 78:117-118, 2003.

12. Evolution of Concept of Teaching and Scholarship • Pre-Boyer • Teaching = academic citizenship • Everyone does it • Little to do with scholarship or promotion • Boyer • Reframed discussion by • Challenging concept that “everyone teaches” • Examining teaching as a form of scholarly work Boyer: Scholarship Reconsidered, 1990.

13. Continued Evolution: Glassick* • Common criteria form foundation for all forms of scholarship Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique *Scholarship Assessed, 1997.

14. AAMC’s Group on Educational Affairs:Promotes Education AcrossContinuum of Medical Education Continuing Undergraduate Graduate RIME

15. GEA Scholarship Project GOAL: Value faculty who support and advance medical education by connecting education, scholarship and academic advancement Phase 3 2005-2006 Phase 4 2007- ? Phase 1 1996-2000 Phase 2 2001-2004

16. GEA Scholarship Project Phase 1 1996-2000 Phase 4 2007- ? Phase 3 2005-2006 Phase 2 2001-2004 • Provided examples of activities and evidence • Simpson & Fincher (1999) • Defined educational scholarship • Outlined infrastructure needed to support educational scholarship • Fincher et al. (2000)

17. Characteristics of Scholarship • Demonstrates the “3 P’s”: • Product that can be reproduced and built upon • Open to peer review • Publicly disseminated • Scholarship moves field forward • Assessment parallels research

18. GEA Scholarship Project Phase 2 2001-2004 Phase 3 2005-06 Phase 4 2007- ? Phase 1 1996-2000 • • Identified types of educational activities & evidence • Simpson et al. (2004) • Characterized infrastructure – Academies • Irby et al. (2004)

19. Education Activity Categories Teaching Curriculum Advising/Mentoring Educational Administration/Leadership Assessment of Learner Performance Simpson D, et al. Acad Med 2004;79:783-790

20. Infrastructure to Support Education • Faculty development centers • Career development • Teaching/educational research skills • Societies or Academies • Recognize accomplishments based on selection criteria

21. GEA Scholarship Project Phase 2 2001-2004 Phase 4 2007- ? Phase 3 2005-06 Phase 1 1996-2000 • GEA Consensus Conf on Educational Scholarship • Defined areas of agreement & concerns in 3 areas • Category inclusions • Evidence & presentation displays for each category • Areas needing further investigation

22. Categories of Education Activity Teaching Curriculum Advising/Mentoring Educational Administration/Leadership Assessment of Learner Performance Simpson D, et al. Acad Med 2004;79:783-790

23. Q2 + Engagement with Community Contribution to education mission Quantity Quality Engagement with education community Scholarly approach to education activity Educational scholarship Draws From Contributes To

24. Existing literature Best practices Resources in field Colleagues locally  internationally Dissemination Peer reviewed forums Local  international audiences Impacts the field Scholarly and Scholarship Scholarly work is influenced by: Scholarship contributes to the field:

25. Documentation: Teaching Category • Brief Description • Role, activity description • Evidence of Quantity • Who, what, when, where, how much, how many • Evidence of Quality • Summary of student/peer evaluations • Short excerpts from supporting letters • Invitations to teach outside department or school • Repeat invitations to teach same group or in course

26. Teaching Definition and Inclusions • Is the design and implementation of activities to promote learning • Includes • Course design • Development of instructional materials • Interactions with students • Formative and summative assessment Act of teaching is not scholarship

27. Teaching: Quantity & Quality Give comparative ratings for each year; compare with *Give comparative ratings for each year; provide comparative ratings with peers if possible *Comparative ratings should be given over time; Compare with peer group if possible

28. Teaching: Quantity and Quality

29. Engagement with Community • Scholarly approach: Learns from • Educational literature • Best practices • Scholarship: Contributes to field • Product: Interactive cases • Peer review: MedEdPORTAL • Public: Web

30. Scholarly Approach • Teaching - learning connection • Consult education literature • Apply intervention to enhance learning • Observe outcomes • Analyze results • Obtain peer evaluation • Use results to improve teaching • Product reflected in student learning Richlin L: Scholarship Revisited: Perspectives on the Scholarship of Teaching. Jossey-Bass, 2001

31. Evidence of Scholarship • Product: Instructional material • Interactive, diagnostic decision making cases: Cough, chest pain • Describe role • Peer review and public dissemination • Website: xx hits; xx schools adapted or adopted at least 1 case; representative feedback comments • Accepted by national peer-reviewed venue; e.g., MedEdPORTAL

32. Curriculum DevelopmentDefinition and Inclusions • Longitudinal designed educational activities • Any training level • Various venues and delivery formats • Includes • Goals and objectives • Learning experiences to achieve goals & objectives • Organization and sequencing to ensure effective learning • Evaluation of effectiveness

33. Curriculum DevelopmentActivity and Role • Title of course • Evidence-based Medicine (Yr. 1 students) • Role: Course director • Organized all lectures and self-directed learning exercises for 5 credit hour course • Developed objectives and content • Recruited and trained facilitators • Worked with clinicians and statisticians to develop illustrative cases

34. Engagement with Community* • Clear Goals • Create a new EBM course for all first-year students that students perceive as clinically relevant. The predecessor course consistently received “very poor student evaluations” and EBM content was “lacking in the curriculum.” • Adequate Preparation • PhD in Public Health • Review of “best practices: McMaster’s curriculum, NBME test content • Review of literature; application to new course *Applying Glassick’s Criteria

35. Engagement: Glassick • Appropriate Methods • Multi-method approach including: • Interactive lecture series • Clinical vignettes • Abstract followed by article critique • Increased collaboration • Small groups co-led by basic scientist/MD • Assessment methods: Test questions assessed application of knowledge

36. Engagement • Significant Results (Outcomes) • Didactic series evaluation • Improved 1 SD from prior offering • AAMC Graduation Questionnaire • EBM rating improved from inadequate exposure to appropriate/excessive • USMLE sub-test score • Improved from below to above average • OSCE performance on ambulatory practice module • Improved

37. Engagement with Community:Scholarship (Contributing to) • Effective Presentation (Dissemination) • Results presented to curriculum committee • Internal review; comparison over last 2 years • Content replicated in clerkships & residencies • Reflective Critique (Next steps for continuing improvement) • Convert course to on-line format to minimize # faculty needed • Submit to AAMC MedEdPORTAL (Peer review)

38. Advising and MentoringDefinition and Inclusions • Advising and mentoring • Educator provides guidance or counsel to facilitate accomplishment of a learner’s or colleague’s goals • Advising • Limited time; advisor serves as guide to enable advisee to achieve her or his goals • Mentoring • Sustained relationship; mentor and protégée obtain reciprocal benefits

39. Advising & Mentoring

40. Educational Leadership Definition and Inclusions • Definition • Exceptional leadership transforms educational programs and advances field • Inclusion criteria • Pursues excellence • Evaluates and engages in self-reflection • Builds on work of others • Disseminates results – advances the field • Garners and maximizes resources

41. Leadership Documentation • Quantity • Describes nature of activity, duration, rationale for change, goals, and leader’s role • Quality • Describes actions, accomplishments, evaluation and resources garnered/utilized • Engagement with educational community • Builds on literature and best practices • Disseminates and reflectively critiques

42. Leadership ProjectBasic Science/Clinical Integration • Need • Courses lacked clinical relevance, poorly taught • Goal • Increase integration across 4 years, clinical relevance, quality • Methods and preparation • Revised courses • Conducted faculty development during 2004-06 • Garnered resources: Support staff and faculty time • Results (Evaluation) • Presentation (Peer review and dissemination • 6 peer reviewed activities • Reflective Critique • Next year plan to… 1 = Poor 5 = Outstanding

43. Learner Assessment Definition and Inclusions • Activities associated with measuring learners’ knowledge, skills, and attitudes • Assessment-related activity(ies) include: • Development • Identifying and creating assessment processes and tools • Implementation • Collecting data using processes and tools • Analysis • Comparing data to correct answer key and/or performance standards • Synthesis and presentation • Interpreting and reporting results

44. Learner AssessmentDocumentation • Quantity • Type of assessment • Learner population • E.g., who and how many • Size/scope • E.g., number of items, frequency of use • Intended uses of results • Faculty member’s role • Quality and engagement • Evidence of adherence to Glassick’s criteria

45. Learner Assessment Example of Presentation

46. Results: Caveats & Remaining ?s • Each institution must determine what counts for promotion • Contributions to institution’s mission (Q2) • Q2 within and across categories • Sustained contributions? • Contributions to the field (Engagement) • Draw from and contribute to field • Sustained contributions? • How to value individual vs group contributions • Core elements education infrastructure

47. GEA Scholarship ProjectNext Steps Phase 4 2007- ? Phase 1 1996-2000 Phase 2 2001-2004 Phase 3 2005-06 • Dissemination & Infrastructure: Individual & Institutional to Support Educators • AAMC-GEA Annual & Regional Meetings + Publish • Institutional Team Workshops • Individual Faculty Workshops