Scholarship in Education

1. Scholarship in Education Rose van Zuilen, PhD Director of Faculty Development Co-Director of Longitudinal Curriculum in Geriatrics, Pain Management, and Palliative Care Division of Gerontology and Geriatric Medicine University of Miami Miller School of Medicine

2. Learning Objectives • Define educational scholarship and distinguish it from other forms of scholarship • Describe a broad range of activities that constitute educational scholarship • Determine existing outlets to publish educational products • Utilize effective ways to capture and document the impact and quality of your scholarly work

4. Department of Medicine, Thomas Jefferson University

5. Typical Model of Scholarship • Scholarship of Research Clinical Care Teaching Service

6. Academic Advancement is often Slower for Clinician Educators • Study at Johns Hopkins School of Medicine • A single track for promotions • The odds of being at a higher rank were 85% less for academic clinicians and 69% less for teacher clinicians than for basic researchers • Rigor of promotion process affected by the paucity of valid evaluation methods Thomas PA et al.(2004) Acad Med 79(3): 258‐264

7. Expanding the Definition of Scholarship • “What we need, then, in higher education is a reward system that reflects the diversity of our institutions and the breadth of scholarship, as well. The challenge is to strike a balance among teaching, research, and service, a position supported by two-thirds of today’s faculty who conclude that, ‘‘at my institution, we need better ways, besides publication, to evaluate scholarly performance of faculty.’’ Boyer, 1990

8. Expanding the Definition of Scholarship

10. Glassick’s Scholarship Assessed • 1) Clear goals – the educator explicitly states the basic purposes for the work, and defines realistic, achievable objectives, including desired goals and outcomes. • 2) Adequate preparation – the educator shows an understanding of relevant existing scholarship and has skills and resources drawn from this research and from prior experience to advance the project. • 3) Appropriate methods – in conjunction with the material and the context, the educator chooses, applies and, if necessary, modifies methods wisely.

11. Glassick’s Scholarship Assessed • 4) Significant results – the educator achieves the goals, and contributes notably to the field in a manner that invites further exploration. • 5) Effective presentation – the educator uses a suitable style and organization to present the work with clarity and integrity in appropriate forums to reach the intended audience. • 6) Reflective critique – the educator thoughtfully assesses the work him/herself and uses the resulting perceptions, along with reviews and critique from others, to refine, enhance or expand the original concept.

12. Association of American Medical Colleges Consensus Conference • Group on Educational Affairs (GEA) met in 2006 • Criteria for educational scholarship • Five categories • Teaching • Learner assessment • Curriculum development • Mentoring and advising • Educational leadership and administration

13. Issues Addressed by the GEA • 1. What educator activities should be included in each category? • 2. What types of evidence should be included for review in the academic promotion process? • 3. How should a faculty member present the activities and associated evidence in a promotion/educator portfolio?

14. Issues Addressed by the GEA • Educators’ contributions can be judged through the effective presentation of evidence pertaining to the quantity, quality, and engagement with the educational community • Evidence of such engagements requires educators to draw upon the literature and best practices (scholarly approach) or contribute to the medical education field (scholarship).

15. Category 1 - Teaching • Any activity that fosters learning, including direct teaching and the creation of associated learning materials • Lectures, small groups, precepting, role modeling, formative assessments and feedback, online courses • Creation of learner materials strictly for enhancing one’s own teaching

16. Teaching Documenting Quantity • Document the frequency and duration of teaching with a description of one’s role at all levels (UME, GME, CME) • Provide a list of instructional materials authored with a brief description of their purpose, format, and length

18. Teaching Documenting Quality • Learner’s evaluations of teaching performance • Peer evaluations of teaching • Teaching awards and honors • Describe the selection process and criteria • Evidence of learning • Pre-post assessments of learner performance • Self-reported learning

19. Teaching Documenting a Scholarly Approach • Describe how one’s teaching is informed by the literature (e.g., competence based education) • Provide a comparative analysis of one’s own teaching and teaching materials with “best practices” in the field • Reflect critically on one’s own teaching and on feedback received by others noting the impact on subsequent teaching activities • Document participation in personal learning

20. Teaching Documenting Scholarship • Evaluations from conference presentations • Inclusion of educational products in a peer-reviewed repository • Data demonstrating adoption by other faculty • References or citations to the product in other peer-reviewed materials

27. Category 2 - Learner Assessment • All activities associated with measuring learner’s knowledge, skills and attitudes • Development: Identifying and creating assessment process and tools • Implementation: Collecting data • Analysis: Comparing data with correct answer key or performance standards • Synthesis and interpretation: Interpreting and reporting data to stakeholders

28. Learner Assessment Documenting Quantity • Provide information about the size and scope of the assessment activity, one’s role, and the number and level of learners assessed Documenting Quality and a Scholarly Approach • Provide evidence that the assessment tool is reliable, valid and informed by the existing educational measurement field.

29. Learner Assessment Documenting Scholarship • Presentations on the assessment process or outcomes to local audiences • Peer reviewed or invited presentations and workshops at professional meetings • Acceptance of the assessment tool in a peer-reviewed repository • Assessment research presented at national conferences or published • Invitations to review assessment tools

31. Category 3 – Curriculum Development • Curriculum is defined as a longitudinal set of designed educational activities that includes evaluation • More than one teaching session or presentation • Can occur at any level of training • Basic science course, clerkship, theme, rotation, fellowship, CME course

32. Curriculum Development Documenting Quantity and Quality • For each curricular piece authored, provide a description of the purpose, intended audience, duration, design and evaluation. • Note specific contributions (if co-authored) • Provide learner reactions and ratings • Show outcomes with graphic displays of improvements over time when possible • Course examinations, NBME subject scores, OSCEs, performance observations

33. Curriculum Development Documenting a Scholarly Approach • Demonstrate that the curriculum design was informed by the literature and “best practices” • a curriculum designed to target nationally accepted competencies • The use of a well-accepted approach to curriculum development (e.g., ADDIE)

34. Curriculum Development Documenting Scholarship • Peer review at the local level (curriculum committee, accreditation reviewers) • Peer reviewed or invited presentations • Acceptance of curricular material to a peer-reviewed repository • List of institutions that adapted the curriculum • Requests to provide consultation • Invitations to conduct peer-reviews

36. Peer Reviews – Getting Involved • Poster sessions (local and national) • Opportunities through colleagues and societies • Conference abstracts and papers • Invited as a society member, referral, and volunteer • Journal articles • Published author or recommended content expert • Solicited or unsolicited applications • Educational products (e.g., MedEdPORTAL) • Solicited applications … so, volunteer • Grants and awards

37. Things to Consider Before Signing Up as a Reviewer • Make sure the journal is a good fit • How many manuscripts does the journal expect you to review – in what time period? • Is there a specific format for reviews • Do reviewers receive feedback? • Is the review system open or closed Sylvia LM & Herbel JL.(2001) Pharmacotherapy, 21(4)

38. Category 4 – Mentoring and Advising • Educators often serve as mentors and advisors for their trainees and colleagues • Mentoring implies a sustained committed relationship from which both parties obtain reciprocal benefit, whereas advising is usually more time limited and one directional • Documentation should describe the nature of the relationship and one’s effectiveness in helping mentees and advisees meet their goals

39. Mentoring and Advising Documenting Quantity and Quality • Provide a list of mentees with their position and time invested (duration, hours) • List mentees significant accomplishments • Publications, presentations, grants, awards • Standardized evaluation of advising effectiveness • Narrative comments or letters from mentees • Solicited or unsolicited – Keep records

40. Mentoring and Advising Documenting a Scholarly Approach • Participation in professional development • Use of mentoring strategies informed by literature • Design of a program guided by current evidence • Leading of initiatives to improve institutional mentoring and advising practices

41. Mentoring and Advising Documenting Scholarship • Secure competitive funding for program development or innovative mentoring projects • Conduct skills enhancement training • Publish peer-reviewed mentoring guides • Receive mentoring awards • Convene a scholarly conference on mentoring • Initiate a faculty learning community on mentoring • Serve as a mentoring consultant • Conduct mentoring research

42. Category 5 – Educational Leadership and Administration • Exceptional educational administrators and leaders achieve transform organizations in their pursuit of excellence • Their work's value is demonstrated through ongoing evaluation, dissemination of results, and maximization of resources. • Leadership roles are varied • Clerkship or course director, associate dean, educational committee membership, educational grant PI, conference organizer

43. Educational Leadership and Administration Documenting Quantity and Quality • Describe the quantity, nature and duration of administrative and leadership roles • Describe the change that occurred as a result of your leadership (problems identified, goals established, action taken) • Provide outcomes (e.g., MCAT scores of applicants have increased, clerkship evaluations have improved, AAMC GQ ratings improved)

44. Educational Leadership and Administration Documenting a Scholarly Approach • Creatively design and evaluate improvements • Show you are making revision based on local feedback, best practices, external peer review • Document ongoing QI • Use evaluation tools to measure outcomes • Pre-post assessments, accreditation surveys, cohort performance on national exams

45. Educational Leadership and Administration Documenting a Scholarly Approach • Demonstrate attainment of objectives or benchmarks (AAMC GQ, course evals) • Use a 360-degree evaluation with peer comparisons, benchmarks, or external review • Employ self-reflection informed by best practices in the field

46. Educational Leadership and Administration Documenting Scholarship • List invited and peer-reviewed presentations and visiting professorship presentations • Document awards received (e.g., best paper) • Provide a list of institutions that have adopted an innovation • Curriculum model published on MedEdPORTAL • List resources obtained • Foundation support, grants, internal awards

47. Funding Sources for Medical Education Research Grants • Agency for Healthcare Research and Quality (AHRQ) Grants • http://www.ahrq.gov/fund/grantix.htm • American Association of Neurology Educational Research Grants: • http://www.aan.com/go/education/eduresearch • Amgen Foundation • http://www.amgen.com/citizenship/IME_overview.html • Arnold P. Gold Foundation: • http://humanism-in-medicine.org/index.php/programs_grants • Arthur Vining Davis Foundations: • http://www.avdf.org/FoundationsPrograms/HealthCare.aspx • Association of Professors of Gynecology and Obstetrics (APGO) • https://www.apgo.org/grants.html • Association for Surgical Education Foundation - Center for Excellence in Surgical Education, Research and Training. • http://www.surgicaleducation.com/mc/page.do?sitePageId=28551&orgId=ase

48. Funding Sources for Medical Education Research Grants - Continued • AstraZeneca Medical Education Research Grants: • http://www.astrazenecagrants.com/ • Fund for the Improvement of Postsecondary Education (FIPSE): • http://www2.ed.gov/about/offices/list/ope/fipse/index.html • The Henry J. Kaiser Family Foundation, • http://www.kff.org/ • HRSA-U. S. Department of Health and Human Services • http://www.hrsa.gov/ • Josiah Marcy, Jr. Foundation: • http://www.macyfoundation.org/apply • National Institutes of Health • http://grants1.nih.gov/grants/index.cfm • NBME Stemmler Medical Education Research Fund: • http://www.nbme.org/research/stemmler.html

49. Funding Sources for Medical Education Research Grants - Continued • NSF Directorate for Education and Human Resources: • http://www.nsf.gov/dir/index.jsp?org=EHR • The PEW Charitable Trust: • http://www.pewtrusts.com/ • Pfizer Medical Education Grants: • http://www.pfizer.com/responsibility/grants_contributions/medical_education_grants.jsp • RSNA Foundation Radiology Education Grants: • https://www.rsna.org/Education_Scholar_Grant.aspx • https://www.rsna.org/Radiology_Education_Research_Development_Grant.aspx • The Robert Wood Johnson Foundation: • http://www.rwjf.org/index.jsp • Society for Academic Continuing Medical Education, Research Grants in Continuing Medical Education • http://www.sacme.org/SACME_grants

50. For Promotion… • Educators can present evidence focused on one category or in multiple categories • Types of evidence may vary but documentation should be quantitative and qualitative • Breadth of engagement (local, regional, national, or international) varies by faculty rank • Display information in easily digestible format using tables, figures, graphs