1 / 27

Faculty Development Issues in Evaluation

Faculty Development Issues in Evaluation. Definition Types of Programs Types of Evaluation Done Evidence of Success Believed Utility Reason for Continuation of programs Need / Evidence of Success Risks to current evaluation methods Impetus for future work.

kedem
Download Presentation

Faculty Development Issues in Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Faculty Development Issues in Evaluation • Definition • Types of Programs • Types of Evaluation Done • Evidence of Success • Believed Utility • Reason for Continuation of programs • Need / Evidence of Success • Risks to current evaluation methods • Impetus for future work

  2. Faculty Development Definition • activities designed to improve an individual’s knowledge and skills in areas considered essential to the performance of a faculty member…… McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997

  3. Types of Faculty Development • Instructional development • Professional development • Leadership development • Organizational development Irby, 1995, from Berquist and Phillips

  4. Faculty Development Schemes for Program Evaluation: • The Faculty Teacher Attitudes/Knowledge/Behaviors Pedagogy Pedagogical Content Knowledge Content Areas, e.g., End of Life, Geriatrics, Organizational Leadership, Professional Skills • Their Students • The System

  5. Faculty Development Schemes for Program Evaluation: • Reaction (customer satisfaction) • Faculty Learning (change in Knowledge, Skills, Attitudes) • Behavior Change (result of program) • Results (change in the system) Blumberg, Deveau, Medical Teacher, 1995

  6. Faculty Development Schemes for Program Evaluation: • Academic dissemination (presentations/publications) • Product development (materials/workshops) • Implementation ( #’s trained, maintained and disseminated) • Institutionalization Blumberg, Deveau, Medical Teacher, 1995

  7. Faculty Development Schemes for Program Evaluation: • Process Evaluation (participant ratings) • Content Evaluation (Change in knowledge, skills, attitudes) • Outcome Evaluation (long term change) • Impact Evaluation (Impact on the system) Holly, 1992

  8. Faculty Development Schemes for Program Evaluation: • The Faculty • Their Students • The System

  9. Faculty Development in CanadaEvaluation methods 3 outcomes: • performance • cognitive learning • self-assessment re teaching skills • Outcome measures – positive direction, but without statistical significance, but small N. Nasmith, Steinert, Saroyan, Daile, Franco, TchLng Med, 1997

  10. Lessons from Faculty Development in Clinical Teaching • Research Evidence for Benefits • Teachers and learners • POSITIVE RATINGS OF EXPERIENCE • IMPROVED EDUCATIONAL OUTCOMES FOR FACULTY - knowledge, skills, and attitudes • SELF-RATINGS • RATINGS OF BEHAVIORS ON VIDEOTAPES • KNOWLEDGE ACQUISITION BY TEACHERS • IMPROVED STUDENT RATINGS OF TEACHERS • IMPROVED OUTCOMES FOR LEARNERS -- MOSTLY IN HIGHER ED LITERATURE* • Institutional / Societal Benefits ? *Murray in Perry and Smart, 1997 Effective Teaching in Higher Education Stanford Faculty Development Program

  11. Faculty Development Outcomes • Reviewed different types of programs in family medicine • Fellowships • Workshops • Results • Changes measured, but no significant performance changes • Few publications • Different needs for different faculty, tenure vs non-tenure Reid, Stritter, Arndt, Fam Med, 1997

  12. Faculty Development Schemes for Program Evaluation: • The Faculty • Their Students • The System

  13. AAMC - 2000 Faculty Development /Faculty Affairs Offices: 76 of 125 institutions -> 93 faculty affairs and 21 development offices Moahan, et al, Acad Med, May 2002

  14. Faculty Development - Evaluation Faculty development offices focused on use of services and mentoring Moahan, et al, Acad Med, May 2002

  15. AAMC - 2000 Faculty Development /Faculty Affairs Offices: 76 of 125 institutions -> 93 faculty affairs and 21 development offices Moahan, et al, Acad Med, May 2002

  16. Faculty Development in Canada • 1986 – positive commitment – poor faculty participation • Recommendations: • establish committees • fund activities • Recognize participation in promotion/tenure • 1986-96: Major changes: • Health care financing • Downsizing • Inpatient -> outpatient • Multi-institutional and multidisiplinary care • Curriculum reform • Less stress on information acquisition • Problem solving / interpersonal skills/attitudes McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997

  17. Faculty Development in Canada • All 16 schools responded • 16 with Faculty Development Program vs 12 in 1986 • Sabbatical programs • Grants to improve instruction/ courses, with major emphasis on information technology/electronic media • Awards for teaching excellence • Increasing feedback on teaching from faculty or administrators (13/16 schools) • Systematic review of teaching by administrator- 6 schools • Senior faculty work with junior faculty – 2 schools • Workshops on management / research / scholarship • Fac Devel committees in 9 schools • ½ of schools indicate inadequate budget • Outcomes reported at 4 schools Stanford Faculty Development Program McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997

  18. Faculty Development in Canada • Most effective practices • Teaching instruction • Use of computers/information mngt • Sabbatical leaves • Teaching Awards • Inexpensive faculty development • Local faculty development stimulated by clinicians with M Ed Degrees Stanford Faculty Development Program McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997

  19. Faculty Development in Canada • Stimuli for change • Curricular change, needing more tutors • Curricular change – Uncertainty in faculty • Changes in health care – examination of career and personal goals McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997

  20. Faculty Development in Canada • Prognosis - - Good; Collegiality and Institutional Support • Problem – lack of evaluation on impact • Need to identify outcomes and methods Stanford Faculty Development Program McLeod, Steinert, Nasmith, Conochie, CMAJ-JMAC, 1997

  21. Lessons from Faculty Development in Clinical Teaching Research Commentary • Hard to do controlled experiments • Few measures of pre-post change • Few measures of long-term change • Mostly participant ratings • Few measures of actual behavioral change in faculty • Difficult for measures of changes in student learning • Little research on institutional change Stanford Faculty Development Center

  22. Faculty Development Outcomes • Are the data hidden in the process we are observing? • Curricular Change? • Emphasis on Teaching? • New Teaching Methods? • New Observations of Faculty Needs? • Requirements for Teaching Improvement Methods by Accrediting Agencies? • Areas for Further Work • Further Enhancement of Pedagogy – Improve the best • Pedagogical Content Knowledge (Shulman) • Needs of the Profession

  23. Types of Faculty DevelopmentAreas for Further Work • Instructional development • Professional development • Leadership development • Organizational development Irby, 1995, from Berquist and Phillips

  24. Types of Faculty DevelopmentYour Beliefs Instructional development • Should be improved? • Can be improved? • Has been improved? Professional development • Should be improved? • Can be improved? • Has been improved? Leadership development • Should be improved? • Can be improved? • Has been improved? Organizational development • Should be improved? • Can be improved? • Has been improved?

  25. Lessons from Faculty Development in Clinical Teaching • Current Challenges/Opportunities • INSTITUTIONAL • Other Alligators • Attitude - good enough mentality? • TIME / MONEY/ ADMINISTRATIVE SUPPORT? • EVIDENCE FOR NEED: Present, ? compelling • WHO IS GOING TO PAY? • TEACHER’S THEMSELVES • Philosophical/practical/self-assessment/ • METHODS: Applicabililty to current environment Stanford Faculty Development Center

  26. Areas for Increased Emphasis in Faculty Development • INSTITUTIONAL BENEFITS • IDENTIFICATION OF NEW AREAS FOR IMPROVEMENT - Consolidation seminar - ->> • STRUCTURAL RECOMMENDATIONS • APPLICATION OF KNOWLEDGE RECOMMENDATIONS FOR INSTITUTIONS, E.G., MDM POLICIES, PATIENT CARE • EDUCATIONAL RECOMMENDATIONS, E.G. TEACHING RESOURCES, ADMINISTRATIVE RESOURCES • SOCIETAL BENEFITS • IMPROVED OUTCOME FOR LEARNERS -->> FUTURE PHYSICIANS/SCIENTISTS Stanford Faculty Development Center

  27. Lessons from Faculty Development in Clinical Teaching Data vs Belief • Importance of Faculty Development • Needs of Educators - Academic / Community • Topics of interest and importance (Content and process) • Benefits of Development Methods • Current Challenges and Opportunities Stanford Faculty Development Program

More Related