Alternative Strategies for Evaluating Teaching

1. Alternative Strategies for Evaluating Teaching Bill Burke Program Manager for Educational Development Teaching and Academic Support Center How many have used end-of-semester student evaluations? How many have used an alternative approach? My comments are based in part on “Evaluating Your Own Teaching” by Dee Fink (published in Improving College Teaching by Peter Seldin (Ed.)

2. Alternative Strategies for Evaluating Teaching Doing good teaching evaluations is like doing good research Need to identify the right questions to ask Need to figure out how to get the data to answer them Teaching evaluations can serve two purposes – Formative feedback Summative assessment

3. Alternative Strategies for Evaluating Teaching Use a multidimensional approach to evaluating what is a multivariable activity A good evaluation program should be ongoing, strategic, comprehensive, multidimensional, integrated, and evolving Five sources of evaluation feedback Student feedback Self-evaluation Recordings Student performance Outside evaluators All sources have unique values and inherent limitations

4. Alternative Strategies for Evaluating Teaching Student Feedback Can be done during the semester and/or end-of-semester 1) Questionnaires + Obtain feedback from whole class + Anonymous + Can provide quantitative and qualitative data - Questions may not be appropriate or relevant or most informative - No chance to probe for further clarification

5. Alternative Strategies for Evaluating Teaching Misconceptions about student evaluations Students don’t know enough to evaluate faculty Those multiple-choice forms can’t be as meaningful as open-ended questions and interviews Easy graders and entertainers get the high evaluations Rigorous instructors will get low evaluations Students may rate me low now, but they’ll appreciate me later

6. Alternative Strategies for Evaluating Teaching Fact: Student input on some items (holding office hours, returning papers in a timely manner) is reliable and the best source Fact: High correlation between student ratings and other sources such as peers and administrators Fact: Comparison of objective questions, written responses to open-ended questions, and group interviews yielded a correlation of 0.82 Fact: Conflicting findings exist on grading and evaluations (But can’t assume high grades & high ratings means no rigor)

7. Alternative Strategies for Evaluating Teaching Fact: Teachers assigning more work and more difficult work tend to get higher evaluations Fact: Students don’t change their opinions later

8. Alternative Strategies for Evaluating Teaching Making student evaluations more meaningful Have a clear idea about your teaching goals and learning outcomes Do evaluations provide feedback on these? If not, ask the students to comment on specific items (write them on the board)

9. Alternative Strategies for Evaluating Teaching Making student evaluations more meaningful Focus on specifics versus global items Focus on teaching characteristics versus personal characteristics Target one or two items to work on Use more than one course (a variety of courses over several semesters)

10. Alternative Strategies for Evaluating Teaching Making Sense of Written Comments Harder to make sense of written comments Some are contradictory Need to impose some structure through systematic analysis

11. Alternative Strategies for Evaluating Teaching Making Sense of Written Comments Cluster evaluations according to overall course ratings Similar comments between high and low raters? Cluster according to comments Do patterns emerge? Use student demographic data when possible and appropriate

12. Alternative Strategies for Evaluating Teaching Making Sense of Written Comments Make a list of common positive and negative characteristics of teachers Put checkmarks next to them based on student comments Helps quantify the diversity of comments Produces a visual as well as quantitative display

13. Alternative Strategies for Evaluating Teaching Making Sense of Written Comments Ask students to comment on – What made you rate the course as you did? What kept you from rating the course higher?

14. Alternative Strategies for Evaluating Teaching Student Feedback 2) Interviews (by you or a third party) Focus group Whole class (e.g., Group Instructional Feedback Technique [GIFT]) + Students identify unanticipated strengths and weaknesses + Interviewer can probe - Whole class may not be represented

15. Alternative Strategies for Evaluating Teaching Self-Evaluation Personal observations during class Reflections / journal Teaching portfolio Teaching philosophy statement - personal and descriptive Evidence of putting philosophy into action Evidence of growth as a teacher You have a voice here Evaluators need a rubric

16. Evaluators need a rubric examples

18. Alternative Strategies for Evaluating Teaching Self-Evaluation + Immediate and constant feedback + Meaningful to you - Subject to your biases, misconceptions, and delusions Self-assess your perceived teaching strengths and weaknesses Get feedback to confirm or refute

19. Alternative Strategies for Evaluating Teaching Recordings Audio and/or Video + Provides objective information - Information is true but may be meaningless by itself in determining impact on student learning

20. Alternative Strategies for Evaluating Teaching Student Performance Course assessments (exams, papers, etc.) Pre / Post tests Classroom research projects + Provides evidence of student learning - Lack of an unequivocal causal relationship

21. Alternative Strategies for Evaluating Teaching Outside Evaluators Observations of teaching Review of materials + Can offer positive and negative observations without a personal cost + Bring professional expertise in content and/or pedagogy - Limited number of class visits (snap shot perspective) Need an agreement on what is to be observed or reviewed and criteria for judgment

22. Alternative Strategies for Evaluating Teaching Outside Evaluators Conducted by - Peers + Low political risk + Empathy - Limited experience and perspective Senior faculty / Administrators + Experience - Political risk Instructional specialists from a center + Objective with no political risk + Expertise in instructional strategies - Limited knowledge of subject matter

23. Alternative Strategies for Evaluating Teaching Use a multidimensional approach to evaluating what is a multivariable activity A good evaluation program should be ongoing, strategic, comprehensive, multidimensional, integrated, and evolving Questions? Comments?

24. Alternative Strategies for Evaluating Teaching