1 / 26

Ritual or reality: do student evaluations have any effect on teacher thinking and practices?

Ritual or reality: do student evaluations have any effect on teacher thinking and practices?. Presentation at the Australasian Higher Education Evaluation Forum (AHEEF), Christchurch, November, 2010 Sarah Stein, Jo Kennedy, Dorothy Spiller, Trudy Harris, Stuart Terry & Lynley Deaker .

shilah
Download Presentation

Ritual or reality: do student evaluations have any effect on teacher thinking and practices?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ritual or reality: do student evaluations have any effect on teacher thinking and practices? Presentation at the Australasian Higher Education Evaluation Forum (AHEEF), Christchurch, November, 2010 Sarah Stein, Jo Kennedy, Dorothy Spiller, Trudy Harris, Stuart Terry & Lynley Deaker

  2. The Project • Unlocking the Impact of Tertiary Teachers’ Perceptions of Student Evaluations of Teaching • Ako Aotearoa (National Centre for Tertiary Teaching Excellence) National Project Fund grant ($150k plus $20k extension) • 2010 - 2011

  3. Project Partners From Left: Lynley, Sarah, Trudy, Dorothy, Jo & Stuart • University of Otago • Sarah Stein • Jo Kennedy • LynleyDeaker • University of Waikato • Dorothy Spiller • Trudy Harris • Otago Polytechnic • Stuart Terry

  4. Research question • How do the current evaluation processes and practices influence teachers’ thinking and behaviours in relation to student learning at all stages of the teaching and learning cycle?

  5. Objectives • Explore tertiary teachers’ perceptions about student evaluations and identify how those perceptions impact on their thinking and practices. • Explore how tertiary teachers make use of information obtained from student evaluations at all stages of the teaching and learning cycle. • Compare teacher thinking and behaviours around evaluations across three participating institutions. • Make recommendations about evaluation processes so that evaluations can be optimally used for teaching development and enhancing student learning.

  6. Where we are up to... • Literature Review – complete • Questionnaire - complete, data analysis continues • Interviews - underway

  7. Literature Review: Questions • gap between acceptance and use to inform practice • using multiple sources of evaluation-how could it help • teacher emotions and receiving student feedback • relationship between teaching beliefs and using evaluations to inform teaching

  8. Literature Review: Questions • What interpretation guidance and PD support would help in the use of evaluations to inform teaching? • What institutional support and incentives would help to integrate student evaluations and professional development? • What are the perceptions about the ownership of the student evaluation process?

  9. Progress to date: the questionnaire • Developed a questionnaire for teaching staff with some variation in language to contextualise to each institution • Piloted with a small sample from Otago and Waikato Universities • Online for 3 weeks from 22 April 2010 and sent to 2445 teaching staff. • Received 1065 responses – or 45%

  10. Progress to date: interviews • 20 interviews are being carried out at each institution • due for completion in early 2011 • Interviewees selected from the questionnaire respondents who volunteered – a range selected • to explore stories behind the questionnaire responses

  11. Next Steps – data analysis • analysing questionnaire data using a range of processes looking at comparisons and variances between and within the groups (ongoing) • coding of the qualitative data into themes – links with questions that emerged from the Literature Review • this analysis has informed the selection of interviewees and the questions to be asked during the interview

  12. Early Data Analysis Questionnaire data

  13. Teacher perceptions of student evaluation data & influences on practice • Q9 – To what extent do your reasons for using student evaluations influence your teaching decisions? (E.g., decisions about learning outcomes, teaching strategies, assessment, etc. 1=A great deal, 5=Not at all) • Q11 – To what extent does your institution’s use of student evaluation data influence your teaching decisions? (E.g., decisions about learning outcomes, teaching strategies, assessment, etc. 1=A great deal, 5=Not at all) • Q13 – My course design refinements are influenced by student evaluation results: (1=A great deal, 5=Not at all) • Q15 – My willingness to try new teaching approaches is constrained by the possible negative effects on my student evaluations. (1=A great deal, 5=Not at all )

  14. Q9 – Teaching decisions influenced by reasons for doing student evaluations Otago Polytechnic differs significantly from the two Universities for Q9 (P-value=0.044) Q9 – To what extent do your reasons for using student evaluations influence your teaching decisions? For example: decisions about learning outcomes, teaching strategies, assessment, etc. 1=A great deal, 5=Not at all

  15. q10 Comment Data - Theme Coding – all Institutions

  16. Q 10 comments(reasons for influencing decisions) 1a I believe we must value how students interpret our teaching methods; process and especially how they value feedback with regard to their progress on a particular course or module. Students evolve intellectually and have a global perspective of where they fit in the world and change with it. We must be aware of those changes and expectations of our students globally; academically and culturally.

  17. Q 10 – comments(reasons for influencing decisions) 1bNot everything is popular! Being confident (but not arrogant) as a teacher is important as well as being reflective. I continually reflect on my teaching.

  18. Q 10 – comments(reasons against influencing decisions) 2aIt is my perception that the evaluations are largely a measure of popularity rather than of learning/teaching. I believe that the students are generally not in a position at the time the evaluation is set to comment on their learning outcomes or my teaching.

  19. Q 10 – comments(reasons against influencing decisions) 2bAfter 15 years I can tell if a course has worked or not. I am currently revising a whole course because I or it was not relevant to the students. I can see that in their faces; attentiveness and attendence/ participation.I don't need a survey to tell me that. Although it did confirm it.

  20. Q9 & Years Teaching in the Tertiary Sector There was a significant difference within the groups based on the number of years experience in tertiary teaching (P-value=0.028) Q9 – To what extent do your reasons for using student evaluations influence your teaching decisions? For example: decisions about learning outcomes, teaching strategies, assessment, etc. 1=A great deal, 5=Not at all

  21. Q11 – Institutional use of student evaluations as an influence on teaching decisions All 3 institutions are significantly different from each other (P-value=0.000), with Otago Polytechnic staff the most influenced. Q11 – To what extent does your institution’s use of student evaluation data influence your teaching decisions? E.g., : decisions about learning outcomes, teaching strategies, assessment, etc. 1=A great deal, 5=Not at all

  22. Q13 - Course/paper design refinements influenced by student evaluations Both Universities are significantly different to Otago Polytechnic staff (P-value=0.004), but not from each other. Polytechnic staff have more of a perception that student evaluation results influence course/paper design. Q13 – My course design refinements are influenced by student evaluation results: 1=A great deal, 5=Not at all

  23. Q15 - Teachers willingness to try new teaching methods constrained by evaluations There is a significant difference between Otago University and the other two institutions regarding teacher perceptions that student evaluation results constrain their willingness to try new teaching methods (P-value=0.000) Q15 – My willingness to try new teaching approaches is constrained by the possible negative effects on my student evaluations. 1=A great deal, 5=Not at all

  24. Q4 – teachers’ activity after they receive results • The categories that were most frequently used were: • Q4(b) Read the open question comments made by the students - 86% rated 1 or 2 • Q4(a) Spend time going over the data and responses – 80% rated 1 or 2 • Q4(g) Actively look for feedback about teaching and assessment – 72% rated 1 or 2 • Q4(d) Compare the data with previous evaluations – 57% rated 1 or 2 • Q4(e) Discuss the results with colleagues/teaching team – 55% rated 1 or 2 • Least used were: • Q4(f) Seek assistance with interpreting the results from others – 19% rated 1 or 2 with 55% rating 4 or 5 • Q4(c) Provide students with feedback on the results – 15% rated 1&2 with 62% rating 4 or 5

  25. Q4 – teachers’ activity after they receive results There is a significant difference between institutions (F=21.210, p=0.000). The difference is between the two universities and the Otago Polytechnic. Q4 – When you receive the results from your appraisals/evaluations do you… 1=always, 5=never

  26. And now…. • Watch this space….

More Related