Benefits of Early "Do-It-Yourself" Teaching Evaluation

1. Benefits of Early "Do-It-Yourself" Teaching Evaluation Dr. David McConnell MEAS Dr. Meg Blanchard Science Ed Dr. Chris Osburn MEAS Ever seen this in your classroom?

2. Why evaluate teaching? • Formative evaluations • Provide feedback to improve teaching effectiveness • Summative evaluation • Provide information to assist administration in making personnel decisions

3. Fact or Fiction The average student‘s attention span is between 10 and 20 minutes Fact (Penner, 1984). But the average college class lasts for 60-90 minutes. Deal with drifting attention by a lively lecturing style, and by breaking up longer lecture into 10-15 minute chunks (Atherton, 2001).

4. Fact or Fiction? Active learning is almost impossible in large lecture sections Fiction Active learning formats that work well are cooperative group work (Johnson, Johnson & Smith, 1991), class-wide discussions (Gullette, 1992), and interactive lectures (Mazur, 1993; Sokoloff, 1994; Van Heuvelen, 1991).

5. Fact or Fiction? Current trends point to more and more large class sizes –small classes may be a thing of the past Fact (McKinnon, 1998). You can exploit advantages of size and diversity with many activities that work better with more students (Wolfman, 2002).

6. For example: You can do this with a large class http://www.youtube.com/watch?v=8O4A903Thlw Pair Share: Confer with a neighbor for 1 minute to try to come up with something you can do or do better with a large class.

7. Fact or Fiction Lecture is as effective as other methods in promoting independent thought or developing students' thinking skills Fiction However, research shows that lecturing is as effective as other instructional methods in transmitting information (Bligh, 1971).

8. Fact or Fiction Research has shown that large class sizes are not necessarily detrimental to education. Fact It is the quality of the instructon that has the greater effect (Godfrey, 1998).

9. Students’ complaints about large class instruction • Lack of interaction • Little attention to personal needs • Little variation (Boring) • Afraid to ask a question

10. Education is not the filling of a pail but the lighting of a fire.

11. About student evaluations • Reliability, validity of student ratings is generally robust • Moderate to strong correlations between student ratings and student performance on final exams • Students well prepared to assess • Clarity of presentation • Instructor-student rapport • Concern for students’ progress Paulsen, M.B., New Directions for Institutional Research, 2002. #114: p. 5-18.

12. About student evaluations Some Caveats • Electives get higher ratings than required courses • Instructors in graduate and upper division courses receive higher ratings than undergraduate and lower division courses • Students ratings are higher when • Instructor is present • Students know evaluations are used for personal decisions • They are not anonymous Paulsen, M.B., New Directions for Institutional Research, 2002. #114: p. 5-18.

13. Instructor Behaviors and Student Evaluations • Student observers visited classrooms of 18 instructors with high teacher evaluation ratings, 18 with low ratings, and 18 with intermediate ratings. • All professors had been teaching for at least three years and had a class with at least 30 students. • All were in the social sciences and represented six departments. • Each teacher was observed at least three times by 6-8 different student observers using a 60-item Teacher Behavior Index (TBI). Murray, H., Journal of Educational Psychology, 1983. 75(1): p. 138-149.

14. Instructor Behaviors and Student Evaluations Which 6 behaviors showed significant differences between low- and high-rated instructors and which 6 behaviors were scored evenly between instructors with different ratings. • Speaks expressively or emphatically • Puts outline of lecture on board • States teaching objectives • Shows strong interest in subject • Advises students about tests • Moves about while lecturing • Uses humor • Smiles or laughs • Uses audiovisual aids • Uses graphs and diagrams • Addresses students by name • Asks questions of individual students Murray, H., Journal of Educational Psychology, 1983. 75(1): p. 138-149.

15. How Can We Evaluate our Teaching? • Self-review of actual teaching (video) • Student review using Group Instructional Feedback Technique • Peer review using Reformed Teaching Observation Protocol

16. The Blanchard Challenge: Experience your class as your students do. Videotape one class Watch the tape Reflect: • How did it go? • What did I want my students to learn? • What would I do differently next time? “Dr. McConnell, may I be excused? My brain is full.”

17. Pedagogical Discontentment Recent study: • 2-years with 28 middle school mathematics and science teachers • Teachers who were less satisfied with what they were doing were 8 times more likely to change their practices to more reform-based (with PD) than teachers who were more satisfied with their teaching (Blanchard, Osborne, and Albert, 2010). Watching a video of your teaching can help you to explore aspects of your teaching that you may find less satisfying.

18. Good Practices for Undergraduate Teaching • Don’t have a video camera handy? Try these: • Encourage student-faculty contact • Develop cooperation among students • Encourage active learning • Provide prompt feedback • Emphasize time on task • Communicate high expectations • Respect diverse talents and ways of learning Review handout checklists for 70 potential practices Chickering & Gamson, AAHE Bulletin, 1987, p. 3-7

19. How Can We Evaluate our Teaching? • Self-review of actual teaching (video) • Student review using Group Instructional Feedback Technique • Peer review using Reformed Teaching Observation Protocol

20. Group Instructional Feedback Technique What is working?What is going well in this class? What is helping you learn? What isn't?Give an example of one or two things that make it more difficult for you to learn in this course. What can be improved?What could use improvement in this course, and what specific suggestions would you make for change? Angelo, T. A. & Cross, K. P. 1993. Classroom Assessment Techniques: Jossey-Bass.)

21. Group Instructional Feedback Technique Case Study Dr. Christopher Osburn, MEAS

22. How Can We Evaluate our Teaching? • Self-review of actual teaching (video) • Student review using Group Instructional Feedback Technique • Peer review using Reformed Teaching Observation Protocol

23. About peer evaluations • Reliability, validity of peer evaluations not as well established as student evaluations • Peers may be better than students in evaluating • Content mastery • Course design • Curriculum development Paulsen, M.B., New Directions for Institutional Research, 2002. #114: p. 5-18.

24. Reformed Teaching Observation Protocol RTOP: Cliffs Notes Version • Range from 0 “Never Occurred” to 5 “Very descriptive” • Use to see how teacher (0-2) to Learner (3-5) centered a lesson is across 25 items on 5 subscales • Score ranges form 0 – 100, with less than 50 being more traditional and above 50 is more reform-based • Descriptive, not judgmental—if it occurred, you code it, if not, you don’t. http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/

25. RTOP Data RTOP range, 20s-70s for Physical Sciences classes Higher RTOP scores correlate with higher learning gains

26. RTOP Data RTOP scores range from high teens to 80s in introductory geology classes Higher RTOP scores correlate with higher learning gains

27. Reformed Teaching Observation Protocol RTOP: 5 Subscales Lesson Design Content Knowledge Procedural Knowledge Communicative Interactions Student/Teacher Relationships Who determines the direction of the lesson, the teacher or the student? How strong is teacher’s content knowledge, connections to real world & conceptual understanding? How much ownership do students take over representing, testing, and reflecting on ideas? How much do students communicate with the teacher, and each other and what is the quality of those communications? How much listening and support and patience does the teacher have for the students’ ideas?

28. Group RTOP Activity Pick an RTOP Subscale Your Task: Review the 5 items in the subscale and discuss in your group: 1) Which item(s) do you most want to reform? 2) How might you do that (strategies)? 3) Present to the whole group Review RTOP rubric handout for tips on what to look for when using scoring scales

29. Benefits of Peer/Self-Evaluation √ Help with end-of-semester evaluations √ Part of faculty development efforts √ Personal reflection on teaching and learning √ Increase faculty credentials for internal and external teaching awards √ Data for use in program review and accreditation √ More enjoyment √ More student learning

31. Fact or Fiction? When students have copies of the lecture notes or a text they are less likely to come to a traditional lecture class Fact A significant percentage of students prefer reading notes rather than attending classes that offer little or no interaction (Edlich 1993; Sullivan & McIntosh, 2002).