Getting Ahead of the Course Evaluation Curve Using student course evaluations to learn about courses, curriculum, and st

1. Getting Ahead of the Course Evaluation Curve Using student course evaluations to learn about courses, curriculum, and students with CoursEval from Academic Management Systems Peter Gold pgold@academicmanagement.com www.academicmanagement.com

2. Peter Gold, College of Arts and Sciences 1-12-07

4. Peter Gold, College of Arts and Sciences 1-12-07

5. What faculty, staff, administrators & students will learn using CoursEval What is rated? difficulty? learning? satisfaction? benchmarking with existing campus questions customizing questions for program evaluation What is not rated? expertise professional preparation How important are ratings biases and benchmarks? overall outcome stable and reliable Can we identify excellent and poor teaching? faculty, chair, dean, provost, student, program, administrators

6. Peter Gold, College of Arts and Sciences 3/15/06

8. Rating: Composition, Calculus, PSY101, World Civ F’05 [‘I would recommend this instructor to other students’] strongly disagree disagree neutral agree strongly agree |----------|----------|----------|----------| {31 DEP’T MEANS} & COMBINED MEAN{……………………….} ENG101102201{…………………………………………….…………………..} MTH121131141{……………………………………… ………………..……….} PSY 101207250{…………………………………………….……………...} UGC111112211{…………………………….………...………………….} Summary: Instructor mean ratings are generally positive Department mean ratings are all positive Beginning courses are rated highly Gen Ed courses are rated similarly to major courses Gen Ed means are similar to department means For UGC, REC sections are rated higher than LEC {....} RANGE of DEPARTMENT MEANS  ENGLISH (71, 30, 19 LEC secs) MATHEMATICS(12, 12, 12 LEC secs) PSYCHOLOGY (7,3,3 LEC secs) GEN ED PROG(16, 2, 13 LEC secs) Data from UBCATS F’05

10. Performance Difference Between Responders and Non-Responders *Each average was calculated among F’04 corresponding HIS, GLY, and SOC 100 level courses, 300 level courses, and 500 level courses. **Among all F’04 courses with high response rates including COM 1XX, PHY 1XX, PHI 2XX, DMS 3XX, CDS 6XX, PSY 6XX. Peter Gold, College of Arts and Sciences 10/12/05 Peter Gold, College of Arts and Sciences 10/12/05

11. TOP RATED LARGE LECTURES (FOR STUDENTS) FALL 06 Peter Gold, College of Arts and Sciences 5/15/06

12. Cautions, Issues, Biases of student-generated course and teacher evaluations • Students provide simplified evaluations and do not share with faculty a common understanding of questions and answers • Specific suggestions/complaints often accompany high ratings of ‘satisfaction’ with courses and teachers • Students do not distinguish clearly the issues of difficulty, time, expectations, achievement, learning, effort, grading, and curriculum • Students with higher GPA and higher grades are more likely to complete evaluation surveys but grades do not determine ratings • Lower response rates and large classes modify the ratings of courses and teachers but do not change the overall outcomes • Several other factors may affect student-generated ratings including random variations in enrollments among similar sections, time of day, course level, etc.