150 likes | 172 Views
EVALUATION OF COURSE EVALUATIONS. by Prof. Peter Friis-Hansen, Assoc. Prof. Niels Houbak and Prof. Peder Klit Department for Mechanical Engineering (MEK), Technical University of Denmark (DTU) DK-2800 Kgs. Lyngby, Denmark. Evaluation of courses ?. When is a course a good course?.
E N D
EVALUATION OF COURSE EVALUATIONS. by Prof. Peter Friis-Hansen, Assoc. Prof. Niels Houbak and Prof. Peder Klit Department for Mechanical Engineering (MEK), Technical University of Denmark (DTU) DK-2800 Kgs. Lyngby, Denmark.
Evaluation of courses ? When is a course a good course? • When the lecturer senses the success. • When most students pass the exam • When the students are happy • When the industry are pleased with the candidates • When the students have learned enough to …. • continue How do we measure it? (the engineering approach) • Attendance • Percentage passing the course exam • Grade average • Student satisfaction • Lecturer satisfaction • Something else?
Final Evaluation Questionnaire • 3 sections. A: The course, B: The lecturers, C: Written comments • Has been used for more than 15 years, the last 3 years in electronic form. • Must be completed one week before end of term (discussed at the last lecture) • Is important for the lecturer and for the department management (study board) • Information is included in overall evaluations of staffs and studies
The Questionnaire. Section A • 1. Are the prescribed course prerequisites adequate? • 2. How is the course material? • 3. Is the form of the course adequate? • 4. A standard course has an average workload of 9 hours pr week; how much time did you spend? • 5. How many lectures did you attend? • 6. What is your general satisfaction with the course? • 7. For courses taught in English; did this influence your outcome?
The Questionnaire. Section B • How did the (named) lecturer present the subject? • Is the lecturer inspiring? • How did you experience the dialog/cooperation with the lecturer? • How is the lecturer as supervisor? • 5. For group works; did you receive criticism for handed in exercises during the course? Section C • I appreciate • I criticize • I suggest
Post processing. • For each question an average value is calculated • Questions are weighed against one another • Average values are multiplied with weights and summed • A course/lecturer utility number is the outcome • This is a subjective one number [0-10] evaluation – far from any truth! • BUT it may contain some information.
Evaluation action. Reactions from the study board • Letters to excellent performing lecturers • Letters to poor performing lecturers – filtered for good excuses. Important to notice, that the tone is: ”What do think you can do about this situation? Can we be of any help?” • Easy to locate/spot changes in behaviour (monitoring 50+ courses is hard)
Conclusions: • It is always worthwhile to evaluate what you are doing. • Evaluations can sometimes be used for something differently. • The anonymous comparison allows for lecturers to see his/her own performance relative to other staff members. • All lecturers notice that their evaluation result has been processed. It is not only the poor performers that are noticed. • The weighing factors indicate what is important when improving the teaching quality. • The defined “Utility-system” generates a “dynamic normal”, i.e. conscientious staff will try to improve the teaching quality to obtain a good position on the cumulative curves. • 7. Tests have shown that radical changes to the weights are needed to form a noticeable change in the ranking