Evaluations for Senior Classes

1. Evaluations for Senior Classes John S. Walton Animal & Poultry Science, OAC University of Guelph (in support of supervised examinations)

2. Outline • Why am I here? • What I do • Why I think supervised examinations are important • How I think these can be improved • What I am really trying to ‘sell’ today?

3. Why am I here? • Seemed like a good time for a ‘rant’ • Pure reaction to reports from the “Academic Integrity” sessions last year

4. I am somewhat concerned for the future of final examinations …. • Insufficient time allowed for grading (nothing new) • Students with conflicting final exam times (relatively new, on the increase)

5. Potential Solutions to Exam Conflicts: • Abandon exams • Ask the other professor to hold an alternate • Create an alternate

6. The Problem of Alternate Exams: • Time consuming • Ensuring equivalent course – coverage and rigor (“Fairness”) • Insufficient novel, meaningful and substantive questions for multiple sittings • For me # 3 is the major problem

7. Potential Outcomes: • Fewer final examinations • “Final” examinations that represent a minor (trivial) part of the overall grade • More unsupervised methods of evaluation with the attendant risk of… “FACILITATED ACADEMIC MISCONDUCT”

8. What I do: I teach – ANSC*3120 Animal Reproduction (3-3) W ANSC*4130 Reproductive Management and Technology (3-3) W ANSC*3120 96 students (‘04) is the prerequisite for ANSC*4130 38 students (‘04)

9. Evaluations: ANSC*3120 Midterm examination 40% (2 h) 20 questions (wk 1-6) Final examination 40% (2 h) 20 questions (wk 7-12) Laboratory books 20% ANSC*4130 (the course I’ll focus on primarily) *Midterm examination 30% (2 h) 3 questions (wk 1-6) *Final examination 30% (2 h) 3 questions (wk 7-12) Laboratories/Hands-on 40% *(6 questions, 3 are selected)

10. My students: ANSC*3120 W’04 BSc. (Agr) BSc.

11. The Correct Distribution? My written exams tended to (and still do) focus on “Do they know anything?” For example: ANSC*3120 Results High proportion of ‘A’ grades despite the fact no student really demonstrated an ‘A’ performance (examination is heavy on retention of factual material)

12. What is an ‘A’ Grade? 80-100 ‘A’ Excellent “An outstanding performance in which the student demonstrates a superior grasp of the subject matter AND AN ABILITY TO GO BEYOND THE GIVEN MATERIAL IN A CRITICAL AND CONSTRUCTIVE MANNER ….” UoG (2003)

13. ANSC*3120 20 questions Prerequisite WHAT DO THEY KNOW? CAN THEY THINK? easy ANSC*4130 3 questions Elective WHAT DO THEY KNOW? CAN THEY THINK? more difficult – but necessary

14. Why I think supervised examinations are important They provide an accurate method of assessing each student The risk of academic misconduct is relatively low (well-defined boundaries) Unsupervised examinations/assignments are relatively susceptible to plagiarism (“facilitated” academic misconduct)

16. How I think supervised examinations can be improved: • My problem has been: • Too many sittings required • Not enough novel questions • Asking the wrong sorts of questions • My current solution is: • Open questions/closed exam

17. For ANSC*4130, therefore Mid-term Final Identical format Both are a 2 h written exam (during wk 7 and during Final Exams)

18. Open Questions Questions published: week 3 (Midterm) week 9 (Final) 6 questions/exam (1 question per week) In pairs (answer either 1 or 2, etc)

19. Closed Exam • At the exam: • Questions to be answered (3 of 6) chosen by flipping a coin • Students write their answers without access to their notes or sample answers • Student identify their answer booklets using their STUDENT NUMBER ONLY

20. Sample Question: After graduation, you are contracted by Health Canada to assist with submissions to the Veterinary Drugs Directorate for approval of the CIDR-B* device for use in lactating dairy cattle. The request for the label-claim is for the regulation of ovulation in dairy cattle, including problem breeders. Review the currently available published evidence (North America) and develop an appropriate recommendation [i.e. prohibit, permit use, permit use with restrictions (milk withdrawal/slaughter)]. Be sure to frame this according to the approval criteria: 1) efficacy; 2) human safety; and 3) potential impact on the cow(short andlong-term). *CIDR-B is an intra-vaginal insert that contains the hormone progesterone, ovulation is blocked while the insert is in place.

21. Open Questions: Students are encouraged to: • Prepare sample answers • Work together if they wish • Seek help from me (facilitated by an in-class overview in week 5 and week 11 and a post-mortem of the mid-term in week 9) • Incorporate material beyond lectures and laboratories (identifying key authors or their locations)

22. Advantages: • Less paranoid students: • Predictable situation (no real surprises) • They have time to seek advice • Almost zero risk of academic misconduct • Allows for more complex and wide ranging questions that are problem-based (and give a better evaluation of understanding) • Improved precision and accuracy (vs. assignments)

23. Advantages: (cont’d) • Exam can be written at almost any time – avoids composing alternate exams • I can assign an ‘A’ with more of a clear conscience

24. Disadvantages: • Helps with, but does not totally solve the need for novel questions • Somewhat unfair to introverted students • Not all students respond to the opportunity (depressing) • Time commitment (instructor)

25. Scalability and transferability: Given the 2 courses I teach – I’d suggest a maximum of ~ 100 students (perhaps more, if it is the only course to be graded) Applicable in any course that lends itself to essay-type questions

26. The Sales ‘Pitch’ • If we don’t want academic misconduct, we should prevent it through supervised evaluations • Consider publishing your examination questions in advance (and then expecting ‘better’ answers). Open questions/closed examinations appear to be working for me