Mechanised Assessments that Promote Deep Learning

1. Mechanised Assessments that Promote Deep Learning …and prevent plagiarism!! Gareth Denyer & Dale Hancock, University of Sydney

2. Multiple Choice • Rapid, convenient • Excellent for large classes • BUT…. do they encourage surface learning? • Questions often designed to test unambiguous facts • Student PERCEPTION • adopt a ‘rote’ approach • Names, facts, numbers • Attempts at deeper, conceptual based possible… • BUT… often lead to ambiguous questions • At worst, questions favour less able students because better students confused! • VERY SKILFUL JOB!!!

3. Marking MCQs • No credit for near misses or process • No point in leaving answers ‘blank’ • So you never know ‘why’ • Guessing can give 20% • Negative marking? • Intimidates students • Forces meticulous non-ambiguity • Further driving surface learning! • Post-Exam revision of mark scheme difficult • Unlike SAQs • where mark scheme can be dynamically changed • Questions often ‘discarded’ • Disadvantages good students

4. Graded Alternative NB. This is a SURFACE example!! What is the capital of Australia? AMelbourne BSydney CGlasgow DCanberra EAuckland • The more complex the question, the more important partial marks become. 0.5 0.5 -1 1 0

5. XL Based Graded Solution Typical student answer grid Marks Key Sheet

6. Calculations • Blah, blah… What is the approximate ε450nm for the coloured compound formed in the assay? • 0.199 mM-1 cm-1 • 250 mM-1 cm-1 • 4 mM-1 cm-1 • 2 mM-1 cm-1 • 19.6 mM-1 cm-1 Each option reveals a particular mistake Especially good with a multi-step calculation Can reward process

7. Opens up Deeper Styles • The “What if?” Cause → Effect • What would be the consequences of inhibition of lipolysis during the first few days of starvation? • Shades of Grey • You decide that you need 50 mU of citrate synthase (CS) in the cuvette. What is the MOST PRACTICAL way of doing this?

8. Advantages • Makes marking flexible • Like an SAQ! • Post-hoc changes possible • Provides an alternative way of scaling • Staff less stressed • More extrapolative questions • Less attention to rigorous ‘fact’ • Less concern about ‘getting it perfect’ • No conflict with colleagues over ambiguous questions • Feedback better • Each option “useful”

9. Plagiarism! • Much attention given to detecting cheating in assignments • Puts you into a conflict situation • But exams make up most of the marks! • Cheating is easy in MCQ exams • Hand signals • Pattern recognition • Several cases • A healthy literature!

10. Islands of corruption adapted from data in: Harpp & Hogan (1993) Crime in the classroom: Detection and prevention of cheating on multiple-choice exams. J. Chem. Educ.70, 306

11. Wise Words Universities obviously want to stamp out cheating But they also do not want any students to be falsely accused 1 in 653,000 Statistical evidence is generally not enough. Pursuing a case is traumatic for all those involvedEspecially if an unwilling cheatee!

12. The Versions Solution Create FOUR versions of the same paper Questions in same order, options just rotated Layout so that each version is isolated from its clones Very easy in Word A database solution can generate even more versions.

14. Implementation • Sweet Justice!! • Appropriate recompense without confrontation • Lots of ‘evidence’ • Sometimes even ‘check’ questions copied • Easy • Just Word and Excel • Rotations take about 10 min per 100 MCQs

15. Take Home Messages • Graded MCQs give flexibility • Different versions disincentivise cheating • All easily implemented • Word and Excel • We are happy to provide templates and instruction Have a go! Easy and Liberating!