1 / 12

OU Workshop : 11th June 2012

Self-assessment: strategies and software to stimulate learning. Self-assessment with Certainty-Based Marking ( CBM ) Tony Gardner- Medwin Physiology ( NPP ), UCL www.ucl.ac.uk /~ ucgbarg. OU Workshop : 11th June 2012. knowledge uncertainty don't know misconception delusion.

clint
Download Presentation

OU Workshop : 11th June 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Self-assessment: strategies and software to stimulate learning Self-assessment with Certainty-Based Marking (CBM)Tony Gardner-MedwinPhysiology (NPP), UCLwww.ucl.ac.uk/~ucgbarg OU Workshop : 11th June 2012

  2. knowledge • uncertainty • don't know • misconception • delusion Decreasing certainty about what is true. Increasing certainty about something false. Increasing "ignorance" Knowledge crucially depends on certainty • A lucky guess is not knowledge • A firm misconception is far worse than acknowledged ignorance • So why do we usually mark students as if these things weren’t true? Knowledge = justified true belief ? ?

  3. Certainty-Based Marking (CBM) <67 % mid >80% • CBM rewards thinking: • identification of uncertainties • or of justifications • Highlights misconceptions • negative marks hurt! • Engages students more • Enhances reliability & validity

  4. Theoretical basis of CBM scoring (T/F Qs) Ignorance (nescience) = -log( P assigned to the correct answer) (Kullback-Leibler divergence)

  5. Multiple Choice (SBA) Qs NB Single-best-answer Qs can fail to identify even very substantial ignorance

  6. Mathematics – where I started using self-tests • Maths / numeracy for 1st year medics • Very mixed abilities • Many don’t expect to benefit from formal maths instruction • Doctors must know what they can / can’t do reliably • CBM approach with unlimited self-test attempts: • Doesn’t offend able students • Allows students to find and extend areas of competence • Motivates learning without humiliation or incentives to cheat

  7. What everyone needs to know about mathematics Danger – I fall into traps. Must always seek help! Stuff I just don’t understand ! Fine – I know what I’m doing here ! OK – but check carefully ! Adapted from: Drive time isochrones around airports in northern Finland. Creative Commons: http://en.wikipedia.org/wiki/File:Drive_time_isochrones_airports_northern_Finland.png

  8. Pattern of initial and eventual success – med maths www.ucl.ac.uk/lapt?medmath No. of students (total 303) No. of repeated attempts at each of 3 maths exercises

  9. LAPT – features built specifically for Self-Assessment London Agreed Protocol for Teaching (and Testing Of Physiology) • Certainty-Based Marking • Instant Feedback for each Q (local software, not server interaction) • Students opt whether or not to answer individual Qs & Sections • Marking normally expressed relative to just those Qs selected • Scores shown with & without CBM to assess reliability judgement • Breakdown showing % correct at each certainty level (C=1,2,3) • Students can make & see comments in context & wiki-edit quizzes • Entire quizzes editable as a single text file for staff convenience • Specific tools for maths & medicine (syntax checks, units, EMQs)

  10. How well do students discriminate reliability ?

  11. Good practice for good self-tests – My own views! Write a stimulating / didactic sequence of Qs mix easy & difficult Qs : for engagement, reward, realism include classic misconceptions – students learn through mistakes make chains of Qs : lead through the logic of a topic Explanations should widen the immediate issue to other contexts prompt the bringing together of different kinds of knowledge Clear navigation – students should be choosing what to do Allow use of study materials – tests shouldn’t be time-limited Encourage anonymous comments & dialogue Encourage working in twos or threes Don’t bother with adaptive tests (conditional jumps to specific Qs) the students should decide & choose what they need Never comment on a student’s mark in a self-assessment !

  12. www.ucl.ac.uk/lapt www.tmedwin.net/cbm/moodle

More Related