1 / 12

Carrots, Sticks, and Engagement and Assessment Strategies (Case Study)

This case study explores the use of engagement and assessment strategies in improving attendance and academic performance among first-year mathematics students. The study examines the effectiveness of using formative EVS (Electronic Voting System) as a carrot for engagement, as well as the impact of a stick approach, in the form of warning emails and potential withdrawal, on attendance. The study ultimately concludes that higher attendance leads to effective learning outcomes for students.

donaldlewis
Download Presentation

Carrots, Sticks, and Engagement and Assessment Strategies (Case Study)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Carrots, sticks, and engagement and assessment strategies (case study)Sean RyanSchool of Physics, Astronomy and Mathematics Mathematics culture • Numerate, logical & methodical • rich in concepts and their application Case module • 1st year, Sem. A. Module taken by all (~100) 1st year maths students: “Financial and Actuarial Mathematics” • All materials on StudyNet– so some students think they can “go solo”

  2. The problem: persistent high failure rates Impervious to style of teaching: • change of lecturer: DTB = 36%, SGR = 37% s.e.≈4%no significant difference • labs/no labs: labs = 37%, none = 37% s.e.≈3%(use class time “better”; move “lab” work to assignments)no significant difference; labs not a motivator

  3. The problem: persistent high failure rates Attendance important in PAM • Clear correlations between attendance and grade:missing 1 class [e.g. out of 24] = lower grade by ~2-3% • Students with lower UCAS scores attend fewer classes • → PAM retention strategy focuses on achieving high attendance and class contact

  4. Building engagement, 2010/11 • 2005-09 = 37% (s.e.=2%), 2010 = 22%, • ? Regular low stakes marked homework w/feedback? – exhausting! • ? Formative EVS? • ? Role of improved UCAS score?

  5. Building engagement, 2010/11 Formative EVS: a carrot for attendance & engagement? Used for short reality checks during class: for student: question own understanding; for lecturer: discover what students have understood Also: promotes discussion that overcomes misunderstandings; helps avoid monotony ? creates interactive element presumed useful; builds engagement ? Look for evidence of attendance ...(mild proxy for engagement)

  6. Was formative EVS a carrot for engagement? Attendance up ~10% during 2010/11 ... ... but was EVS usage responsible? (half?) ... or higher UCAS grades? (half?) ↑30 UCAS pts = ↑1-2 lectures/24 = ↑4-8% attendance

  7. Assessment (and UCAS) changes in 2011/12 Formative EVS used for daily feedback and reality checking; CW = 80% three tests, 20% one assignment (all medium stakes) Look (again) for evidence of attendance ...

  8. Was formative EVS a carrot for engagement? By week 4, attendance drifting down, despite higher UCAS points and EVS use (from wk 3) ...what to do?

  9. Arresting poor attendance: a stick? Mail to selected students after third week of teaching: … you have already missed several lectures in the first three weeks. … you are likely to be withdrawn from the programme. ... Mail to selected students after fourth week of teaching: … you have missed the majority of your classes … I have now begun the process of formally withdrawing you from your studies … and will record you as having withdrawn in our forthcoming Semester A report to the government. … [This is] your penultimate opportunity to rescue your status as a student of the University… you need to take two steps to avert your withdrawal …… establish yourself as actively studying … … make an appointment to see me …

  10. Arresting poor attendance: a stick? Probably, word spread. By week 5, attendance rose 20% above previous year ... and stayed higher all semester – with no more emails.

  11. Was higher attendance helpful to students? 20% ↑ in attendance implies 5 extra classes, or ~10% ↑ in grade. 2010/11 average grade = 48%; 2011/12 average = 55% Still see similar grade-attendance dependence (good), and same y-axis intercept implies assessment not easier, but now fewer students in poor-attendance tail, so fail-rate reduced.

  12. Outcome in 2011/12 Conclusions: EVScarrot: good for feedback, but only limited evidence it improved attendance (↑5%?) or achievement HoS stick: produced sustained change in attendance? (↑UCAS+EVS+HoSstick) → ↑attendance → effective learning Attendance translates into learning for PAM students(cf. “mature” students at OU – more self-disciplined?)

More Related