Engaging students with formative assessment (and feedback)

1. Engaging students with formative assessment (and feedback) Margaret Price 10 December 2013

2. This session aims to : • To examine the factors that influence levels of student engagement with formative assessment and feedback • To critically evaluate approaches to increasing levels of engagement with formative assessment • Emphasize a programme approach and link with session programme-focused assessment

3. Structure • Engagement with assessment • What causes non engagement • How can these be addressed • Engagement with feedback • Research findings • Relevance of the whole learning experience • Resources and feedback • Approaches to supporting engagement with feedback

4. Sources of problem Students How to help the students engage ( overload? partying? boredom?) Check assumptions. Setting expectations, modelling and influencing culture (tolerance of ‘laziness’) Assessment design Audit - relevance, how interesting, have students got a chance of developing expertise Staff expertise Downsides of engagement methods (surface learning) Level of assessment literacy e.g peer assessment

5. Why don’t students engage with formative assessment • Each group generate as many reasons as you can in 5 minutes. • Which do you think are the 3 most impactful reasons • Choose the reasons that students would give (assuming an honest reflective answer)? • What are the reasons that most staff attribute to the problem of non engagement? (the culture of the organisation)?

6. Ways to get students to engage? • Each group list as many ways as you can in 5 minutes • Which are the 3 most commonly used methods? • Which are the 3 most effective methods? • Which are the 3 that would meet most resistance?

7. Perspectives on engagement with assessment task • Compulsion • Incentivising • Nudging • Developing self motivation through assessment literacy • Self motivated • Intrinsic engagement

8. Prevention is better than cure Use the cards to create effective ‘formulae’ that lead to student engagement. Cards available: Engagement problem Actions to take Alert cards (red) highlight further issues to be addressed.

9. Engagement with Feedback Don’t start here – part of the assessment and learning package.

10. Plan • Research findings • Relevance of the whole learning experience • Resources and feedback • Approaches to supporting engagement with feedback

11. Look beyond feedback product • Clarity of purpose • Learning effectiveness (and student engagement) is strongly influenced by opportunity to apply feedback to future performance This relies on: ability to understand feedback (legibility and interpretation) expectations of the utility of feedback perception of self efficacy • The relational dimension of feedback is key to student engagement • Dialogue supports understanding and engagement) • (Price et al 2010)

12. Conceptual shifts Self and peer assessment need to be seen as essential graduate attributes (i.e. learning outcomes themselves, rather than simply processes) Feedback needs to be seen as a dialogue (rather than a monologue)

13. Feedback moments • Where there is a clear need to apply feedback • Pre assessment • Reflection points Identify them within each programme

14. Feedback moments? Learning development Phase 1 Phase 2 Phase 3 Radical movement involving zones of discomfort, ‘threshold concepts and troublesome knowledge’ (Meyer and Land, 2006) Changes in ‘epistemology and knowledge structures’ (Basil Bernstein in Moore et al, 2006)

15. Sources of feedback Staff: Traditional – personal, generic. Feedback on Self Assessment e.g. 5mins oral feedback for 140 students Personal tutor consultation • Peers: Peer review and peer assessment • Peer assisted learning Self: Require self assessment

16. Peer marking using model answers (Forbes & Spence, 1991) • Scenario: • Engineering students had weekly maths problem sheets marked and problem classes • Increased student numbers meant marking impossible and problem classes big enough to hide in • Students stopped doing problems • Exam marks declined (Average 55%>45%) • Solution: • Course requirement to complete 50 problem sheets • Peer assessed at six lecture sessions but marks do not count • Exams and teaching unchanged • Outcome: Exam marks increased (Av. 45%>80%)

17. Peer feedback - Geography (Rust, 2001) Scenario • Geography students did two essays but no apparent improvement from one to the other despite lots of tutor time writing feedback • Increased student numbers made tutor workload impossible Solution: • Only one essay but first draft required part way through course • Students read and give each other feedback on their draft essays • Students rewrite the essay in the light of the experience • In addition to the final draft, students also submit a summary of how the 2nd draft has been altered from the1st in the light of the feedback Outcome: Much better essays

18. Peer feedback - Computing (Zeller, 2000*) The Praktomat system allows students to read, review, and assess each other’s programs in order to improve quality and style. After a successful submission, the student can retrieve and review a program of some fellow student selected by Praktomat. After the review is complete, the student may obtain reviews and re-submit improved versions of his program. The reviewing process is independent of grading; the risk of plagiarism is narrowed by personalized assignments and automatic testing of submitted programs. In a survey, more than two thirds of the students affirmed that reading each other’s programs improved their program quality; this is also confirmed by statistical data. An evaluation shows that program readability improved significantly for students that had written or received reviews. [*Available at: http://www.infosun.fim.unipassau.de/st/papers/iticse2000/iticse2000.pdf]

19. Resources and effectiveness • Review resource allocations in total (OU resourcing model) • Feedback methods: • Oral(F2F)/audio rather than written (see the Sounds Good website at: http://sites.google.com/site/soundsgooduk/) • Exemplars/model answers including modelling improvement following feedback • Generic (including ‘quick and dirty’), feedback on a draft, MCQs & quizzes, etc. • Feedback workshop • Require students to demonstrate how they have used feedback in subsequent work. • Target resources where most needed in programme • Use self and peer methods more frequently • Target feedback moments

20. Figure 1: Peer-review as a method of encouraging students to discuss and compare their understanding of assessment criteria

21. Figure 2: the use of 'exemplars' as amechanism for encouraging dialogue about assessment criteria

22. Figure 3: Generic feedback and self critique

23. Activity In groups: Anticipate arguments or factors that will militate against change – how will you counter them Begin to develop a checklists for programme teams to use in the process of developing assessment strategies at programme and module level.