Use of audio feedback for summative purposes

1. Use of audio feedback for summative purposes Kathryn McFarlane

2. Aims Explore a possible rationale for using audio feedback Identify advantages and disadvantages of this approach Explore the practicalities of producing audio feedback Consider potential application strategies in own environment

3. Outline Rationale for using audio feedback Evaluative project and outcomes The techy bit Application to practice

4. Rationale for using audio feedback

5. Activity • In groups of four, write a list of possible feedback methods on post its • Now organise these on the “Race grid” 13 September 2014

6. High High Low Cost / time Impact on student learning Low

7. The feedback dilemma.... Feedback (and feed forward) identified as important to learning (e.g. Taras, 2002) Student dissatisfaction with assessment feedback (HEFCE, 2009) Many students do not collect feedback Written feedback – lowest impact on student learning and not cost effective? (Race, 2008)

8. Why audio feedback? “Narrow bandwidth” of text based communication - reduces the potential to make judgements about meanings of words (Goodyear, 2001) Greater richness of detail, guidance for academic and professional enhancement, and messages to engage students in thinking (Merry and Orsmond, 2008)

9. Why audio feedback? Students suggest: Clearer Includes strategies to solve problems, not just problems More personalised Engaged more deeply with it – e.g. using it to prepare for future assignments Tone of voice helpful for highlighting importance of various aspects Accessibility, e.g. can listen while engaging in other activities (Various, e.g. Ribchester et al, 2008; France and Wheeler, 2007; Rotherham, 2008)

10. Why audio feedback? Inclusion Contemporary students may be less comfortable in processing written information (Merry and Orsmond, 2008) Could be inclusive to “Aural” learners (Fleming and Baume) Could be inclusive to diverse students, e.g. international, dyslexic (Rotherham, 2008) Potential for inclusive feedback – offering a choice (Waterfield and West, 2008)

11. Why summative feedback? Disparity between staff and students’ perceptions: students do use the guidance, and view the mark and supporting feedback as interdependent. (O’Brien and Sparshatt 2007) Students initially focus on grade, but use feedback to improve, and some revisited previous work when preparing subsequent assignments (Carless 2006, p.225)

12. Evaluative project and outcomes

13. Project background Postgraduate Certificate in Higher and Professional Education – for teaching / learning support staff in HE First module (for most): Supporting Learning – includes a focus on the factors which impact on student learning, and student diversity Participants are mature work-based learners (mainly “digital immigrants”?)

14. Evaluation questions • What is the impact of using the medium of audio on the messages conveyed by summative feedback? • To what extent does audio feedback contribute to “feed forward”? • What is the impact of preparing audio files on tutor workload?

15. Evaluation Action Research approach – collaborating with the participants Methods: Expert walk through Initial questionnaire pre-feedback On line “summative” questionnaire Focus groups PebblePad e-journal More details soon to be published in Innovative Practice in HE Journal

16. Formative evaluation results • Guidance on project organisation from expert walkthrough, e.g. reducing file sizes, limiting feedback time • Guidance on accessibility from participants (e.g. using PebblePad rather than Blackboard) • Audio more explanatory, provided more examples, in everyday language, linked to learning from module

17. Summative evaluation results…. Problems with verbal delivery – e.g. muffled Grade point at the end – would have preferred it at the beginning Access and technical problems Not having any text Lack of time to engage with the feedback

18. Summative evaluation results…. Novel format for receiving feedback More detailed comments Strategies gained for future assignments - feed forward Impact on professional practice Personalisation Explaining academic language Improvements needed were clear Voice giving added meaning

19. Other themes Inclusion – language; learning style Practicalities, e.g. open plan environment Emotional response Digital native v. digital immigrant

20. Tutors’ views More enjoyable Feel more “exposed” – at first Takes equal time – give more feedback in time available Technicalities (e.g. second marking, sharing feedback) more time consuming – at first Need to learn to keep feedback down to a reasonable length Practicalities, e.g. quiet place to mark

21. Future strategies Consider where to put the mark Encourage participants to listen with the assignment in front of them Keep participants informed, e.g. feedback release dates, when mark will be communicated Use with other modules – formative and summative Offer choice – text or audio (inclusive feedback) Research possible correlation between response to feedback and grade attained

22. The techy bit......

23. Hand-held digital recording device Demonstration One I prepared earlier....... Example feedback

24. Audacity Available via http://audacity.sourceforge.net/ Demonstration

25. Other useful tools Any Audio Converter – to reduce file sizes / convert to MP3 – available via http://www.any-audio-converter.com/ Wavepad Sound Editor – to edit files – available via http://www.nch.com.au/wavepad/index.html

26. Application to practice

27. Practical guidance Consider using a combination of summative individual feedback and overview commentary Limit the length of recordings to avoid the danger of providing too much feedback Need for increased sensitivity in providing critical feedback by means of this medium (Ribchester et al, 2008; France and Wheeler, 2007)

28. More practical guidance Which device? Second marking External examiner Storage space Quiet space to mark Dissemination to students Feedback from students

29. Getting started.... Pilot with a small group? Use for formative feedback? Use in tutorials?

30. Discussion • Working in pairs / small groups: • Discuss the potential application of this approach to your practice • What strategies could work in your environment? E.g. Whole group or individualised audio feedback? Formative or summative feedback? • What would you need to do, in order to prepare students, staff and others? (e.g. External Examiner) • Optional extra – have a go at producing some audio feedback

31. Finally..... Summary Comments / questions Thank you!

32. References • Fleming, N. and Baume, D. (2006) Learning Styles Again: VARKing up the right tree! Educational Developments, SEDA. Issue 7.4., p.4-7. • France, D. and Wheeler, A. (2007) Reflections on using podcasting for student feedback. Planet No. 18. [Online] Available from http://www.gees.ac.uk/planet/p18/df2.pdf [accessed 26.1.09] • Goodyear, P. (2001), Effective networked learning in higher education: notes and guidelines: Networked Learning in Higher Education Project (JCALT) [online]. Available from: http://www.ioe.ac.uk/ccs/dowling/cmc2004/papers/goodyear-guidelines_final.pdf [accessed 3.1.09]. • HEFCE (2009) National Student Survey [online]. Available from: http://www.hefce.ac.uk/Learning/nss/data/2009/ [accessed 7.3.10]. • Merry, S. and Orsmond, P. (2008), Students’ Attitudes to and Usage of Academic Feedback Provided Via Audio Files. [Online]. Available from http://www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.aspx [accessed 26.1.09]. • O’Brien, R. and Sparshatt, L (2007), Mind the Gap! Staff Perceptions of Students’ Perceptions of Assessment Feedback [online]. In: Higher Education Academy Annual Conference, Harrogate, 3.7.07 – 5.7.07. Available from: http://www.heacademy.ac.uk/resources/detail/mind_the_gap [accessed 9.3.09]. • Race, P. (2008). Aligning assessment to current trends in HE. Staffordshire University, 23.5.08. • Ribchester, C., France, D. and Wakefield, K. ‘It was just like a personal tutorial’: Using podcasts to provide assessment feedback. In HE Academy Annual Conference 2008, Harrogate, 1.7.08 – 3.7.08. [Online]. Available from: http://www.heacademy.ac.uk/assets/York/documents/events/conference/2008/Chris_Ribchester.doc [accessed 26.1.09]. • Rotherham, R. (2008a), Sounds Good: Quicker, better assessment using audio feedback. [Online]. Available from: http://sites.google.com/site/soundsgooduk/Home [accessed 14.3.09]. • Taras, M. (2002), Using Assessment for Learning and Learning from Assessment. Assessment and Evaluation in Higher Education. 27(6) p.501-510. • Waterfield, J. and West, B. (2005) Staff-Student Partnership for Assessment Change and Evaluation (SPACE) Project. [Online]. Available from http://www.plymouth.ac.uk/pages/view.asp?page=10494 [accessed 26.1.09].