e-AFFECT: What have we learnt?

1. e-AFFECT: What have we learnt? Dr Anne Jones Centre for Educational Development

2. Approach to project after Cooperrider & Whitney (2005)

4. Participating programmes

5. Lessons about Assessment & Feedback business processes

6. Managing assessment and feedback • Not all Schools have assessment policies • Sometimes there is one for an area within a School • Some are very detailed • Others are very prescriptive • Hard copy versus electronic submission of coursework • Variation in the release of feedback and marks on coursework • Variation in the provision of feedback on exams

7. Lessons about Assessment & Feedback timelines

8. Assessment and feedback timelines Adapted from the ESCAPE project

9. Assessment and feedback timelines

10. Level 1 Level 3

11. Level 2 Level 3

12. Student and staff views

13. Student and staff questionnaires Student questionnaire Staff questionnaire Assessment and feedback strategies Extent to which practice is occurring About themselves Response rate = 47.3% • Study and learning strategies • Extent to which practice is occurring • About themselves • Response rate = 20.4%

14. Student profile

15. Student study and learning strategies

16. Study and learning strategies • Amount and distribution of study effort 6 statements • Assignments and learning 8 statements • Quantity and timing of feedback 6 statements • Quality of feedback 6 statements • What students do with feedback 6 statements • The examination and learning 6 statements Based on Gibbs and Simpson (2004)

17. Analysis Significance p ≤ 0.05

18. Key findings

19. Student and staff perceptions of practice

20. 27 statements based on ATLAB questionnaire (Whitelock and Cross, 2011) • Fully occurring – not in a position to say • Assessment criteria are written and used • All marking criteria are shared with students in advance and feedback refers to them • Feedback is intended to acknowledge, consolidate and promote student learning • Students get feedback which corrects errors and supplies further information • Students have opportunities for cooperative and collaborative assessment

21. Analysis Significance p ≤ 0.05

22. Students

23. Students and staff

24. Lessons from the methodology

25. Appreciative Inquiry approach after Cooperrider & Whitney (2005)

26. Engendering dialogue – time and space

27. Bespoke technology workshops

28. Lessons from activities

30. Civil Engineering 1 Opportunity to act on feedback Help clarify good performance Development of self-assessment and reflection

31. Civil Engineering 2 Opportunity to act on feedback Help clarify good performance Development of self-assessment and reflection Encourage time and effort on challenging tasks

32. Phonetics ‘Fast feedback’s really important in phonetics. You need to establish good habits right from the start.’ also include video and audio stimuli, to target particular phonetic features. Opportunity to act on feedback Help clarify good performance Development of self-assessment and reflection Encourage time and effort on challenging tasks Encourage interaction and dialogue around learning

33. Phase 2 activities • Some of the activities are represented in the Marketplace • Other activities include: • Marking and feedback workshops with students • Students indicating how they have used feedback • The use of Vimeo for uploading films • E-submission/marking/feedback • QuestionMark Perception for formative and summative activities • Peer review using PeerMark

34. Phase 3 – planned activities • Development of feedback comments bank • Student generated MCQs using PeerWise • QuestionMark Perception for online objective testing • Vodcasts • Screen capture feedback on draft work • Clarification of assessment criteria and standards – guides and workshops

35. The Marketplace 2014

36. – Collaborate project A Cross-Institutional Approach to Assessment Redesign: Embedding Work-Integrated Assessments within the Curriculum Dimensions of a Work- Integrated Assessment Designing Work- Integrated Assessments Using a Model for Work-Integrated Assessment: Six Dimensions Aligning Technologies to the Model: ‘Top Trumps’

37. School of History and Anthropology: Student generated MCQs ‘The lecturer got us to sign up to a site called 'Peerwise' which involves students creating multiple choice questions and the other students are able to answer them. I thought this was a brilliant idea as it’s great for revision.’ Module evaluation, Deviance History, QUB, 2013

38. Centre for Medical Education A strategy to help students to actively engage with feedback provided: sharing experience from the undergraduate medical student selected component (SSC) programme Examples of feedback justify reasons for and against this …. make greater use of relevant cases and sources to support discussion here…. background should be more focused on… Student responses I constructed a more balanced argument… I incorporated more cases to illustrate.. I cut down… to allow more room for…. I put more emphasis on this point….

39. e-AFFECT ...Shared and jointly owned standards leading to consistency of feedback C described his practice in giving feedback when students previously had had to provide a reflection on the strengths and weaknesses of their work when they requested feedback.  He read their reflective statements and incorporated these comments into the feedback he had already written.  A dialogue was thus developed.

40. QUMS – Queens Online assignment tool

41. Environmental Planning - Jing Implementation As a method of delivering formative feedback on visual work to enhance skill development To enhance skill development with Jing Screencast Tutorials

42. Environmental Planning – Acrobat Pro Implementation Annotation over drawings is a standard way of giving feedback for design based modules. Students can access this feedback through their tablets, computers or smart phones. The module consisted of three stages and Acrobat Pro was used to give feedback for the first of the three submissions in the module.

43. Environmental Planning - VoiceThread Implementation For this module we made four 15 minute videos on a range of planning subjects and asked specific questions that students need to answer. All students can see all the comments that other students and the tutors make.

44. Drama – VoiceThread for student engagement • How did I use it? • PowerPoint lectures were put into VoiceThread with my commentary as a voiceover. • Students were asked to watch the lectures before the class. • Students were required to add substantive comments or questions to the VoiceThread. • 15% of the marks were allocated for this activity

45. Centre for Medical Education – Virtual Patients Turning flat word-processed patient cases into interactive learning objects Storyboard

46. Computer Science - Offline objective marking 105 Level 3 students take a series of assessed practicals and a final assessment in QMP. As the students work through the practicals they complete a worksheet in QMP. The teaching team used Excel to score the responses

47. References • Cooperrider, D & Whitney, D 2005Appreciative Inquiry: a positive revolution in changeBerrett-Koehler Publishers, Inc. San Francisco • Gibbs, G & Simpson, C (2004) Conditions under which assessment supports learning, Learning and Teaching in Higher Education 1, 3-31 • Nicol, D (2009) Transforming assessment and feedback: enhancing integration and empowerment in the first year, The Quality Assurance Agency for Higher Education, Mansfield • Whitelock, D. & Cross, S. (2011). Assessment Benchmarking: Accumulating and accelerating institutional know-how for best practice. International Journal of e-Assessment (IJEA), Vol 1. No. 1 http://journals.sfu.ca/ijea/index.php/journal/article/view/18

48. Resources • e-AFFECT Jisc Design Studio http://jiscdesignstudio.pbworks.com/w/page/50671059/e-AFFECT%20Project • e-AFFEECT Resources JiscDesign Studio http://jiscdesignstudio.pbworks.com/w/page/68656954/e-AFFECT%20Resources • e-AFFECT blog http://blogs.qub.ac.uk/e-affect/ • REAP project http://www.reap.ac.uk/ • Viewpoints Project Jisc Design Studio http://jiscdesignstudio.pbworks.com/w/page/29227748/Viewpoints%20project and http://wiki.ulster.ac.uk/display/VPR/Home