Sarah Chesney – ESC & Caroline Marcangelo

1. Digitally-enhanced Patchwork Text Assessment : a JISC LTI project Sarah Chesney – ESC & Caroline Marcangelo JISC DEPTA JULY 2011

2. This presentation: • Clarify terminology - PTA • Harnessing technologies to enhance learning through assessment • Using work/practice for learning in our Pg Cert HE & CPD programmes • the JISC-funded DePTA project JISC DEPTA JULY 2011

3. Clarifying terminology: Patchwork Text Assessment Individual students carry out regular short writing tasks known as “patches” throughout the ‘learning module’ Each patch shared is with a small peer group for discussion and formative feedback Tutor might feedback on a sample of these ‘patches’ Typically the patches embody different genres of writing, and different elements of a broader topic – number varies Concluding and summative activity is to produce a reflective, synoptic, ‘stitching together’ of the patches Original work undertaken by Winter et al in early 2000s JISC DEPTA JULY 2011

4. What advantages are there to PTA? Continuous workflow in module Opportunity to try different forms of presenting work Cummulative assessment enabling rapid formative feedback Opportunity to see peer responses to the task – aid to self-assessment and understanding of learning outcomes Integration of learning through reflection Fostering self-management of learning through meta-cognitive self-reflection on the learning journey JISC DEPTA JULY 2011

5. Technology to enhance learning • The PTA process can be enhanced – even ‘enabled’ – through technology that has developed significantly since the initial work undertaken by Winter et al (2003) • Facilitates more opportunity for the style and genre of patches • Can include patches that are non-text • Enables easier sharing and feedback with peers and tutors – and records different versions JISC DEPTA JULY 2011

6. Background to project • UoC PgC for academic staff new to HE • e-portfolios* as a personal learning system • PTA to enable recognition of the range of workplace activities where learning occurs • Peer- collaboration and formative feedback • Modelling and developing good practice • CPD scheme uses e-portfolios for UKPSF recognition process • * 3 year JISC-funded project – ‘Flourish’ www.flourishproject.org JISC DEPTA JULY 2011

7. Patchwork Text Assessment Predominantly viewed as a positive method of assessment – flexible, reflective, covered a broad range of interests, logistically possible, demonstrated metacognition A very small number viewed doing small pieces of work more difficult than one straight essay Acknowledgement that frequent sharing and feedback encouraged deeper self-assessment and critical thinking Chesney, S. & Marcangelo, M. 2010 'There was a lot of learning going on' Using an e-portfolio to support learning activities in a professional course for new HE lecturers Computers & Education 54 (2010), pp. 701-708 DEPTA JULY 2011

8. Direction of travel ….. Later iteration uses “patchwork assessment” – encouraging the use of non-text patches and reflexive commentary Participants using audio, video, verbal reflective commentary, and conceptual mapping software Using e-portfolios with staff continues to ‘normalise’ and there is evidence that they cascade their use with students DEPTA JULY 2011

9. Aims of DePTA … This one-year project was funded through the JISC Learning and Teaching Innovation Grant (5) programme with the aim of evaluating the use and effectiveness of digitally supported patchwork text assessments in traditional subject disciplines DePTA project webpage: http://dpta.wordpress.com/ JISC DEPTA JULY 2011

10. JISC DePTA project • 5 participating universities in different disciplines • Psychology (Wolverhampton, Bedfordshire) • Law (Law) • Photography (Cumbria) • Physics (Liverpool) • Archaeology (Liverpool) • Final year modules giving the students an opportunity to produce artefacts that showcase graduate capabilities and employability skills • using ‘regular’ technology for that HEI/discipline DEPTA JULY 2011 /May'11

11. Project evaluation frame: Evaluation focused on a range of areas including: • Practicability in terms of institutional technological resources • Capacity to enhance and evidence student learning at a high-stakes level • Sustainability in relation to staff workload JISC DEPTA JULY 2011

12. Project evaluation: contexts JISC DEPTA JULY 2011 12

13. generally existing technologies were used often chosen on pragmatic grounds and that students and staff were familiar with them. The timeframe of a single year presented challenges for participants to make modifications to working practice that required university approval e.g. assessment regulations and module protocols. In some cases this had an impact upon the extent of student participation, as the amended assessment was offered as an optional activity. Project evaluation: JISC DEPTA JULY 2011 13

14. Peer to peer sharing: sharing work between students has been part of some but not all partner projects. Where peer feedback was encouraged it still presented considerable challenges due to lack of familiarity with the collaborative nature of the tasks, particularly in the online environment. When in a more familiar setting such as face-2-face seminar peer discussion including challenge and feedback was not regarded as problematic. there appeared to be a discipline-related culture related to valuing individual achievement rather than co-operative development. Not all allowed online comments; some retained face-2-face meetings for such feedback/discussion, and one plans to move in this direction next year. Formative tutor feedback: Unsurprisingly students appreciated receiving tutor feedback Project evaluation: Feedback: JISC DEPTA JULY 2011 14

15. Where students were geographically distributed with limited opportunities to meet (e.g. on work placement), the technologies were seen as valuable in maintaining connection/momentum. Whilst most of the patches were text based, other approaches including video and concept mapping were also employed Project evaluation: JISC DEPTA JULY 2011 15

16. Whilst one of the project aims anticipated the significance of subject benchmark and employability statements, there was evidence that this was not always realised in practice. In most cases the emphasis was upon the discipline rather than generic institutional graduate attribute statements Project evaluation: JISC DEPTA JULY 2011 16

17. the majority of tutors see considerable potential in extending/expanding the DePTA approach for a variety of reasons including being able engage and motivate learners for the full length of a module, developing a range of discipline specific skills Project evaluation: JISC DEPTA JULY 2011 17

18. Students are more confident in their learning through requirement to express it regularly and consider others’ development to the level of commenting on it. The technology provides a constant record so they can view there development over a period, where their progress is evident, thus further improving confidence. (Liverpool Physics case study) Project evaluation: JISC DEPTA JULY 2011 18

19. Results: To be discussed