1 / 31

Learning from assessment: insights about student learning from programme level evidence

Learning from assessment: insights about student learning from programme level evidence. Dr Tansy Jessop, TESTA Project Leader Launch of the Teaching Centre School of Politics and International Relations University of Nottingham 15 May 2014. TESTA premises.

tuari
Download Presentation

Learning from assessment: insights about student learning from programme level evidence

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning from assessment: insights about student learning from programme level evidence Dr Tansy Jessop, TESTA Project LeaderLaunch of the Teaching CentreSchool of Politics and International Relations University of Nottingham 15 May 2014

  2. TESTA premises Assessment drives what students pay attention to, and defines the actual curriculum (Ramsden 1992). Feedback is significant (Hattie, 2009; Black and Wiliam, 1998) Programmeis central to influencing change.

  3. Thinking about modules modulus (Latin): small measure “interchangeable units” “standardised units” “sections for easy constructions” “a self-contained unit”

  4. How well does IKEA 101 packaging work for Sociology 101? Furniture • Bite-sized • Self-contained • Interchangeable • Quick and instantaneous • Standardised • Comes with written instructions • Consumption Student Learning • Long and complicated • Interconnected • Distinctive • Slow, needs deliberation • Varied, differentiated • Tacit, unfathomable, abstract • Production

  5. What is TESTA?Transforming the Experience of Students through Assessment HEA funded research project (2009-12) Seven programmes in four partner universities Maps programme-wide assessment Engages with Quality Assurance processes Diagnosis – intervention – cure

  6. TESTA ‘Cathedrals Group’ Universities

  7. Canterbury Christchurch Edinburgh Edinburgh Napier Glasgow Greenwich Sheffield Hallam University of West Scotland Lady Irwin College University of Delhi

  8. TESTA “…is a way of thinking about assessment and feedback” Graham Gibbs

  9. Based on assessment principles Time-on-task Challenging and high expectations Students need to understand goals and standards Prompt feedback Detailed, high quality, developmental feedback Dialogic cycles of feedback Deep learning – beyond factual recall

  10. TESTA Research Methods(Drawing on Gibbs and Dunbar-Goddet, 2008,2009) PROGRAMME AUDIT ASSESSMENT EXPERIENCE QUESTIONNAIRE FOCUS GROUPS Programme Team Meeting Case Study

  11. Case Study X: what’s going on? • Mainly full-time lecturers • Plenty of varieties of assessment, no exams • Reasonable amount of formative assessment (14 x) • 33 summative assessments • Masses of written feedback on assignments (15,000 words) • Learning outcomes and criteria clearly specified ….looks like a ‘model’ assessment environment But students: • Don’t put in a lot of effort and distribute their effort across few topics • Don’t think there is a lot of feedback or that it very useful, and don’t make use of it • Don’t think it is at all clear what the goals and standards are • …are unhappy

  12. Case Study Y: what’s going on? • 35 summative assessments • No formative assessment specified in documents • Learning outcomes and criteria wordy and woolly • Marking by global, tacit, professional judgements • Teaching staff mainly part-time and hourly paid ….looks like a problematic assessment environment But students: • Put in a lot of effort and distribute their effort across topics • Have a very clear idea of goals and standards • Are self-regulating and have a good idea of how to close the gap

  13. Two paradigms…

  14. Transmission Model

  15. Social Constructivist model

  16. Focus Group data In pairs/groups, read through quotes from student focus group data on a particular theme. What problems does the data imply? What solutions might a programme develop to address some of these challenges? A3 sheets provided to tease out challenges and solutions.

  17. Student voice data

  18. Theme 1: Formative is a great idea but… If there weren’t loads of other assessments, I’d do it. If there are no actual consequences of not doing it, most students are going to sit in the bar. I would probably work for tasks, but for a lot of people, if it’s not going to count towards your degree, why bother? The lecturers do formative assessment but we don’t get any feedback on it.

  19. Theme 2: Assessment isn’t driving and distributing student effort We could do with more assessments over the course of the year to make sure that people are actually doing stuff. We get too much of this end or half way through the term essay type things. Continual assessments would be so much better. So you could have a great time doing nothing until like a month before Christmas and you’d suddenly panic. I prefer steady deadlines, there’s a gradual move forward, rather than bam!

  20. Theme 3: Feedback is disjointed and modular The feedback is generally focused on the module. It’s difficult because your assignments are so detached from the next one you do for that subject. They don’t relate to each other. Because it’s at the end of the module, it doesn’t feed into our future work. You’ll get really detailed, really commenting feedback from one tutor and the next tutor will just say ‘Well done’.

  21. Theme 4: Students are not clear about goals and standards The criteria are in a formal document so the language is quite complex and I’ve had to read it a good few times to kind of understand what they are saying. Assessment criteria can make you take a really narrow approach. I don’t have any idea of why it got that mark. They read the essay and then they get a general impression, then they pluck a mark from the air. It’s a shot in the dark. We’ve got two tutors – one marks completely differently to the other and it’s pot luck which one you get.

  22. Main findings Too much summative; too little formative Too wide a variety of assessment Lack of time on task Inconsistent marking standards ‘Ticking’ modules off Poor feedback: too little and too slow Lack of oral feedback; lack of dialogue about standards Instrumental reproduction of materials for marks

  23. Summative-formative issues Students and staff can’t do more of both. Reductions in summative – how many is enough? Increase in formative – and make sure it is valued and required. Debunking the myth of two summative per module. Articulating rationale with students, lecturers, senior managers and QA managers.

  24. 1. Examples of ramping up formative The case of the under-performing engineers (Graham, Strathclyde) The case of the cunning (but not litigious) lawyers (Graham, somewhere) The case of the silent seminar (Winchester) The case of the lost accountants (Winchester) The case of the disengaged Media students (Winchester)

  25. 2. Examples of improving ‘time on task’ The case of low effort on Media Studies The case of bunching on the BA Primary

  26. 3. Engaging students in reflection through improving feedback The case of the closed door (Psychology) The case of the one-off in History (Bath Spa) The case of the Sports Psychologist (Winchester) The conversation gambit

  27. 4. Internalising goals and standards The case of the maverick History lecturer (a dove) The case of the highly individualistic creative writing markers

  28. Changes

  29. Impacts Improvements in NSS scores on A&F – from bottom quartile in 2009 to top quartile in 2013 Three programmes with 100% satisfaction ratings post TESTA All TESTA programmes have some movement upwards on A&F scores Programme teams are talking about A&F and pedagogy Periodic review processes are changing for the better.

  30. www.testa.ac.uk

  31. References Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31.Gibbs, G. & Dunbar-Goddet, H. (2009). Characterising programme-level assessment environments that support learning. Assessment & Evaluation in Higher Education. 34,4: 481-489.Hattie, J. (2007) The Power of Feedback. Review of Educational Research. 77(1) 81-112. Jessop, T. and Maleckar, B. (in press). The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher Education. Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88. Jessop, T, McNab, N & Gubby, L. (2012) Mind the gap: An analysis of how quality assurance processes influence programme assessment patterns. Active Learning in Higher Education. 13(3). 143-154. Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517Sadler, D.R. (1989) Formative assessment and the design of instructional systems, Instructional Science, 18, 119-144.

More Related