Assessing research-led learning

1. Assessing research-led learning Peter Kahn Educational Development

2. Outline • Introduction • Challenges for assessing research-led learning • Responses: Principles • Responses: Examples • Pair discussions on specific assessment tasks, criteria and feedback • Conclusions

3. Introduction • The university is looking to develop its use of research-led learning and teaching.  • QAA Institutional Audit (March 2009) • ‘give serious consideration to developing and supporting models of integration between research and teaching across the full range of its curricula’ • ... plus plenty of potential gains. • We focus in particular on learning and teaching that is based on engaging students as active researchers.

4. Challenges • Assessment often proves a substantive stumbling-block for research-led learning, and can mitigate against gains. • Research-led learning typically results in students achieving, creating or experiencing a wide range of: • learning outcomes, products arising from learning, processes as part of their learning, contexts in which learning occurs, social structures underpinning learning.

6. Such variety poses substantive challenges for assessment tasks, criteria and feedback: • Demands on staff and student, as support is required to follow an unfamiliar research process, adjust to a new context, and master an ability; student by student. • Ensuring fairness across a cohort of students given variation in the tasks/contexts/level of support involved. • Shift often required from traditional assessment methods, but the culture of assessment is conservative.

7. Responses: Principles • Contingency; efficiency; alignment; fairness. • Assessment is formative to the extent that evidence about student performance is elicited, interpreted, and used by teachers, learners or their peers, to make decisions about the next steps in research processes ... (adapted from Black and William, 2009)

8. Moments of contingency arise naturally within research-led learning: • Unpredictable and uncertain aspects of work within authentic contexts. • Research processes unfold over time, with scope to shift direction in light of evidence or to plug gaps in understanding. • Opportunity for discussion as an integral aspect of the learning. • Contingency ties in to one of the most substantive benefits of research-led learning, namely student motivation.

10. Efficiency • Be realistic in the extent of the research involved or recognise the demands via credit size • Students conduct research in groups or pairs • Use of coursework requirements • a condition that a student must meet to pass a module or assignment • ... and combine this with • sampling of student work and the involvement of others in commenting on student work (e.g. self, peers, GTAs, clients, professionals, technicians ...)

12. Alignment • Learning outcomes • Appreciate uncertainty and limits of knowledge, manage own learning, initiate projects, initiative and personal responsibility, decision making in unpredictable circumstances, complex contexts, autonomously, identify a range of solutions, ... • Consider how assessment addresses such capabilities: • favours an increased use of formative assessment and alternative examination papers and other assessment tasks.

13. Products arising from learning • Assessment to incorporate reports, papers, artefacts, logs, performances, personal accounts, ... • Processes involved in learning • Directly assess or give feedback on student contribution to the processes involved, as with facilitating group-work, managing own learning or dealing with uncertainties or un-forseen eventualities. • Contexts in which learning occurs • Taking account of authentic, professional, unpredictable contexts in setting level of support, tasks and criteria. • Social structures underpinning learning • Assessment and feedback that builds on the involvement of others in pairs and groups, pattern of meetings, or through specific roles.

15. Fairness to students • Variation in tasks/contexts/level of support involved • Draw up assessment criteria that take into account the varying demands faced by students, especially when contexts differ significantly (as with lab and desk-based research). • Ensure that the credit size reflects the actual amount of student work involved in the module, or scale down your expectations. • Use of coursework requirements can avoid penalising students in cases where learning is contingent on factors beyond their control.

16. Responses: Examples • Learning a language through enquiries carried out in pairs (Morley and Truscott, 2004) • native speakers of two different languages • research-based task (discourse analysis, translation project, newspaper study or journal survey) • collaboration on task and mutual language support • assessed by short individual reports corrected on first draft by the partner.

17. Discussions - 1 • Consider a given assessment task. How could you adapt this task to: • draw from moments of contingency; take greater account of the full range of learning outcomes, products arising, processes, social structures and contexts involved; introduce efficiencies in staff time; and ensure fairness to students? • Option A Use your own example(s) • Option B Use one of the examples provided

18. Discussions - 2 • Consider a given set of assessment criteria and feedback mechanisms. How could you adapt these criteria and mechanisms to • draw from moments of contingency; take greater account of the full range of learning outcomes, products arising, processes, social structures and contexts involved; introduce efficiencies in staff time; and ensure fairness to students? • Option A Use your own example(s) • Option B Use one of the examples provided

19. Conclusions • Carrick model suggests that we should value renovating practice, and that embedding requires extensive personal engagement. • Change benefits from a developed network. • Significant shifts in understanding and expectations required to effect this agenda. • Curriculum Review offers scope to shift assessment approaches.

20. References • Black, P. and William, D. (2009) Developing the theory of formative assessment, EducAsseEval Acc 21, 5-31. • Kahn, P.E. and O’Rourke, R. (2005) ‘Understanding enquiry-based learning’ in eds. T Barrett, I M Labhrainn and H Fallon, Handbook of enquiry and problem-based learning: Irish case studies and international perspectives, NUI, Galway, pp1-12. • Morley, J. and Truscott, S. (2004) Collaborative enquiry-based projects for advanced language learning and academic literacy, in Learning Based on the Process of Enquiry, September 2003, University of Manchester, Manchester, pp 118-28. • McKenzie, J., Alexander, S., Harper, C. and Anderson, S. (2005) Dissemination, adoption and adaption of project innovation in higher education, Carrick Institute report, University of Technology, Sydney [Online, http://tinyurl.com/lpd5u7].