Assessment which works: Enhancing student learning without killing the tutors

1. Assessment which works: Enhancing student learning without killing the tutors Prof. Sue Bloxham Susan.bloxham@cumbria.ac.uk

2. Some assessment challenges Low satisfaction rates; More students – more marking; Anxiety about innovation in assessment (extra work, QA); Difficulty engaging students in formative assessment, peer assessment. Focus today on pragmatic ways of transforming our assessment practice

3. Certification to identify and discriminate between different levels of achievement, and between students; providing a license to practice in the case of professional programmes; enabling selection of students for further study and employment. This is assessment of learning.

4. Quality assurance to provide evidence for relevant stakeholders (for example, external examiners, QAA, professional bodies); to enable them to judge the appropriateness of standards on the programme. This is assessment of learning.

5. Student Learning to promote effective learning; formative and diagnostic; steering students’ approach to studying; giving the tutor useful information to inform changes in teaching strategies. This is assessment for learning.

6. Lifelong learning: sustainable assessment to achieve an understanding of standards; to learn how to make judgments; to be able to use criteria; to be able to tell when you really understand something. This is assessment as learning.

7. The unbalanced purposes of assessment Assessment for and as learning Certification & QA

8. Assessment for certification & QA involves: Formal feedback; 2nd marking; Moderation; External examiners; Assessment boards. Slow, costly & high stakes Assessment for and as learning can: Be immediate; Regular; In-class; On-line; Involve students in their own assessment ; Uses peer assessment. Quick, cheap & low stakes

9. Characteristics of assessment which promote learning and employability • Formative, involving dialogue; • Demands higher order learning; • Learning and assessment are integrated; • Students are involved in assessment; • It promotes thinking about the learning process; • Assessment expectations should be made clear; • Involves active engagement of students, developing independent learning; • Tasks should be authentic and involve choice; • Tasks align with important learning outcomes; • Assessment should be used to evaluate teaching.

10. Exams & tests • They often come at the end of a course; • They are not integrated into the learning; • The criteria are oblique; • Tend to assess factual & conceptual knowledge only. • They rarely result in useful feedback (sometime usefully programmed into a test); • When students do badly we tend to blame the students rather than use the results to diagnose problems with our teaching.

11. Essays have the potential to meet many characteristics but: • Often no formative element; • Questions may ask students to ‘evaluate’ or ‘critically assess’ a topicbut if students can pass adequately by regurgitating others’ evaluation or criticism (from lectures or reading), they may avoid higher order learning; • Students not involved in assessment or thinking about learning; • Rarely authentic; • Criteria often oblique.

12. Discussion • Briefly discuss current practice with those sitting near you: • How well balanced are the different purposes of assessment? • How much are students engaged in their learning? • How satisfied are students with assessment and feedback?

13. Psychology Redesign • 560 students in groups of 7-8; • 3 week cycle culminating in 700-800 word essay e.g.’Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible? • Guidance provided for tackling the question and working in a group; • Best essays posted on VLE as feedback; • Students used familiar language to discuss academic concepts – Dialogue. Nicol 2008

14. Field-based enquiry • includes formative stages, students can get help and feedback in a low stakes way • Expectations available to students both in written criteria and embedded in feedback • Potential to integrate learning from university with learning from other contexts • integrated with the learning • encourages independent and active learning • involves students in the assessment process (avoiding grades in the early stages) • higher order skills, complexity • authenticity, choice

15. Peer assessment in lab reports • Formative • Students develop understanding of quality in analysing and reporting science • Higher order learning, focus on the science rather than a description of the process • Students involved in assessment, developing skills of evaluation

16. Assessment tasks for engagement & independent learning • Patchwork texts (students have regular tasks which they bring to class for formative feedback – tutor marks a final, reflective synthesis of these tasks – see P. Ovens for example in Science education) • Tutor sets readings with questions each week. Students bring draft answers for discussion with peers to class, groups can ask the tutor if they are stuck, then individuals submit revised answers (4-500 wds). Tutor posts generic feedback on VLE. Portfolio of final answers submitted. Tutor only marks week 4, 7, 9 for feedback & grade (selects the weeks after submission). • Students submit draft assignments on line for anonymous peer assessment. Students submit revised assignment with copies of their reviews and statement on how they responded to their reviewers (Liverpool example). Key feature is that the students see this formative work as directly contributing to summative work. Don’t call it formative!!

17. Caution • Multiple methods have been associated with negative learning outcomes for students • Extra workload for students (perhaps this is a good thing?)

18. Engaging students in formative assessment – key requirements • It clearly feeds into summative assessment tasks; • The students must submit it in some way (bring to class, post on line, hand it in) and action is taken if they don’t; • Students receive useful feedback on it; • It is not contaminated by summative purposes.

19. Selling peer assessment Evidence shows students find their peers a useful and more approachable source of help with assignments but we need to stress the main value in peer assessment is standing in the shoes of the assessor – not being assessed – because: • learning about standards – absolutely crucial to making progress and understanding feedback • Seeing other ways of going about the task – develops strategies for taking action • Key employability skill – being able to judge own performance and assessing and giving feedback to others • More opportunity for dialogue • Chance for more formative feedback Peer assessment needs to become a regular feature of programmes so that it is taken seriously and taken for granted as part of learning at this level.

20. Staff work load A heavy staff workload in assessment is not necessarily helpful to student learning. Improving assessment needs to be accompanied by a shift in effort from tutors to students: • more student activity and engagement (doing tasks, reviewing progress, self-regulation) and less staff activity (marking) • Formative assessment has the potential to achieve both of these and is most likely to increase achievement.

21. Tutor friendly formative feedback • Tutor posts good examples or model answers on VLE; • Students peer assess tasks using assessment criteria; • Tutors give feedback on posters, presentations in class; • Oral/ on-line feedback to group after marking a sample • Tasks done on-line (e.g EMQs, MCQs), auto marked and give immediate feedback; • Tutors put main effort into marking drafts (agreed with examiner), just checking for change and putting mark on final piece.

22. Example of module-level approaches: the use of exemplars annotated with feedback to encourage dialogue about assessment criteria From Oxford Brookes FDTL project on feedback Students write and submit individual assignm-ents Tutor assesses assignm-ents and prepares feedback Out of class activity Tutor leads discussion of exemplars previously marked and annotated with feedback Tutor hands back assignments and leads discussion on feedback In-class activity 22 Module timeline Submission point

23. From Oxford Brookes FDTL project on feedback Example of module-level approaches: Generic (non-personalised) feedback on drafts plus reflective commentary 4. Students rewrite and submit assignments with reflective commentary on how they have incorporated the generic feedback 1. Students draft & submit individual assignm-ents 2. Tutor marks sample of assignments and prepares generic feedback 5. Tutor assesses assignm-ents Out of class activity 6. Tutor hands back assignments with minimal formative feedback 3. In-class discussion of generic feedback In-class activity 23 Module timeline Submission point

24. Conclusion Addressing student dissatisfaction and staff workloads requires us to: • Balance the different purposes of assessment; • Encourage and reward student engagement; • Ensure that students’ study efforts are directed towards meaningful learning; • Provide sufficient low stakes, informal assessment and feedback for students to grasp both the standards required and the strategies needed to achieve them. • Use tutor time most effectively for learning and limit the time absorbed by QA to essential summative items;

25. Additional slides which delegates may find useful Prof. Sue Bloxham Susan.bloxham@cumbria.ac.uk

26. Dealing with concerns regarding assessment which isn’t so easily quality assured for certification Potential ways forward • Take a programme approach to assessment design • Make greater use of assessment methods which combine different purposes (e.g. inquiry tasks,) • Use peer & self assessment methods in low stakes ways, e.g. for formative assessment and feedback. • Make sure you have a robust method for dividing group marks (See University assessment guide) For example, peer, group and self assessment, in class assessment, presentations, feedback at the draft stage

27. Benefits to students of moving to in-class, on-line, ongoing assessment and feedback • Immediate feedback • More feedback • Assessment & teaching/learning are integrated • Students involved in assessment – gaining better understanding of standards and own performance • Potential for greater student engagement throughout modules • More independent study • integration with work experience • Raise expectations regarding study workload Quicker, cheaper and low stakes

28. Feedback from different sources collaborative tasks case discussions team assessment peer assessment self feedback work-based mentors, etc These methods allow students to check out their understanding and practice judgements in relative safety. Seeing how other students go about things is a key source of feedback on their own work and helps them find ways to fill the ‘gap’ between current performance/ knowledge and what is expected. 28

29. Feedback • Be clear about the purpose of the feedback ie what should students expect to get from it in particular instances. • Ensure students have sufficient support in understanding criteria and standards to relate to the feedback provided. • Require students to self assess against generic/cohort feedback. • Model the process of feeding forward for the students. • Require students to demonstrate how they have used feedback in subsequent work.

30. Some other assessment methods • Evaluation of journal article or other paper • Film or radio programmes about a specific topic • Research Grant applications • Lay commentary on specialist material, e.g. journal article • Poster – presenting information clearly & concisely • Presentation – oral communication • Problems and case study analysis • Reflective Journals, Diaries & learning logs • Writing tasks: newspaper articles, press releases, executive summaries, information sheets. • Wikipedia entry Shorter pieces can demand just as much student time but involve significantly less staff marking time.