What is current research telling us about assessment and feedback in higher education?

What is current research telling us about assessment and feedback in higher education? TEAN

‘there is more leverage to improve teaching through changing assessment than there is in changing anything else’ (Gibbs & Simpson, 2004:22)

Academic engagement transparency Criteria (e.g. LOs, prof. standards, grade descriptors Need for dialogue & assessment discourse Tension with standards marking Holistic v analytical moderation Innovative assessmentmethods formative assessment Skills of judgement Assessment of professional competency Changing views of feedback Learning –oriented assessment Student-tutor power relationship Computer assisted assessment Peer & self assessment

Professional judgement comes from: ‘repeated engagement in the appropriate activities in the company of those with expertise’ (Arnal & Burwood (date): 388)

This is a ‘community of practice’ approach to learning academic standards. We learn informally, through participation; through being partners in the assessment processes, not through being instructed. (O’Donovan & Rust, 2008).

Approaches to developing student understanding of assessment criteria Active student engagement 4. The ‘cultivated’ community of practice approach Tacit standards communicated through participation in informal knowledge exchange ‘seeded’ by activities 3. The Social Constructivist Approach Active methods used to communicate tacit knowledge, e.g. marking exemplars using criteria The future Formal activities and inputs informal activities and inputs The past • The traditional model • Tacit standards absorbed informally and serendipitously 2. Explicit model Standards explicitly articulated and passively presented to students Adapted from O’Donovan et al (2008) Passive Student engagement

Transparent standards? grade descriptors learning outcomes assessment criteria assignment guidelines marking schemes

The drive for transparency is based on the theoretical idea that if students know what standards they are being judged against, they are more able to fill the gap between their present performance and those standards (Sadler 1989), and marking will be fairer and more reliable.

Factors How many exams Variety of ass. methods How much summative assessment How much formative assessment How much oral feedback How much written feedback Timeliness of feedback Explicit criteria & standards Alignment between outcomes & assessment Can influence Student effort How much of the syllabus they cover Usefulness of feedback Use of feedback by students Whether students know what is expected of them Whether they focus on deep or surface approaches to learning Whether exams encourage learning Programme assessment environment From Gibbs & Dunbat-Goddet (2007)

Learning what is required This cannot be short circuited by trying to write down the criteria; it is tacit knowledge. • Repeated cycles of formative assessment allow students to gradually become part of a subject community of practice.

“Students…narrowed their attention and their effort to those things they were told would be assessed, put in less effort, covered less of the syllabus, adopted less of a deep approach and gained less satisfaction from their studies.’ From Gibbs & Dunbar-Goddet (2007) p 24

Characteristics that appear to support success and retention • Academic engagement • Social engagement • Active learning Crosling, Thomas & Heagney (2007) Improving Student Retention in Higher Education: The Role of Teaching and Learning

Spoon feeding • Too much guidance prevents students engaging properly with the task, with ‘doing’ the subject • Instead they seek to give us exactly what we have set out • Miller (2007) talks about ‘painting by numbers’ but not ever really getting the whole picture of what assignments are about.

Dialogue • Research is increasingly stressing the importance of student/ tutor dialogue in order to help them understand both feedback and guidance.

Challenges in marking research • Disjunction between espoused and actual marking (Orrell) - criteria featured in neither much (teacher educators may be an exception); • Tutors use internalised, holistic judgement • Staff work backwards from holistic judgement to the published criteria (Grainger et al, Tan & Prosser) • Many studies show variations between tutors in marking and interpretation of criteria in practice • Impact of power relations and negotiation in marking (Orr)

Issues in Assessing Professional Competence (Tang 2008) • consistency between different assessors, • the dual (conflicting?) role of facilitating student learning and judging whether the standards have been achieved • judgements made less by use of the published standards but more holistically drawing on the tacit knowledge and expertise of the assessor. • concern about how much assessment takes into account different contexts. • Tang argues that a mechanistic approach to standards may lead to a focus on discrete teaching behaviours rather than a broader assessment of good teaching.

Discussion • To what extent do the findings explored so far match your experience of assessment in teacher education and higher education in general?

Don’t take a deficit approach • It is easy to conflate the idea of ‘ability’….with [a]..lack of awareness and experience of the values, assumptions and practices of higher education.’ • ‘problems in decoding and responding to expectations appear to be particularly acute in relation to assessment criteria (Haggis 526 & 528)

Assessment of learning • Assessment for learning • Assessment as learning

Characteristics of assessment which promote learning and employability Formative, involving dialogue Demands higher order learning Learning and assessment are integrated Students are involved in assessment It promotes thinking about the learning process; Assessment expectations should be made clear; Involves active engagement of students, developing independent learning; Tasks should be authentic and involve choice; Tasks align with important learning outcomes Assessment should be used to evaluate teaching.

Formative assessment Consistent evidence shows formative assessment has particular benefits for low attainers and disadvantaged learners Black & Wiliam 1998

Feedback Agenda for Change It needs to be acknowledged that high level and complex learning is best developed when feedback is seen as a relational process that takes place over time, is dialogic, and is integral to learning and teaching. Feedback must be seen as a process not a product. There needs to be recognition that valuable and effective feedback can come from varied sources, but if students do not learn to evaluate their own work they will remain completely dependent upon others. The abilities to self and peer-review are essential graduate attributes.

comes at the draft stage; Is part of the teaching & learning; is forward looking; Involves dialogue; focuses on skills rather than content. Is relational Arrives after the work is finished; Is not integrated into the teaching; Doesn’t involve any dialogue; Focuses on the work completed; Is anonymous But written feedback often Studies suggest that useful feedback to students What does timely feedback really mean?

The problem with written feedback is that ‘monologue’ is trying to do the work of ‘dialogue’ Writing more and better feedback on assignments won’t improve matters because it cannot provide the interaction necessary to help students clarify their understanding. (Nicol 2008)

Feedback from different sources collaborative tasks team assessment peer assessment work-based mentors studio feedback, etc These allow students to check out their understanding in relative safety and see how others go about things. 30

Example of module-level approaches: the use of exemplars annotated with feedback to encourage dialogue about assessment criteria From Oxford Brookes FDTL project on feedback Students write and submit individual assignm-ents Tutor assesses assignm-ents and prepares feedback Out of class activity Tutor leads discussion of exemplars previously marked and annotated with feedback Tutor hands back assignments and leads discussion on feedback In-class activity 31 Module timeline Submission point

From Oxford Brookes FDTL project on feedback Example of module-level approaches: Generic (non-personalised) feedback on drafts plus reflective commentary 4. Students rewrite and submit assignments with reflective commentary on how they have incorporated the generic feedback 1. Students draft & submit individual assignm-ents 2. Tutor marks sample of assignments and prepares generic feedback 5. Tutor assesses assignm-ents Out of class activity 6. Tutor hands back assignments with minimal formative feedback 3. In-class discussion of generic feedback In-class activity 32 Module timeline Submission point

Conclusions • How we help students to become part of our academic communities, really understanding what is expected of them, and thus able to be in control of their achievement; • How we integrate feedback more thoroughly into our teaching and make it part of a dialogue with students; • How we broaden our diet of assessment methods in order to validly assess the broad range of learning outcomes that characterise modern higher education without confusing students about expectations; • How we tackle the reliability issues of assessing students holistically using tacit knowledge • How we help students understand assessment and become partners in the process

Any questions?

What is current research telling us about assessment and feedback in higher education?