350 likes | 551 Views
TMA feedback: can we do better? Mirabelle Walker. Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT). FAST project findings (1). Quantity and timing of feedback Sufficient feedback is provided, often enough and in enough detail
E N D
TMA feedback: can we do better?Mirabelle Walker Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT)
FAST project findings (1) • Quantity and timing of feedback • Sufficient feedback is provided, often enough and in enough detail • The feedback is provided quickly enough to be useful to students • Quality of feedback • Feedback focuses on students’ performance and learning, and on actions under their control • Feedback is appropriate to the purpose of the assignment and to its criteria for success • Feedback is appropriate in relation to students’ understanding of what they are supposed to be doing • Student response to feedback • Feedback is received and attended to • Feedback is acted upon by the student Gibbs & Simpson (2004–5)
FAST project findings (2) What students said about their TMA feedback • Received plenty of it • Motivated by praise and encouragement • Mainly received within three weeks • Read feedback but rarely acted on it
Idea of ‘depth’ of comment • Depth 1 – acknowledges e.g. ‘more needed here’; ‘good’ • Depth 2 – corrects / amplifies e.g. ‘you needed to mention xxxx’; ‘a good introduction’ • Depth 3 – explains e.g. ‘you needed to mention xxxx because …’; ‘a good introduction because …’
FAST project findings (6) Significant weaknesses in current practice: • Too much emphasis on justifying the grade • Lack of shared understanding of assessment criteria (students & CT) • ALs good at articulating students’ weaknesses; explaining strengths problematic • Lack of holistic assessment of students’ work
My project • Replicate some of FAST investigations in Technology – results similar? • Courses chosen: T173, T209 & T224 • Analysis of feedback on sample TMAs • Telephone interviews with students • Follow-up action with ALs, monitors, CTs, STs as appropriate – and review
Progress so far • Feedback analysed on all three courses • Some results circulated to T209 & T224 ALs • Some comparisons with Science made • Telephone interviews conducted on T209 & T224 – T173 next month • Some analysis of telephone feedback done – awaiting T173 to finalise
Category percentages compared – Technology courses & Science
Category percentages compared – Technology (not T209) & Science
Type of content comment compared – Technology courses & Science
Action taken on T209 and T224 • Document ‘Using TMA comments to good effect’ prepared for each course • Explains ideas of ‘depth’ and ‘feed forward’ • Contains course-specific examples of depth 2 & depth 3 comments • Sent out in first tutor mailing (2006) with commendation from Course Chair • Monitors briefed accordingly
Telephone surveys • T209 and T224 complete – carried out immediately after end of course • Examined • student’s perception of usefulness/helpfulness of feedback • whether (& how) student had used the feedback in any future TMA / the ECA / the exam • student’s preferences regarding placing of comments: on PT3, script, (pro-forma)
Preliminary findings (1) • Students are eager to receive their marked TMAs and do read the feedback … • … but they do not necessarily use the feedback again in the rest of the course (approx 20% said they never used it) • T209 students were likely to make more use of the feedback later in the course than T224 students • T209 students particularly mentioned using skills development feedback
Preliminary findings (2) • Not all students found that tackling the TMA, and the feedback they subsequently received, encouraged them to study the rest of the course • A small number of students said they were disappointed with the quality of the feedback (but some were surprisingly accepting)
Preliminary findings (3) • Overwhelming majority of students value comments on the script the most: • ‘where lost marks made clear’ • ‘tells me exactly where the mistake is’ • ‘very specific’ • ‘more evidenced against actual text’ • ‘easier seeing my work with comments relating to it’
Aside about feedback and eTMAs • We can’t assume that students read the PT3 first – or even at all • We can’t assume that students find and read a separate marking document sent back with the marked TMA • Some (most?) students find juggling documents on the screen awkward/difficult • It’s easy for tutors to place comments exactly where they apply • Turn-around times longer: symptom of a problem?
Action taken on T209 and T224 • T209 and T224:Students reminded to look for and read the PT3 • T209:Separate pro-forma dropped for eTMAs; tutors asked to copy and paste grids at end of questions
FAST project findings in Technology? Significant weaknesses in current practice: • Too much emphasis on justifying the grade • Lack of shared understanding of assessment criteria (students & CT) varied • ALs good at articulating students’ weaknesses; explaining strengths problematic • Lack of holistic assessment of students’ work
Good feedback in Technology? • Quantity and timing of feedback • Sufficient feedback is provided, often enough and in enough detail In general • The feedback is provided quickly enough to be useful to students In general • Quality of feedback • Feedback focuses on students’ performance and learning Too much biased towards performance on this TMA • Feedback is appropriate to the purpose of the assignment and to its criteria Purpose and criteria often implicit • Feedback is appropriate in relation to students’ understanding of what they are supposed to be doing Sometimes, but more explanations would be helpful • Student response to feedback • Feedback is received by students, and attended to Yes • Feedback is acted upon by the student Not sufficiently
Implications for Course Teams • Different sorts of questions and criteria elicit different types of feedback • (or) To elicit particular feedback, write the question and criteria accordingly – maybe even write the course material accordingly
More implications for CTs • Be explicit with ourselves and with ALs and students about what an assignment’s purpose and criteria are • Don’t assume that ALs will instinctively know what sort of feedback we’re hoping for – be explicit in the marking guide (or elsewhere)
Implications for Associate Lecturers • Need for shift towards emphasis on supporting student’s learning and progress through course, rather than just explaining what was wrong in this particular TMA • That implies more student-centred feedback – and more holistic feedback on PT3s • It may also imply giving as much (more?) emphasis to feedback as to marks
Implications for monitors • Need to shift emphasis from ‘Was the mark OK?’ towards ‘Was the feedback OK?’ • Need to encourage appropriate forms of feedback (and discourage non-appropriate ones?)
References • FAST presentations given at Open University, 10 February 2005 • Gibbs, G & Simpson, C (2004–5) ‘Conditions under which assessment supports students’ learning’ Learning and teaching in higher education 1(1) pp 3–31; available via http://www.open.ac.uk/science/fdtl/pub.htm
Centre for Open Learning of Mathematics, Science Computing and Technology (COLMSCT) The Open UniversityWalton HallMilton KeynesMK7 6AA http://cetl.open.ac.uk/colmsct