Adjusting students' attitudes so that they benefit from feedback

1. Adjusting students' attitudes so that they benefit from feedback Quintin Cutts, Peter Saffrey Dept of Computing Science, Glasgow Uni Steve Draper Dept of Psychology, Glasgow Uni Emily Durrant Centre for Confidence and Well-being, Glasgow Contact: quintin@dcs.gla.ac.uk 8th Programming Workshop

2. Carol Dweck - Mindset • Two opposing attitudes to one's ability to learn • Fixed mindset • Ability is a fixed commodity • Performance level demonstrates ability level • All feedback interpreted as a measure of ability • Failures hit motivation and self beliefs hard • Growth mindset • Ability can be grown through effort • Feedback interpreted as information to assist growth • Failure is an opportunity to gain feedback & to learn • Experiments show that Growth M/S can be cultivated 8th Programming Workshop

3. Messages about learning make a difference • Mueller and Dweck 98 • "You must be smart at those problems" • "You must have worked hard at those problems" • Simpson, Open University 07 • Multiple short tutoring phone calls • Strongly emphasised growth mindset and personal motivation in every call • Dar-Nimrod and Heine 06 • Exposure to experience-based learning theory affected female performance in maths • Aronson, Fried & Good 01 • Writing exercise telling others about the growth mindset 8th Programming Workshop

4. Mindsets & Programming • Suggestion • Learning to program at university can easily generate a fixed M/S • Very high number of failure messages • School & university programming widely different • Very wide ability range in Yr 1 classes • Paradoxically • Experienced programmers must have a growth m/s • but a fixed mindset towards learners– "programmers born not made" • (Heslin, Wandewalle & Latham – Managers mindset affects quality of coaching) 8th Programming Workshop

5. 3 Implementations of Dweck • Explicit Mindset Training - (M) • 4 x 10-minute teaching sessions, led by tutors • Blackwell, Trzesniewski & Dweck (2007) • Crib sheet / Wiki - (W) • originated from McCartney et al (2007) – "35 ways that students get unstuck" • Feedback 'inducement' - (F) • Message on feedback sheet encouraging engagement • Remember, learning to program can take a surprising amount of time & effort – students may get there at different rates, but almost all students who put in the time & effort get there eventually. Making good use of the feedback on this sheet is an essential part of this process. 8th Programming Workshop

6. Experiment • Context • Level 1, 2 semester (24 wk) course • 2 lectures per week + 2hr lab/tutorial - 12 tutorial groups • Wk 6, Wk 13 class tests and Wk 12 lab exam • Interventions spread across groups • 2 Control, 2M, 2W, 2F, 1MW, 1MF, 1WF, 1WMF • Tutor training sessions and delivery • 1 for Wiki – used from Wk 2 • 4 for Mindsets – delivered in Wks 2-5 • no training for F – sheet given in Wks 3, 5, 7, 9 • Measures • Wk 1 and Wk 7 measures of mindset, self-efficacy, hope, +/- affect • Class tests and lab exam • Self-reported measure of previous experience 8th Programming Workshop

7. Results • Week 6 – first Class Test (CT1) • W, F main effects against CT1, p < 0.05 • Week 13 – second Class Test (CT2) • F main effect against CT2, p < 0.005 • Could this be due to imperfect random assignment of conditions? • PREV[0,1,2] against PREV[3] • p < 0.0007, effect size 0.78, mean diff 1.5 coarse grades • Uneven distribution of PREV[3] among groups • Test effect of F against PREV[0,1,2] and PREV[3] separately • adjusted effect size 0.36, mean diff 6%, or 0.6 of a coarse grade • Time of day not significant against F • but independently significant, effect size 0.5, or 0.9 of a coarse grade 8th Programming Workshop

8. What to make of it… • M and W imperfectly applied • so cannot draw any strong conclusions from results • F definitely delivering a result – Why? 8th Programming Workshop

9. Interpretation by student of feedback • Dweck suggests one very clear interpretation: • "This is saying something about my ability" • but experience regularly shows strategic interpretation • "It's saying something about whether I've put enough effort in" • We provide f/b hoping to enhance deep learning… • "It's advising me how to improve my subject skills/knowledge" • PDP training, and evident student behaviour, suggests • "It's showing me how my study skills could be improved" • Other interpretations are: • "The marker's an idiot, the feedback is wrong" • "This has nothing to do with me, it's random" 8th Programming Workshop

10. Action follows interpretation • Ability • fixed mindset – avoidance, drop-out etc • Amount of effort • if high enough, move on to something else, given my overfull life • Improve subject skills/knowledge • read feedback closely, incorporate into my understanding, via a range of practices • Improve study skills • reflect on activity, identify and carry out changes • Mark/feedback is wrong • initiate the complaints procedure • Random • Do nothing 8th Programming Workshop

11. 8th Programming Workshop

12. Sheet is directive and informative… • … right at the point of receipt of the feedback • We state the interpretation we think is most important • Learning to program takes time. You'll get there with sufficient effort. It's worth making use of this feedback. [and we almost hide the 'mark'] • We give appropriate actions • "Look at items…" <the circled items> • "F/b points – mostly worth reading, even if not identified for you" • …and the feedback is plentiful • far more than any tutor could hand write • Whole sentences, real explanations, with references • Feedback sheets become a potentially major resource • Not cheap to develop, but speeds up tutor marking time 8th Programming Workshop

13. Web questionnaire – 35 responses • 70% found the sheets (v.) helpful • 50% motivated to work harder • 55% find sheets helped later • Comments • Although the program may have worked, the sheet provided comments on how you could have improved the program… • … gave me the confidence and then the joy to continue to work hard… • I was getting problems outlined - I started working harder on next programs, so I could sort these problems before submission. I have started meeting deadlines much more efficiently. • Plenty of good common sense pointers are even now continuing to be helpful. Especially on structure. • Feedback in this course is better than that in other courses. =]. 8th Programming Workshop

14. Conclusions • Students amenable to subtle messages • and in their absence, apply their own interpretations/actions • We need to promote our attitude towards learning • in this case, on the values we place on feedback • and on the actions we expect them to take • We're now applying the rubric across the class • and will look for any effects in summer diet • We'll be exploring the other two interventions further • direct teaching about mindsets clearly valuable in other contexts • giving students strategies for solving problems is a key part of the growth mindset 8th Programming Workshop