Closing the gap Evidence-based use of the pupil premium

1. @ProfCoe www.twitter.com/ProfCoe Closing the gapEvidence-based use of the pupil premium Robert Coe Closing the Gap in North Yorkshire, Harrogate, 27 June 2014

2. Outline • What can research tell us about the likely impacts and costs of different strategies? • How do we implement these strategiesto … • Focus on what matters • Change classroom practice • Target areas of need • Produce demonstrable benefits Improving Education: A triumph of hope over experience http://www.cem.org/attachments/publications/ImprovingEducation2013.pdf

3. Evidence about the effectiveness of different strategies

4. www.educationendowmentfoundation.org.uk/toolkit Impact vs cost Most promising for raising attainment 8 May be worth it Feedback Meta-cognitive Peer tutoring Early Years Homework (Secondary) 1-1 tuition Effect Size (months gain) Collaborative Behaviour Small gp tuition Phonics Parental involvement Smaller classes ICT Social Summer schools Individualised learning Small effects / high cost After school Teaching assistants Mentoring Homework (Primary) Performance pay Aspirations 0 Setting £0 £1000 Cost per pupil

5. Clear, simple advice: • Choose from the top left • Go back to school and do it For every complex problem there is an answer that is clear, simple, and wrong H.L. Mencken

6. Why not? • We have been doing some of these things for a long time, but have generally not seen improvement • We do not know how to get large groups of teachers and schools to implement these interventions in ways that are • faithful, • effective • sustainable

7. So what should we do?

8. Four steps to improvement • Focus on what matters • Think hard about learning • Change classroom practice • Invest in good professional development • Target areas of need • Evaluate teaching quality • Produce demonstrable benefits • Evaluate impact of changes

9. 1. Focus on what mattersThink hard about learning

10. True or false? • Reducing class size is one of the most effective ways to increase learning [evidence] • Differentiation and ‘personalised learning’ resources maximise learning [evidence] • Praise encourages learners and helps them persist with hard tasks [evidence] • Technology supports learning by engaging and motivating learners [evidence] • The best way to raise attainment is to enhance motivation and interest [evidence]

11. www.educationendowmentfoundation.org.uk/toolkit Impact vs cost Most promising for raising attainment 8 May be worth it Feedback Meta-cognitive Peer tutoring Early Years Homework (Secondary) 1-1 tuition Effect Size (months gain) Collaborative Behaviour Small gp tuition Phonics Parental involvement Smaller classes ICT Social Summer schools Individualised learning Small effects / high cost After school Teaching assistants Mentoring Homework (Primary) Performance pay Aspirations 0 Setting £0 £1000 Cost per pupil

12. Poor Proxies for Learning • Students are busy: lots of work is done (especially written work) • Students are engaged, interested, motivated • Students are getting attention: feedback, explanations • Classroom is ordered, calm, under control • Curriculum has been ‘covered’ (ie presented to students in some form) • (At least some) students have supplied correct answers, even if they • Have not really understood them • Could not reproduce them independently • Will have forgotten it by next week (tomorrow?) • Already knew how to do this anyway

13. A better proxy for learning? Learning happens when people have to think hard

14. Hard questions about your school • How many minutes does an average pupil on an average day spend really thinking hard? • Do you really want pupils to be ‘stuck’ in your lessons? • If they knew the right answer but didn’t know why, how many pupils would care?

15. 2. Change classroom practiceInvest in effective CPD

16. Improving Teaching • Teacher quality is what matters • We need to focus on teacher learning • Teachers learn just like other people • Be clear what you want them to learn • Get good information about where they are at • Give good feedback

17. How do we get students to learn hard things? Eg • Place value • Persuasive writing • Music composition • Balancing chemical equations • Explain what they should do • Demonstrate it • Get them to do it (with gradually reducing support) • Provide feedback • Get them to practise until it is secure • Assess their skill/ understanding

18. How do we get teachers to learn hard things? Eg • Using formativeassessment • Assertive discipline • How to teachalgebra • Explain what they should do

19. What CPD helps students? • Intense: at least 30contact hours, preferably 50 • Sustained: over at least two terms • Content focused: on teachers’ knowledge of subject content & how students learn it • Active: opportunities to try it out & discuss • Supported: external feedback and networks to improve and sustain • Evidence based: promotes strategies supported by robust evaluation evidence Do you do this?

20. 3. Target areas of needEvaluate teaching quality

21. Why monitor teaching quality? • Good evidence of (potential) benefit from • Performance feedback(Coe, 2002) • Target setting (Locke & Latham, 2006) • Accountability (Coe & Sahlgren, 2014) • Individual teachers matter most • Teachers typically stop improving after 3-5 years • Everyone can improve • Judging real quality/effectiveness is very hard • Multidimensional • Not easily visible • Confounded

22. Monitoring the quality of teaching • Progress in assessments • Quality of assessment matters (cem.org/blog) • Regular, high quality assessment across curriculum (InCAS, INSIGHT) • Classroom observation • Much harder than you think! (cem.org/blog) • Multiple observations/ers, trained and QA’d • Student ratings • Extremely valuable, if done properly (http://www.cem.org/latest/student-evaluation-of-teaching-can-it-raise-attainment-in-secondary-schools) • Other • Parent ratings feedback • Student work scrutiny • Colleague perceptions (360) • Self assessment • Pedagogical content knowledge

23. Teacher Assessment • How do you know that it has captured understanding of key concepts? • vs ‘check-list’ (eg ‘;’=L5, 3 tenses=L7) • How do you know standards are comparable? • Across teachers, schools, subjects • Is progress good? • How have you resolved tensions from teacher judgments being used to judge teachers? • Summative assessment includes teacher feedback

24. Lesson Observation • Two teachers observe the same lesson, one rates it ‘Inadequate’. What is the probability the other will agree? a) 10% b) 40% c) 60% d) 80% • An observer judges a lesson ‘Outstanding’. What is the probability that pupils are really making sustained, outstanding progress? a) 5% b) 30% c) 50% d) 70% www.cem.org/blog

25. Evidence-Based Lesson Observation • Behaviour and organisation • Maximise time on task, engagement, rules & consequences • Classroom climate • Respect, quality of interactions, failure OK, high expectations, growth mindset • Learning • What made students think hard? • Quality of: exposition, demonstration, scaffolding, feedback, practice, assessment • What provided evidence of students’ understanding? • How was this responded to? (Feedback)

26. 4. Produce demonstrable benefitsEvaluateimpact of changes

27. A research-engaged school • Draws on knowledge and understanding of research to inform • Pedagogical practice • Decisions about strategy and policies • Attempts to implement and embed more effective practices • Robustly evaluates • Its ongoing performance on a range of outcomes • The impact of any changes made

28. Key elements of good evaluation EEF DIY Evaluation Guide • Clear, well defined, replicable intervention • Good assessment of appropriate outcomes • Well-matched comparison group What could you evaluate?

29. RISE: Research-leads Improving Students’ Education • With Alex Quigley, John Tomsett, Stuart Kime • Based around York • RCT: 20 school leaders trained in research, 20 controls • Contact: aj.quigley@huntington-ed.org.uk

30. www.cem.org Summary … @ProfCoe Think hard about learning Invest in good CPD Evaluate teaching quality Evaluate impact of changes Robert.Coe@cem.dur.ac.uk