Using research to get the best value from the Pupil Premium

1. Using research to get the best value from the Pupil Premium Steve Higgins, School of Education, Durham University s.e.higgins@durham.ac.uk @stig_01 National Middle Schools’ Forum Conference 2013 21st October 2013 Stratford Manor Hotel, Stratford on Avon

2. Sutton Trust/EEF Teaching and Learning Toolkit • Why we wrote it • Best ‘buys’ on average • Key messages for spending the Pupil Premium • Currently used by about a third of schools http://educationendowmentfoundation.org.uk/toolkit

3. The pupil premium • Aims: • to close the attainment ‘gap’ between the highest and lowest achieving • to increase social mobility • to enable more pupils from disadvantaged backgrounds to get to the top Universities • to provide additional resource to schools to do this • Was £600 in 2012-13 for fsm1 pupils; increased to £900 in 2013-14 and £1300 in 2014-15. 1 any child registered for fsm in the last six years and all looked after children, smaller premium for children of Service families

4. Resources and learning • Above a minimum threshold – no simple link • Conclusion: spending more won’t guarantee improvement - no simple solution • More money ≠ more learning

5. Smaller classes? • Complex evidence- no clear link with class size and achievement • Experimental trials suggest Classes need to be less than about 17 for 0.2 effect size… And teachers need to change the way they teach… But support from teaching assistants not as effective • The maths: £900 x 25 pupils x 3 classes with 50% on fsm= £33,750 = 1 extra teacher per 3 classes Class size reduction from 25 to 19 – expensive for little gain

6. One-to-one tuition • Highly effective I hour/ day over at least 6 weeks Support for class teacher to re-integrate Effect size 0.44 • The maths… 6 weeks x 5 days x 1 hour = 30 hours 4 days teacher time (more effective with an experienced teacher) Approx £700 (ECC models less costly) • Expensive but very effective Consider using pairs or triplets?

7. What should the Pupil Premium ‘buy’? • Secondary £5,200 per pupil • Primary £4,284 • Wide variation • Secondaries £4,000 to £9,000 • Primaries £3,000 and £8,000 • Middle Schools £3,300 - £8,000 (median £4,100) • Excludes Academies and Free Schools • Will £1,300 buy an extra three or four months learning for each pupil eligible for the Pupil Premium? (In England, data from 2009-10: https://www.gov.uk/government/publications/funding-for-primary-and-service-childrens-education-schools )

8. The Bananarama Principle • It ain’t what you do it’s the way that you do it… • So how do you spend to “get results”? • Or, what does the evidence say is a good investment or a poor investment for learning? • It ain’t what you spend it’s the way that you spend it…

9. What we tried to do • Summarise the evidence from meta-analysis about the impact of different strategies on learning (tested attainment) • As found in research studies • These are averages • Apply quality criteria to evaluations: rigorous designs only • Estimate the sizeof the effect • Standardised Mean Difference = ‘Months of gain’ • Estimate the costsof adopting • Information not always available

10. Best ‘buys’... New entry Toolkit

11. Good ‘buys’... New entry Toolkit

12. Low range ... Toolkit

13. Worst ‘buys’...(on average) Revised entry New entry Toolkit

14. Summaries • What is it?How effective is it?How secure is the • evidence?What are the costs? • What do I need to know?

15. The methodology • Systematic searches for meta-analyses of interventions and quantitative estimates of impact • Effect sizes aggregated (fixed effect model) • Converted to months progress • Moderator variables summarised • Costs estimated in terms of outlay for schools • Variation within toolkit themes greater than variation between them: it ain’t what you do… • Technical appendix available on EEF website

16. Toolkit

17. Overview of value for money Promising 1.0 Feedback Could be worth it Meta-cognition EY intervention Effect Size (potential months gain) Peer tutoring Homework (Secondary) 1-1 tutoring Summer schools Digital technology Phonics Smaller classes Parental involvement After school Needs careful thought Individualised learning Teaching assistants Performance pay 0 Ability grouping £0 £1000 Cost per pupil

18. Key messages • Some things that are popular or widely thought to be effective are hard to make work well in terms of tested attainment Ability grouping (setting); After-school clubs; Deployment of teaching assistants; Performance pay • Some things look more ‘promising’ Effective feedback; Meta-cognition and self regulation strategies; Early years intervention; Peer tutoring; Small group/intensive tuition; Parental involvement and engagement

19. Issues and limitations • Based on meta-analysis – averages of averages • Conversion to ‘months progress’ is a rough estimate • Intervention research is compared with ‘normal’ practice which is varied • Not ‘what works’ but what has worked – ‘good bets’ to support professional enquiry

20. The challenges (1): implementation • These strategies have been shown to be cost-effective in research studies • But when we have tried to implement evidence-based strategies we have not seen system-wide improvement (e.g. AfL, SureStart) • We don’t know how to get schools/teachers who are not currently doing them to do so in ways that are • True to the key principles • Feasible in real classrooms – with all their constraints • Scalable and replicable • Sustainable

21. The challenges (2) : context • This is what has worked (on average) • Where is there leverage for improvement in a school or class? • Will it build capacity? (i.e. is it efficient?) • For learners? • For teachers? • Is it inclusive?

22. For disadvantaged /struggling learners… • One ‘intervention’ won’t be enough Identify areas of greatest need Clear focus on improving learning, not (just) behaviour Mid and high attaining learners can be disadvantaged too! • Effects will need to be cumulative What will build learning capacity and capability? Need to track and evaluate – our best guesses are not always good enough

23. Evaluate impact EEF’s DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

24. Establishing a baseline • Teacher assessment vsstandardised tests? • Individual progress vs comparison group? • Test score increases or age-standarised progress? • Evidence suggests Teacher Assessment tends to over-estimate progress BUT reliance on standardised tests narrows the curriculum and encourages test preparation

25. One-to-one SEAL Behaviour Small group tuition Phonics TA support (Parent involvement) (Early years intervention) Feedback Meta-cognition Self regulation Peer tutoring

26. But remember…. “It ain’t what you do, it’s the way that you do it… that’s what gets results” Effective identification Adopting solutions, not bandwagons High quality intervention/support Evaluate to ensure learning progress Quality counts

27. Some Links The full report can be found on the EEF’s website: http://educationendowmentfoundation.org.uk/toolkit/ The toolkit is recommended by the Department for Education: http://www.education.gov.uk/schools/pupilsupport/premium/b00200492/ppstrategies Official information about the Pupil Premium and LA allocations is available at: http://www.education.gov.uk/schools/adminandfinance/financialmanagement/schoolsrevenuefunding/a00200697/pupil-premium-2012-13 Ofsted’s report is available at: http://www.ofsted.gov.uk/resources/pupil-premium Find out how much each school gets: http://educationendowmentfoundation.org.uk/toolkit/pupil-premium-calculator DIY Evaluation Guide: http://educationendowmentfoundation.org.uk/library/diy-evaluation-guide

28. Questions…

29. Using research to get the best value from the Pupil Premium: workshop session Steve Higgins, School of Education, Durham University s.e.higgins@durham.ac.uk @stig_01 National Middle Schools’ Forum Conference 2013 21st October 2013 Stratford Manor Hotel, Stratford on Avon

30. ‘Best value’ • Identify target group • Improve effectiveness of provision/support • Evaluate impact • Track target pupils • Adjust plan in response to tracking • Use funding to achieve wider outcomes?

31. High Impact Low cost High cost Low Impact

33. Preparing for Ofsted • Communicate your strategy • Teachers • Governors • Website • Track target pupils • Record data on spending • Evaluate accurately • Improve plan • Justify/argue!

34. Tracker links Ofstedresources: http://www.ofsted.gov.uk/resources/pupil-premium-analysis-and-challenge-tools-for-schools Leading Learner blog:http://leadinglearner.me/2013/10/10/pupil-premium-analyser-and-tracker/ NAHT: http://www.naht.org.uk/welcome/advice/advice-home/governance-and-infrastructure-advice/pupil-premium-reporting-2012-2013/ TES: http://www.tes.co.uk/teaching-resource/Track-Impact-and-Spend-of-Pupil-Premium-6121277/ Deepings School (CfBT Academy): http://www.deepingschool.org.uk/162/pupil-premium http://www.deepingschool.org.uk/uploads/asset_file/How%20are%20the%20Pupil%20Premium%20pupils%20doing.pdf

35. and WRONG! H.L. Mencken 1880-1956 For every complex problem there is a solution that is simple, neat…