Pam Sammons & Kathy Sylva University of Nottingham/University of Oxford

1. Maintaining Momentum in Primary School: messages from research and evaluation Presentation Prepared for the Social Mobility and Life Chances Forum Maintaining Momentum, Milton Hill Oxford December 3-4 2004 Pam Sammons & Kathy Sylva University of Nottingham/University of Oxford

2. Content of Presentation • What is the impact of multiple disadvantage and does pre-school promote better child outcomes at primary school? - Evidence from the EPPE study • Does the primary school a child attends ‘make a difference’ to their educational outcomes? - Evidence from school effectiveness research • What is the evidence of ‘improvement through inspection’ and has this benefited disadvantaged groups? - Evidence from an evaluation of the impact of Ofsted • What kinds of specific interventions promote better outcomes for disadvantaged groups of children? – Evidence from the SPOKES study

3. Multiple Disadvantage Index Primary Reading standardised score Maths 6 standardised score N mean sd N mean sd 0 588 104.1 13.7 587 106.7 14.3 1 679 102.7 14.2 680 102.7 15.0 2 532 98.8 15.0 531 99.8 14.9 3 336 96.2 13.7 333 96.1 14.7 4 221 92.1 15.0 219 90.0 13.6 5 plus 167 89.9 13.9 165 89.9 14.6 All pupils 2532 99.6 15.0 2515 100.2 15.6 Attainments of EPPE Sample at end of Year 1 by Multiple Disadvantage

4. Impact of quality and duration

5. Effect of pre-school (v. no pre-school) on social-behavioural outcomes at school entry

6. The contribution of social class and pre-school to literacy attainment (age 7) WRITING at key stage 1, social class and pre-school experience READING at key stage 1, social class and pre-school experience

7. What reduces the risk of SEN? • Higher quality & longer duration of pre-school • Integrated settings and Nursery schools • Good home learning environment and employed parent(s) Pre-school reduced proportion of children ‘at risk of SEN from 1:3 to 1:5

8. Focus of SER • The central focus a belief in the potency of social institutions ‘the idea that schools matter, that schools do have major effects upon children’s development and that, to put it simply, schools do make a difference’(Reynolds & Creemers, 1990) ‘Effectiveness is not a neutral term. Defining the effectiveness of a particular school always requires choices among competing values … the criteria of effectiveness will be the subject of political debate’ (Firestone, 1990)

9. Aims & Goals of Early SER to promote Equity and Excellence • Clientele - poor/ethnic minority students • Subject matter - basic skills reading & maths • Equity - children of urban poor should achieve at same level as those of middle classes

10. Focus on Student Outcomes ‘For us the ‘touchstone’ criteria to be applied to all educational matters concern whether children learn more or less because of the policy or practice’ Reynolds 1997 ‘An effective school is one in which students progress further than might be expected from consideration of its intake’Mortimore 1991 SER seeks to identify the ‘Value Added’ by schools to student outcomes

11. Methodology • mainly quantitative, but case studies important • values reliability and replicability • seeks to make generalisations • works in partnership with practitioners • values the views and perceptions of teachers, students and parents

12. The Impact of Intake ‘ ‘Natural justice demands that schools are held accountable only for those things they can influence (for good or ill) and not for all the existing differences between their intakes’ (Nuttall 1990) SER seeks to disentangle the impact of prior attainment and background characteristics from the impact of school and classes/teachers on students’ progress/social or affective outcomes

14. Value added effectiveness category AAP Mathematics AAP Reading n % n % Positive Outlier (p<0.05) * 10 23 5 11 Positive (non-significant) As expected 7 16 17 39 Negative (non-significant) As expected 15 35 19 43 Negative Outlier (P<0.05) * 11 26 3 7 Example of value added feedback from Improving School Effectiveness Project: Primary Schools’ AAP Results N of primary schools= 44 * p<0.05 , MacBeath & Mortimore, 2000

15. Defining Consistency Within school comparisons focus on internal variation in effects • For different cognitive & non-cognitive outcomes • By different year groups within each school, including variations in class or teacher effects • For different pupil groups - boys/girls - initial low/high attainers, - low SES/high SES

16. Effectiveness is a relative concept which is time and outcome specific • Effective in promoting which outcomes? the what of effectiveness • Effective for which student groups? the who of effectiveness • Effective over what time period? thewhenof effectiveness

17. Differential Effectiveness • The size of school effects for black students were almost twice as large as for white students in the US (Coleman et al 1966) • Differences between public and private schools almost twice as large for low SES students as for middle class students, differences between schools for high SES students small in US (Bryk & Raudenbush, 1992) • School effects vary for students by race and low prior attainment in England. School effects larger for initially low attaining and for black Caribbean students (Nuttall et al 1989) • Primary school effects vary for students with low compared with high initial attainment in England, being larger for low initial attainers (Sammons et al 1993)

18. Equity Implications • Dutch primary schools are highly stable in effectiveness across grades for low SES students, less stable in effectiveness across grades for high SES students (Bosker 1995) ‘Schools matter most for underprivileged and/or initially low achieving students. Effective or ineffective schools are especially effective or ineffective for these students’ After Scheerens & Bosker 1997

19. The Processes of Effective Schools After Teddlie & Reynolds 2000

20. Non-rational approach to evidence fear of outsiders dread of change capacity for blaming external conditions set of internal cliques lack of competencies for improvement ..may have inside itself multiple schools formed around cliques and friendship groups .. There will be none of the organisational, social, cultural and symbolic tightness of the effective school The ‘ineffective’ school (Reynolds 1995)

21. Empirical Confirmation of SE : Meta-Analyses • cooperation • school climate • monitoring at school and class level • opportunity to learn (content coverage - homework - time) • parental involvement • pressure to achieve • school leadership After Scheerens & Bosker 1997 The most powerful factors are located at the classroom level. Schools should address ‘proximal variables’ like curriculum, instruction and assessment which emphasis student outcomesWang et al 1993

22. Processes for School Improvement • Clear leadership • Developing a shared vision & goals • Staff development & teacher learning • Involving pupils, parents & community • Using an evolutionary development planning process • Redefining structures, frameworks, roles & responsibilities • Emphasis on teaching & learning • Monitoring, problem-solving & evaluation • Celebration of success • External support, networking & partnership

23. Significance of School Effects Although the differences in scholastic attainment achieved by the same student in contrasting schools is unlikely to be great, in many instances it represents the difference between success and failure and operates as a facilitating or inhibiting factor in higher education. • When coupled with the promotion of other pro-social attitudes and behaviours, and the inculcation of a positive self-image,the potential of the school to improve the life chances of students is considerable. Mortimore 1998:143

24. Impact of Inspection: Outcomes of special measures over 10 years

25. Perceptions of benefits of inspections: 2002/03; comparison of head teachers’ and teachers’ views

26. Primary schools 23 44 26 8 Secondary schools 24 43 25 8 0% 20% 40% 60% 80% 100% Excellent/ very good Good Satisfactory Unsatisfactory/ poor Judgements of extent of Improvement of primary and secondary schools since their last inspection (2002/03 Annual Report)

27. Leadership and management 24 32 27 13 4 9 35 42 12 2 Teaching 0% 20% 40% 60% 80% 100% Significant improvement Improvement No change Deterioration Significant deterioration Primary schools: change of inspection judgements from first to second inspection (percentage of schools)

28. Percentage of 11 year-old pupils reaching level 4 and above in English, mathematics and science

29. Theproportion of good or better teaching in primaryschools

30. International Comparisons of Reading Attainment 2001:IEA

31. Is improvement greater in schools facing challenging circumstances?

32. 80 70 60 50 40 30 20 10 0 -10 Lessons from S10 inspections when schools went into special measures Lessons from S10 inspections two years after coming out of special measures Percentage of unsatisfactory /poor lessons in primary schools going into special measures and two years after coming out (2002/03) Percentage of lessons unsatisfactory or poor

33. Disadvantaged pupils are over-represented in schools judged to require special measures

34. Improving City Schools: key features of teaching • a high degree of consistency across the school • high expectations of pupils, matched by well planned support to help them meet the challenges of the work • skilful management of pupils in classrooms and effective use of time and resources • motivating teaching methods & materials, planned with the improvement of basic skills in mind Ofsted 2000

35. Challenges for 21st century Pupils from disadvantaged backgrounds are more likely than others to experience educational failure Reasons for eradicating school failure • philosophical/ethical- to promote fairness improvement in quality of life and opportunities for all groups, to encourage positive attitudes to future learning and self-esteem • political- to promote social cohesion and inclusion and empower young people as citizens to participate in a successful democracy • economic- to promote future prosperity & prevent waste of talent & avoid social/economic burden on Governments

36. Maintaining Momentum in the Primary Phase : messages from research & evaluation • Pre-school provides children with a better start to school and is particularly important in improving attainment for low SES pupils, the impact is still evident at age 7 years • Schools vary in their effectiveness. For disadvantaged groups the effectiveness of the primary school attended is particularly important. • SER provides an important evidence-base on the correlates of effective schools and teachers and has stimulated school improvement initiatives at national and local level. • Inspection, has helped raise overall attainment levels and improved the quality of teaching in primary schools. • Inspection has acted as a powerful catalyst for improvement of weaker schools and this has benefitted disadvantaged pupil groups because they are over represented in such schools. • For the most vulnerable groups of pupils intensive, structured and targetted interventions are needed at an early stage.

37. The EPPE team Kathy Sylva University of Oxford Edward Melhuish Birkbeck, University of London Pam Sammons University of Nottingham Iram Siraj-Blatchford Institute of Education, University of London Brenda Taggart Institute of Education, University of London http://www.ioe.ac.uk/projects/eppe http://www.dfes.gov.uk/research/ EPPE is an ESRC TLRP (Affiliate) project