The Three Rs: The Scope for Literacy and Numeracy Policies to Raise Achievement

1. The Three Rs: The Scope for Literacy and Numeracy Policies to Raise Achievement Stephen Machin* and Sandra McNally** * University College London; CEE and CEP, London School of Economics ** CEE and CEP, London School of Economics

2. Background Long lower tail of education and skills distribution in some countries: high percentage of population with low skills in literacy and numeracy. This is bad for the people concerned, for their children and communities and for national productivity. The higher frequency of adults with poor basic skills has been linked to perceived failures in the education systems of these countries – especially poor teaching.

3. Issues and Questions How can we ensure that the next generation of adults does not suffer from the same problems of poor and inadequate basic skills? Are there cost effective ways in which teaching of literacy and numeracy can be altered to raise achievement levels? Is it better (for children’s schooling outcomes) to tell teachers what to do by means of structured literacy and numeracy programmes?

4. Structure of Talk Description of extent to which better literacy and numeracy skills matter. Literacy and numeracy strategies. National trends. Empirical approach. Results Conclusions and implications (and a bit on what more we will do in future).

5. Adult Basic Skills and Wages - IALS

6. Wages and Childhood Cognitive Tests- BCS70

7. GCSE Performance and Key Stage 2 English and Maths - NPD

8. Intergenerational Correlations in Cognitive Skills – Children of BCS70 and BCS70

9. Our Earlier Work – Literacy Hour Highly structured hour of literacy teaching introduced to some schools through National Literacy Project in in 400 schools in England in September 1996 and September 1997. Constituted a discrete, well-defined and systematic change in how literacy is taught within primary schools (and where teaching standards were perceived to be low before). Daily literacy hour: 10-15 minutes whole class reading or writing; 10-15 minute session on word level work (e.g. phonics, spelling); 25-30 minutes directed group activities; plenary session at the end.

11. Our Earlier Work – Literacy Hour Gives setting where some children exposed to up to two years of the literacy hour, when other children were not. Used a difference-in-difference framework and statistical matching methods to compare outcomes in treatment and control group before and after policy was introduced (and taking care of pre-policy trends). Found significant improvement in KS2 English (of the order of 2.7-3.5 percentage points in the probability of achieving Level 4), that there was no evidence of this being explained by pre-policy trends/mean reversion and that the NLP was extremely cost effective. [You can do the same exercise for the national numeracy project – you get 3.3-4.1 percentage points on Probability of getting ≥Maths KS2 Level 4]

12. Literacy and Numeracy Strategies • ‘Centrally planned’ teaching approach for literacy and numeracy implemented in all primary schools in England (but not other parts of UK). • Essential components: 1 hour of literacy and numeracy per day; prescribed method and content. • ‘Projects’ had been introduced by previous (Conservative) government in 1996/97. National roll-out for ‘literacy hour’ was in September 1998; ‘numeracy hour’ in September 1999 (but announced at same time).

13. Research Questions • Does being exposed to the ‘literacy hour’ or the ‘numeracy hour’ matter for achievement? • Does length of exposure matter? • How does the national strategy compare to the more targetted ‘pilots’? • How did national performance in England with the strategies compare to countries without such strategies? • Are there longer lasting effects beyond primary school? • Are there any broader effects of the strategies? (e.g. on behaviour; attitudes to school; other subjects?)

14. Trends in the Percentage of Pupils Achieving the Required (National Curriculum) Standard in Key Stage 2 Tests at the End of Primary School

15. Introduction of Literacy and Numeracy Hours • The ‘literacy hour’ and ‘numeracy hour’ were both initially introduced into some schools in a small number of Local Education Authorities (12/150 for literacy; 13 for numeracy); 3 LEAs exposed to both for overlapping time period (but not schools in those LEAs) • Within the ‘pilot’ LEAs, schools were introduced in different waves: - Literacy hour: wave 1 started in 1996/97; wave 2: 1997/98. National Roll-out: Sept. 1998. - Numeracy hour: wave 1 started in 1996/97; wave 2: 1997/98; wave 3: 1998/99. National roll-out: Sept. 1999.

16. Methodology (2) • This structure means that a certain number of schools were exposed to the ‘literacy hour’ or ‘numeracy hour’ at a time when other schools were not and we can make before and after comparisons (so we can do difference-in-differences). • It also means that there is variation in the length of exposure to these teaching strategies faced by different cohorts of pupils. • In Wales, there was no ‘national strategy’. Every Welsh local authority drew up a local strategy and scope for more discretion by schools. We can use Wales as a comparison group.

17. Methodology (3)

18. Methodology (4)

19. Data • National Pupil Database: contains census of all pupils in English schools. Test results and school-identifiers from 1995/96 onwards (for age 11 pupils). More personal characteristics from 2002 onwards. Importantly, this contains the pupil’s test result in English and Maths at age 11 (national tests that are externally set and marked – undertaken by all pupils; used to compile ‘performance tables’ for primary schools). • School-level data: LEA and School Information System (e.g. number of pupils; school type; % eligible for free school meals; % non-white etc.) available for many years. School-level performance data available from 1994/95 onwards.

20. Data (2) • Welsh school data available from 1997/98 and pupil-level from 1996/97. Same tests at age 11 up to about 2000 after which there are some differences in the papers actually given to pupils between England and Wales. • Longitudinal Survey of Young People: data on about 14,000 15-year olds in English schools from 2001. Very rich data and we can link the data with whether the student went to a primary school that was in the ‘literacy’ or ‘numeracy’ projects.

21. Literacy Hour – effects of years of exposurePercent Achieving Level 4 Key Stage 2 English(13890 Primary Schools, 1995-2002, Sample Size 95412)

22.  Numeracy Hour – effects of length of exposurePercent Achieving Level 4 Key Stage 2 Mathematics(13933 Primary Schools, 1995-2003, Sample Size 123144)

23. Effects of the Pilots Versus Strategies

24. England Versus Wales: 1997-2000, Regression Models

25. National data: Proportion achieving expected standard in English

26. National data: Proportion achieving expected standard in Maths

27. Difference-in-Difference Estimates By Year (England Versus Wales)

28. Longer Term Achievement – GCSE Performance at End of Secondary School 1

29. Longer Term Achievement – GCSE Performance at End of Secondary School 2

30. Longer Term Achievement – GCSE Performance at End of Secondary School 3 • But most of these secondary school effects explained by primary school gains. • Include KS2 measures and effects mostly go away (small literacy hour effect remains in England comparison; small numeracy hour effect remains in England and Wales comparison). • The literacy and numeracy hours boosted primary school performance.

31. LSYPE Analysis of Exposure to Pilots Initial analysis of first wave of the Longitudinal Survey of Young People in England with respect to the ‘literacy hour’ finds no evidence for any effect of the literacy hour on behavioural outcomes (e.g. truancy, anti-social behaviour of various types) or attitudes to school work or school in general.

32. Summary Both the the literacy and numeracy hours were effective for raising primary school achievement. More years of exposure seems to have generated bigger achievement gains. The effect of both policies in ‘pilot’ areas was stronger than when policies were rolled out nationally, but the national strategies show better outcomes than Wales which has no strategy. Those who benefited from the literacy and numeracy hours do better in secondary school BUT this is mostly explained by the primary school gains.

33. Implications • There has been concern about the ‘plateauing out’ of KS2 performance. • Our results show the national literacy and numeracy strategies did enhance achievement and importantly that KS2 performance improved more in England under the strategies regime than in Wales which has no national strategy. • Training teachers to deliver a prescribed form of structured teaching seems to have been beneficial.

34. Future Work Separating out time effects and exposure effects in more detail – tests versus ‘saturated’ model. Explore heterogeneity of teaching methods in Wales in more detail. Impacts on other subjects and complementarities between English and Maths (but not vice versa).

35. Year-by-year effects of numeracy hour (where significantly different…previous table)