Evaluation of Education Maintenance Allowance Pilots

1. Evaluation of Education Maintenance Allowance Pilots Sue Middleton - CRSP Carl Emmerson - IFS

2. The Policy Problem (1) • Very large increases in participation in the later 80s and early 90s had levelled off by the mid-90s

3. The Policy Problem: Post-16 Participation and Socio-Economic Group Source: Youth Cohort Study, Statistical Bulletin 02/2000

4. The Policy Problem: Post-16 Participation and Gender Source: DfES Statistics 2001 http://www.dfes.gov.uk/statistics/DB/SFR/index.html

5. Reasons for Non-participation • Little evidence about the reasons • Money is one factor among many“...not surprisingly, money looms large in the accounts given by disaffected young people of their lives. They report one of the key barriers to further and higher education to be (lack of) money”. (Newburn, 1999)

6. The Policy Response • Multiple policy responses • Education Maintenance Allowance • to encourage participation, retention and achievement in post-16 education • focusing on young people from low income families

7. Education Maintenance Allowance • Household income < £13k • weekly allowance up to £40 per week • Household income £13k - £30k • weekly allowance tapers to minimum £5 per week • Retention and achievement bonuses • available to ALL awarded EMA • Receipt subject to compliance with Learning Agreement • 4 Variants being piloted

8. EMA Variants

9. EMA Main 10 LEAs Leeds and London 5 LEAs EMA Transport 5 LEAs EMA Extensions 4 LEAs EMA Evaluations

10. AREA VISITS • DATA COLLECTION • Labour market • Education profile • Take up of EMA • Socio-demographics • ROUND TABLE DISCUSSIONS • EMA Implementation groups • INTERVIEWS • LEA administrators • Careers Service representative • TEC representative • Employer organisations Evaluation in the LEAs

11. Qualitative Interviews with Young People/Parents Year 1 – Participants/Non-participants (young people and parents) Year 2 – Longitudinal interviews with young people and early leavers

12. Face to face October 1999 Face to face October 2000 Telephone October 2000 Telephone October 2001 Telephone October 2001 Telephone October 2002 Telephone October 2002 Telephone October 2003 Quantitative Design WAVE 1 WAVE 2 WAVE 3 WAVE 4 COHORT 1 COHORT 2

13. The data • Questionnaires have detailed information on: • all components of family income • household composition • GCSE results • mother’s and father’s education, occupation and work history • early childhood circumstances • current activities of young people

14. Matching approach • Involves taking all EMA eligible young people in the pilot areas and matching them with a weighted sum of young people who look like them in control areas • Difference in full-time education outcomes in pilot and control areas in this matched sample is the estimate of EMA effect • Crucial assumption is that everything is observed that determines education participation

15. How is this done? • Don’t match on all X’s, but can instead match on the propensity score (Rosenbaum and Rubin, 1983) • Propensity score is simply the predicted probability of being in a pilot area given all the observables in the data • Use kernel-based matching (Heckman, Ichimura & Todd, 1998) • The matching is undertaken for each sub-group of interest

16. Variables are matched on: • Family background • household composition, housing status, ethnicity, early childhood characteristics, older siblings’ education and parents’ age, education, work status and occupation • Family income • current family income, whether on means-tested benefits • Ability (GCSE results) • School variables • Indicators of ward level deprivation

17. Analytic Strategy for EMA Propensity Score Matching: Measures the Impact of EMA BUT Requires Large Sample Sizes Weighting Issues Limited Disaggregation Descriptive Analysis: Provides Contextual Detail Allows Disaggregation Overcomes Weighting Issues BUT Cannot Measure Impact

18. The Impact of EMA on Participation • EMA has increased participation by 5.9 percentage points • EMA had a larger effect on young men than young women Base: Eligible young people in Cohort 1 & 2

19. The Impact of EMA on Participation • Draw is from both those who would have been in work and those who would have been NEET Base: Eligible young people in Cohort 1 & 2

20. EMA and Retention at Year 13 • Slightly larger impact on participation in year 13 • Suggests that retention has not fallen Base: Eligible young people in Cohort 1 & 2

21. Participation and Retention by Variant

22. Conclusions (1) • EMA effect around 6 percentage points • Plays an important role in reducing gender differences in post-16 participation • Important to control for local area effects • matching on ward level data important

23. Conclusions (2) • More effective paying EMA to young person rather than parent • Bigger retention bonuses have significantly larger effect on retention than other variants • Increase in participation drawn from both work and other groups

24. What We Learn, And When: From PSM

25. What We Learn And When: From Descriptives (1)

26. What We Learn And When: From Descriptives (2)

27. What We Could Learn