Larysa Minzyuk - Felice Russo

1. “Improving Education through Accountability and Evaluation Lessons from Around the World” Rome, 3-5 October 2012 The Causal Effect of Class Size on Pupils’ Performance: Evidence from Italian Primary Schools Larysa Minzyuk - Felice Russo Department of Management and Economics- University of Salento (Lecce) larissa.minzuk@unisalento.it felice.russo@unisalento.it

2. OUTLINES • Motivations and purposes • Identification strategy • Regression Discontinuity design • Data and procedure • Results and conclusive remarks

3. Motivations and purposes • The relationship between class size and education attainment has been widely explored, but the existing evidence on the class size effect is still contrasting and somewhat inconclusive • In this paper, we estimate the class size effect in Italian public primary schools by using Regression Discontinuity (RD; Thistlewaite-Campbell, 1960, JEP) design, which has recently become a standard evaluation framework for solving causal issues with non-experimental data (not only in education) • in Italy a limited research has been done on this issue so far • e.g. data limitations; among the known studies, see Bratti et al. (2007, GE), Brunello and Checchi (2005, EER), Quintano et al. (2009, REST), Russo (2010) • Our main results • In Italian data, we do not find a significant evidence which supporting class-size reduction policy • There is an evidence of sorting of pupils’ characteristics around cut-offs points (25 pupils): pupils with “unfavorite” socio-economic background are in smaller classes

4. Regression Discontinuity Design An RD-based evaluation is appropriate when increases in grade-enrolment (forcing variable) are linked with jumps in class-size (treatment variable) as predicted by: (a) the threshold rule generating a (b) discontinuous relation between the two variables Individuals (schools and families) cannot precisely manipulate the grade-enrolment in order to receive or avoid treatment (i.e. to affect whether or not the fall on one side of the threshold or the other) Smoothness condition: Othervariables are smooth functions of the forcing variable conditional on treatment (i.e. the only reason pupils’ outcomes should jump at the cut-off is due to the discontinuity in the level of treatment) If 1.,2.,3. jointly hold, effects of class size on pupils’ test scores can be interpreted as the local average treatment effect of class size

5. Data and procedure (1) We conduct our study using information from two sources: • The first is the INVALSI test results of V grade pupils in primary schools in 2008/09. These results are available for 150,000 pupils coming from 5,303 public and private primary schools (‘circoli didattici’) from all Italian regions • we restrict our analysis to the public schools whose testing procedure was assisted by INVALSI supervisors • The second source of information is school-level administrative data from the Italian Ministry of Education (MIUR; those data do not contain information about schools in regions with a special statute) • we matched the INVALSI data sample with the dataset on school characteristics and class size coming from MIUR • because of missing data on pupils characteristics in INVALSI data, we have 25,407 pupils coming from 1,561 school units

6. Data and procedure (2) • Predicted class size (Angrist and Lavy, 1999) Φisis the V grade enrolment at schools where the pupil i studied in 2008/09, int(•) is the function that takes the greatest integer less than the given argument Average and Predicted Class Size, 2008/09

7. Compliance of Schools to the Rule, 2008/09

8. Data and procedure (3) • A standard model of fuzzy RD can be described as follows (van der Klaauw, 2002, p. 1262): Pisis the test score of pupil’s iin school s, CSisis average class size in V grade at school level, Φisis the V grade enrolment at school level, indicates the cut-off values of enrolment (multiples of 25), and α (•) and β (•) are functions of enrolment • When enrolment is a discrete variable, class-size effect can be estimated only parametrically (Lee and Card, 2008) • we decide a linear specification for both control functions α(•) and β(•), choosing the piecewise linear splines whose kinks correspond to the values of cut-offs (Urquiola and Verhoogen, 2009, AER; Zada et al., 2009) (2SLS). For instance (1st stage; 2 knots): CSis = β + β1 1[Φis> 25] + β2 1[Φis> 50] +β3 Φis + β4 (Φis- 25)1[Φis> 25] + β5 (Φis- 50)1[Φis> 50] + μis

9. First stage and base IV specifications (+/- 3 pupils intervals), 2008-09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

10. Reduced form of selected observables (+/- 3 pupils intervals), 2008/09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

11. First stage and base IV specifications (+/- 5 pupils intervals), 2008-09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

12. Reduced form of selected observables (+/- 5pupils intervals), 2008/09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

13. Treatment effect (+/- 3 and 5 pupils intervals), 2008-09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

14. Number of schools in enrolment intervals (+/- 3 pupils), 2008/09 Number of schools in enrolment intervals (+/- 3 pupils, sample compliant), 2008/09

15. Reduced form estimates of selected observables, sample of compliant schools (+/- 3 pupils intervals), 2008/09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

16. In this paper we make an attempt to estimate the class-size effect on the pupils' performance using the data from Italian primary schools. We base our estimation strategy on RD design. To apply RD estimation strategy, practitioners have to test if the assumptions of RD analyses are not infringed, otherwise it would invalid to infer a “treatment” effect of class size on pupils' test results In Italian data, we do not find a significant evidence which would strongly support class-size reduction policy When focusing on small intervals (+/-3 and 5 pupils) around selected enrolment cut-offs, selection problem is not evident, as, on the one hand, we do not observe clear stacking behaviour of schools at the thresholds, and, on the other hand, pupils' characteristics result to be distributed smoothly in the large majority of cut-offs for this subsample Conclusive remarks (1)

17. Conclusive remarks (2) • In contrast, as we observe in our data, we find that the stacking behavior is more evident in the reduced sample of compliant schools (+/-3 pupils)and there is a clearer evidence of sorting of pupils' characteristics around cut-offs points: in this sample, right sides of cut-off intervals include more pupils with “unfavorite” socio-economic background  class size are largely used in primarypublic schools as a kind of compensatory policy (West-Woessman, 2006) • Urquiola and Verhoogen (2009) have found an evidence that Chilean schools might exercise selection policy on enrolment. The authors suggest caution when using RD, especially in application to private schoolsthat may have a better control over enrolment compared to public schools • Zada et al. (2009) have found an evidence of “selection” policy in public secondary schoolsin Israel as well. It is worth noting that the authors have not found it in Israelian publicprimary schools.