Mexico: School-Based Management

1. Mexico: School-Based Management Paul Gertler Harry Anthony Patrinos Marta Rubio-Codina

2. Motivation • Increasing number of projects with School-Based Management (SBM) components • Relatively little evidence being produced on outcomes • Analysis of large-scale compensatory education program in Mexico with SBM component, thru a retrospective evaluation

3. Outline • Background on compensatory program $ SBM component • Data & identification strategy • Results • Conclusions

4. Mexico’s Compensatory Programs • Supply-side intervention started in 1991 • Managed by SEP (Secretariat of Public Education), implemented by CONAFE (Consejo Nacional de Fomento Educativo, or National Council of Education Promotion) • Channel resources to the worst performing schools: • Reach the most disadvantaged • Reduce schooling inequalities • Increase schooling availability and school quality • Fine targeting of rural areas according to marginality • Since 1991, program has constantly evolved in terms of targets & interventions

5. CONAFE Interventions • Learning Materials Provision: • Provision of school and student supplies • Teacher Training • AGEs (Programa de Apoyo a la Gestión Escolar, or School Management Support Program): • Monetary incentives and other support to increase parental involvement in school management. • Parental Associations exist by law but are rather dysfunctional • Other interventions (not evaluated): • Improved technology and infrastructure • Teacher monetary incentives

7. Highlights • Each year 4.5 million primary & lower secondary students receive school materials (notebooks, pencils, rulers, geometry kits); 16,000 primary schools & telesecundarias received educational materials & computers • Construction or rehabilitation of educational facilities, including classrooms, toilets, playgrounds within schools: almost 10,000 works each year • About 13,400 teachers receive monetary incentives intended to reduce mobility & improve teaching; 122,000 teachers in 40,000 schools receive technical & pedagogical feedback • Parental organizations from more than 12,000 kindergartens & 47,000 schools receive funds to improve school • Initial Education component serves almost 0.5 million parents through orientation in early stimulation & child-care techniques • Multicultural and indigenous education activities undertaken

8. Compensatory Education in Mexico

9. Program Relatively Low-cost $50/student – • Compared to: $527 telesecundaria, $477 general middle school

10. AGEs • Support & finance training for Parent Associations (APFs) • Parents trained in school management of funds transferred to APFs, participatory skills, information on achievements of students & ways parents can help improve learning • Financial support to AGEs consists of annual grants transferred quarterly to APFs’ school accounts, from $500-$700 a year, according to size of school

11. AGEs: Example of SBM with Limited Powers

12. More on AGEs • Cannot spend money on wages & salaries for teachers • Most of the money goes to infrastructure • AGEs designed to promote school-parent cooperation • AGEs increase school autonomy through participation • Gets parents into the school, thru formal channel: • AGEs allow parents to see teacher show up • Parents can witness if their children pay attention • Participation gives parents power

13. 46% of Primary Schools have AGEs

14. Identification I • Treatment Schools: schools intervened (continuously) between 1998 - 2001 • Control Schools: schools intervened starting in 2001 or not yet intervened • CONAFE Treatment: • Dummy =1 if intervention year • Number of Periods school has received CONAFE • OPORTUNIDADES Treatment: • Dummy =1 if intervention year • Ratio of OPORTUNIDADES beneficiaries in the school (intensity) • Intermediate School Quality Measures: • Repetition Rates • Failure Rates • Intra-Year Drop Out Rates

15. Identification II • School level dif-in-dif estimation: School FE (as): changes in outcomes as a function of changes in treatment status • Identification assumption: change in controls measures what change in treatment would have been without AGEs (counterfactual); cannot test directly, but test pre-intervention trends are equal, likely that assumption is fine • Test: pre-intervention trends equal between treatment & control schools • State * Time Dummies: xst : controls time varying changes in state education policy and other state changes in economic trends • Treatment-Specific Time Trends (eg, different evolutions b/w treatment & control schools over time) • Time varying school characteristics: intensity of Carrera Magisterial • Robust SE Clustered at the School (allow serial correlation within a school over time)

16. Data Sources & Sample • CONAFE Administrative Data (1991 - 2002) • OPORTUNIDADES Administrative Coverage Data (1997-2003) • Outcomes: School Census (Censo Escolar 911), 1995- 2003 • 2000 Mexican Census & 1995 Conteo • 2 sub-samples: • All non-indigenous (general) rural schools • Q34: Non-indigenous rural schools in the top 2 quartiles of the CONAFE 2000 Targeting Index Distribution • Sample likely to better balanced • Pre-intervention trend

17. Sample Sizes

18. Sources of Variation • Number of schools attended increases over time • Differences in CONAFE and OPORTUNIDADES phase-in over time & space • Intensity of OP treatment (OP Ratio) varies across schools • Variation in timing of type of treatment within CONAFE schools

24. Threats to Identification I • Placement Bias: interventions are non-randomly allocated • Sign of bias depends on whether better/worse schools are intervened first • Time*State dummies capture shifts/changes in the allocation of resources in each state • Treatment status correlated with school characteristics: • If time invariant characteristics, school FE solve the problem • If time variant characteristics… • … re-do analysis on the sub-sample of CONAFE Schools that receive all 3 CONAFE interventions (AGEs & Supplies & Teacher Training) • Trend * Treatment capture evolution of treated schools over time • Results robust although larger estimated AGEs effects Better schools might be receiving AGEs first. AGEs effect might disappear… …CAPAGEs intervention starting in 2003: provide guidance in the administration of the AGEs monetary support

25. Threats to Identification II • Treatment might affect the distribution of students’ skills in the school; if so, then treatment correlated with unobserved ability • Check whether changes in total school enrollment over evaluation period • No significant changes in enrollment observed • Spillover effects within the school: • School average rate includes OP beneficiary & non-beneficiary students • If positive spillover effects from OP beneficiaries to non-beneficiaries, aggregate school effects can come from learning improvements from beneficiary and/or non-beneficiary students • OPORTUNIDADES effect potentially over-estimated • Spillover effects across schools/geographical areas: • Better/worse performing students select themselves into CONAFE schools • Unlikely given: (i) school choice is unlikely to be a dependent variable in rural Mexico (ii) small non-significant variation in enrollment in CONAFE school

26. AGEs are Cheap and Effective • While unit cost of compensatory program overall is $50 • AGEs are much less costly component • Parents at participating school receive $500-$700 a year • 45,000 schools; 4.5 million students • Total cost of AGE school grants is $26 million a year • Annual unit cost: $5.86 per student • Oportunidades – also effective, with other important benefits – pays students at primary level $100-$200 depending on grade • Pathway: Not infrastructure – but Parental Participation • Qualitative assessment confirms

27. Conclusions • Demand-side intervention have effect on learning outcomes: • nutritional component? • OPORTUNIDADES is very effective in keeping students in school • Results by intervention show large effects from AGEs that remain even after controlling for other education policies: • decentralization at the local level is effective & low cost Large efficiency gains • Difficult to econometrically identify the effect of school input provision • No evidence of synergies