A Randomized Evaluation of a School Scholarship Program in Rural India

1. A Randomized Evaluation of a School Scholarship Program in Rural India Karthik Muralidharan (with Michael Kremer, Venkatesh Sundararaman) Conference on Public Private Partnerships in Education World Bank, Washington DC 7-8 June, 2007

2. Outline • Background • Research Questions & Experimental Design • Questions for Discussion/Feedback

3. Private Schools in Rural India • Private fee-charging schools are widespread in rural India • Especially in areas with poorly performing public schools • 28% of villages had private schools in 2003 (probably much higher now) • 18% of school enrollment in rural India is in private schools (2006) • Measures of private school performance are superior to public schools – especially with village fixed effects • Teacher attendance, teaching activity, student attendance, test scores • Children attending private schools come from advantaged backgrounds relative to those attending public schools • Difficult to infer causal effect of private schools on performance • Omitted variables and also additional years of schooling • High levels of elite exit from public schooling • Several important questions need to be answered to assess the suitability of a voucher/scholarship/PPP model for improving school education outcomes in India

4. Andhra Pradesh Randomized Evaluation Study • 3-way partnership between Government of Andhra Pradesh, Azim Premji Foundation, and the World Bank • 5 year MoU with GoAP to systematically study the most promising policy options to improve developing country education via randomized allocation of programs • 4 major interventions have been implemented and evaluated so far (Muralidharan & Sundararaman 2006) • Group and Individual-level performance pay for teachers • Use of contract teachers as opposed to civil-service teachers • Cash block grants to schools • Starting a long-term study on the impact of school choice/scholarships/vouchers for disadvantaged children to attend private schools under AP RESt

5. Existing Research on School Vouchers • Several studies consider the impact of randomly offering students a chance to attend private/charter schools • Rouse in Milwaukee • Peterson et. al in New York City, Washington DC, and Dayton, Ohio • Angrist et. al in Colombia • Cullen et. al in Chicago • Hoxby & Rockoff in Chicago • These report varying extent of benefits to voucher recipients • No one finds voucher recipients doing worse off • But more evidence is needed on the overall impact of school vouchers on all the students in a system • Hoxby (2003), Lavy (2006), Hsieh and Urquiola (forthcoming)

6. Decomposing Peer Effects • Randomly allocated voucher programs typically create 4 distinct categories of students • In public school, don’t apply for voucher • Apply for voucher, but don’t win (and typically go to public school, but sometimes go to private school anyway) • Apply for voucher, and win (and likely to go to private school) • In private school, independent of the voucher program • Best studies typically compare the 2nd and 3rd groups • ITT and ToT measures • But we don’t know if the first and fourth groups are worse off as a result of peer effects • The public school potentially loses its most motivated students • The private school gets students below its existing average • Ideally, we would want to randomize entire communities into voucher programs and compare the result for all students with the results of all students in similar “control” communities without vouchers

7. Proposed Research Design • An Indian village is pretty close to a “closed economy” in terms of school choice – especially for primary schooling • Identify ~200 villages that all have an existing private school • Two stage randomization • Randomly select half the villages to receive school vouchers • Randomly select children in “voucher villages” to receive them • Track learning outcomes of all children in all schools in both treatment and control villages • Child-level comparison gives the impact on participants • Village-level comparison gives the overall effect of the voucher program • Phase 1 (Pilot) with ~32 villages starting now, with a planned expansion to ~200 villages next year

8. Scholarship Program Design (1 of 2) • The universe of children eligible for the scholarship will be those currently in government schools (grades 1-3) • Provides a sample of disadvantaged children without having to do means testing • The number of scholarships to be offered in a village will be capped at ~30% of the enrollment of all the government schools • Don’t want to empty out the government school • Parents will be required to apply for the scholarship to be eligible (should provide higher first stage but we will find out) • Once a child receives a scholarship, he/she will continue to receive one till the end of primary school subject to meeting attendance requirements and taking the end of year tests • Differential exposure to program across cohorts of recipients

9. Scholarship Program Design (2 of 2) • No topping up – scholarship amount will be set at around the 80th percentile of the private school fee distribution across all villages • Private schools can determine the number of such places at this scholarship rate (but must accept all students who are allocated to these places by a lottery – limits cream skimming) • All expenses for books, uniforms, and school supplies are being covered by the scholarship. • A transport subsidy may be provided in some cases, but may not be required if the choice is being exercised within the village. • The total scholarship spending per child (all inclusive) is expected to be around Rs. 3,200/child per year (USD 80/year). • This is significantly less than the spending per child in the government schooling system • At least Rs. 4,000/year counting only pure variable costs • Over Rs. 5,000/year including various overhead costs

10. Resource Equalization • Even if the “scholarship villages” do better it could be a reflection of additional resources in these villages • Resource equalization is a problem for many other studies as well • Possible solutions: • Make public school lose money for every student who leaves • Not easy since the main expense is very lumpy (teacher salaries) • Also politically much more difficult • Provide matching resources to the public schools in control villages • Doubles “scholarship cost” of program • Operationally more difficult because it becomes another “treatment” • Use estimates from other studies to net out the effect of additional resources in public schools • Most likely course of action – especially since this in Andhra Pradesh • Existing studies in AP will provide estimates of the returns to the 2 main categories of inputs to public schools – additional teachers, and cash grants with school-level flexibility on spending (no ‘infrastructure’ though)

11. Questions We Are Getting At • Do private schools perform better even after accounting for the unobserved variables that might determine private school enrollment? • Answer by comparing scholarship winners to losers in voucher villages • Can think of the question as whether marginal spending on education is best routed via the private sector • What is the aggregate impact of the program (and is any group worse off)? • Answer by comparing voucher villages to non-voucher villages • Unit of analysis is the grade-level average score • What is the functional form of peer effects? • How do parents exercise choice?

12. Questions We Are Not (Yet) Getting At • Effects of competition on productivity of public schools? • Not at this point – negative incentives for public schools are quite difficult practically (lumpy inputs), politically, and potentially ethically • Positive incentives for retention could be considered (your thoughts?) • Benefits of better matching? • Unlikely to be the focus at the primary level • Adequacy and nature of supply response? • Not at this point • Scholarships will only be redeemable at schools existing prior to study • Combining choice with the effects of information to parents on school performance? • Not in Phase 1, but could be an orthogonal/additional treatment

13. Issues for Discussion (1 of 2) • Take up • Aspiration gap, uncertainty of funding/unanticipated expenses • Larger distances to travel • Have required an expression of interest in applying (want a good first stage), and the number appears high (~90%) • Medium of Instruction/SR Adjustments • Most private schools are “English Medium” • Need to allow enough time for SR adjustments to be made • What grades to target? • Had initially intended grades 1-4 (moving to 2-5) • Now only looking at KG, 1, 2 (moving to 1, 2, 3) • Scholarship amount? • Don’t want top ups, 80th percentile makes it attractive for most schools and covers marginal costs for almost all schools

14. Questions for Discussion (2 of 2) • Transport - to provide or not? • More competition versus greater complexity • Very hard to provide a “standard” option for transport. • Best option may be to provide a subsidy to parents • Piloting both options • Thoughts on ways to provide incentives for the public school to retain children without explicit penalties • What expectations should the public school teachers be operating under? • How much and what kind of information should we provide to parents? • Thoughts on verification/payment/fraud prevention?