The Performance of Decentralized School Systems: Evidence from Fe y Alegria in Venezuela

1. The Performance of Decentralized School Systems: Evidence from Fe y Alegria in Venezuela Public-Private Partnerships in Education Conference, World Bank June 7, 2007 Washington, DC

2. Introduction and motivation • Fe y Alegria is a private subsidized confederation of Jesuit schools that targets disadvantaged youths • It is a form of government contracting educational services: • Central Governments pay the salaries of teachers and the principal • Local and international foundations and agencies as well as voluntary fees from the community pay for infrastructure expenses • FyA maintains complete managerial autonomy and significant curricular flexibility • Less than 3% of the people who work there are of a religious order • The school principal and board can hire and fire teachers without public-sector union constraints, and the national FyA coordination appoints principals • Fundraising and major infrastructure projects are coordinated at the national level (some efforts are coordinated by the international federation)

3. Introduction and motivation • Founded in 1955 in Caracas by Father Jose Maria Velaz as an outreach program of Andres Bello Catholic University • Its expansion throughout the region after its inception in 1955 has led many observers to view it as a significant success (515,000 students in formal education in 2005) • Descriptive academic studies have documented its positive performance relative to the public school system (Swope and Latorre, 2000; Navarro and De La Cruz, 1998, Gonzalez and Arevalo, 2005) • However, no attempt at a rigorous econometric evaluation of its effects has been undertaken

4. Data • In 2003, 413,607 high school graduates took the mandatory “Prueba de Aptitud Academica”, which is Venezuela’s SAT for college admission • Test administrators collect extensive socioeconomic data on each individual and their families (mother’s education, family income, house quality, etc.) • We use 48,697 of these exam takers in that year: Fe y Alegria or public school graduates who finished their studies that year and who were between 14 and 22 years of age • Our final dataset includes 46,460 public school students and 2,237 FyA students (4,5% of total)

5. Key information for our strategy • FyA schools are oversubscribed (admit rates are around 35% and schools choose based on observables (wealth and geographic location) • We construct a socio-economic status (SES) variable from factor analysis on family characteristics and find that FyA students are of the same SES that public school students within each municipality • Although school placement was originally targeted at lower income areas, many of these areas have developed over time, so it is not clear that they are correlated with test scores within a municipality • Thus we assume that unobservables do not substantially affect both FyA enrollment and test scores

6. Estimation Strategy • We originally constructed an instrument based on the number of FyA schools in a municipality, but it had too little variation to obtain meaningful estimates (program intensity across the 330 municipalities is very low and even if restricted to those where it is >0, it is about 5%) • We estimate the Average Treatment Effect (ATE) via OLS controlling for a set of 54 dummy variables capturing family characteristics. OLS is consistent if there are no omitted variables and if the treatment effect is homogeneous • The OLS model is where T is the treatment indicator and X is a vector of dummy variables indicating {Venezuelan, Male, Married, Age, Student Works, Father's Profession, Mother's Education, House Quality, Income, Number of Siblings, How School Fees Are Paid, Transportation to School, Social Class}

7. Estimation strategy • Given the OLS baseline, we estimate the ATE via propensity score matching • The propensity score is estimated using a standard probit of participation on the observables: • The matching estimator simulates the counterfactual outcomes based on the “nearest neighbor” (J=4):

8. PSM • We drop observations in the control group that have propensity scores that are outside the support of the distribution of the treatment group, but don’t otherwise trim the sample, since we have a significant number of observations with P(X)<0.1 • All the data comes from the administration of the same test, with the same demographic questions asked of each student (no treatment heterogeneity; HIT(1997) “common economic environment”) • We drop observations in municipalities without a FyA school since there are significant cross-municipality differences in test scores and SES [ P(X)>0 ] • The test is conditional on high school graduation, but there is no selection into the test itself

9. Distribution of propensity scores • The support of the propensity score distributions in treatment and control groups is very similar

10. Results • The OLS results controlling for a number of observables give a 5% of a standard deviation on the verbal section of the test and a 6% of a SD on the math section • Coefficients on control variables are interesting: • Younger students tend to do better, as do students with fewer siblings • However, the effects of family income and house quality seem to be in an inverted-U shape, the middle income group does best • PSM results are qualitatively similar: 11% of a SD in verbal (not significant) and 8% of a SD in math • The difference between OLS and PSM can be due to heterogeneous treatment effects (Angrist, 1998)

11. Heterogeneous treatment effect

12. Concluding Remarks • FyA treatment improves performance by about 10% of the average, which is quantitatively significant • The difference between OLS and PSM arises due to heterogeneous treatment effects: the poor benefit the most from FyA treatment • We believe FyA performs better because of higher school-level autonomy, labor flexibility and a high esprit de corps

13. An expanding organization