Measuring the Achievement Effects of Charter Schools in North Carolina

1. Measuring the Achievement Effects of Charter Schools in North Carolina Helen F. Ladd (Duke) Presentation for SREE conference Based on joint research with Robert Bifulco (U. of Connecticut)

2. Policy issues • How do charter schools affect the achievement of students who attend them? (EFP, Jan. 2006) Topic for this talk • Other related issues -- How do charter schools affect racial segregation and why? (JPAM forthcoming) -- How does competition from charter schools affect student achievement in the traditional public schools? (EFP, Jan. 2006)

3. Methodological issues • Two main approaches Use of administrative data (NC, TX, FL, ID) Use of natural experiments (e.g.NYC and Chicago) Both have strengths and weaknesses • What is the question? • Effects of the enabling legislation? • Effects of particular schools on the achievement of students attracted to that type of school?

4. Charter schools in NC • Enabling legislation (1996) Cap of 100 schools Moderately permissive Full state funding of operating spending • Race Legal requirement: in line with racial mix in the district In fact, black students are overrepresented in charter schools and many charter schools are predominantly black

5. North Carolina Data • From the NCER Data Center • Test scores for all students in grades 3-8 in reading and math 1996-2002 • Includes students in charter schools • Scores normalized by grade and year • Student scores matched over time (VIP) => we can follow students over time

6. Sample information • About 1.5 million observations on about 440,000 students in grades 3-8. • Five cohorts of students, starting in 3rd grade, continuing through grade 8 or 2002. • About 8700 students observed in charter schools Preferred model identifies charter school effects based on test scores for almost 6000 students for whom we have data on test score gains in both regular public and charter schools (i.e. students who switch in or out of charter schools).

7. Regression models • Achievement (level or gains) = f(CH, control variables) CH is 0-1 indicator for enrollment in a charter school • Empirical challenge: To avoid the bias associated with unmeasurable characteristics of students who choose charter schools. • Three forms of the model -- test scores (levels) -- gains in test scores -- gains in test scores for the same students, i.e. with student fixed effects (preferred model)

8. Control variables • Control variables included in levels and gains models: -- race and gender of student -- parental education -- change in school from previous year (structural and non structural) -- year and grade indicator variables • Control variables included in fixed effect (student) model -- change in school from previous year -- year and grade indicator variables

9. Regression results Effects on student achievement (standard deviations) levelsgainsgains with fixed effects Reading -0.158** -0.062** -0.095** Math -0.255** -0.076** -0.160** • Bottom line : On average, charter schools have a negative effect on student test scores

10. Potential downward biasesrelated to the identifying sample (i.e. Coefficients are too negative) 1. Overrepresentation of the exiters => 5-6 percent bias 2. Underrepresentation of younger students Size of bias unclear

11. Other potential biases 1.Possible positive competitive effect of charters on traditional public schools 2. Possibility of declining test scores prior to move to a charter school 3. Differential effects of student characteristics by grade None of these appear to be a major problem

12. Extensions • Disaggregation by year of operation of the school. • Biggest negative effect during first year of operation • But negative effects persist through five years of operation (compare TX and FL)

13. Extensions (cont.) • Disaggregation by year of operation and years of student experience in a particular charter school • Students do the least well during their first year in a charter school, regardless of the age of the school. • Compared to those who stay in a charter school, those who ultimately choose to leave the school do less well during the years they are in the charter school. • After the first year in a school, those who remain in the charter school do no worse after the first year than if they had been in a traditional public school. But the net effect is still negative.

14. Conclusion • Generalizability of NC to other states? -- Compare TX, FL, Idaho • Review of strengths and weaknesses of this approach to measuring effects of charter schools