Does School Choice Work? Effects on Integration and Student Achievement

1. Does School Choice Work? Effects on Integration and Student Achievement Julian R. Betts, Lorien A. Rice, Andrew C. Zau, Y. Emily Tang, and Cory R. Koedel

2. Overview of Entire Report • Entire book is downloadable from www.ppic.org • Today will focus on lottery-based assessment of impact of choice on math and reading achievement • Will briefly highlight other aspects of report: • “Who chooses” • Effects on integration • Non-lottery study of charters and achievement

3. Specific to this School Choice Study (I) • Many of the previous studies of effect of choice on achievement lack convincing statistical identification • Compare switchers to those students left behind • Compare switchers to local students at current school • This study uses lottery data to study the three main choice programs. Compares lottery winners to losers • Student fixed effects models for charter schools only that compare individual student to herself in years before switch

4. Choice Programs in San Diego • Busing available • VEEP (Voluntary Ethnic Enrollment Program) • Magnets • Both are legacies of 30-year-old court desegregation order • NCLB-ordered busing from “failing schools” • Not studied: minor program, tiny enrollments • No busing • Choice (State-mandated open enrollment program) • Charter Schools (1 charter pays district for busing)

5. Between 2001 and 2003, School Choice Enrollment Share Rose from 25% to 28%

6. Outline • Introduction • Who Chooses to Leave and Why? • The Effect of School Choice on Integration • The Effect of Choice on Achievement • Policy Implications

7. Who Exercises Choice - Personal Characteristics • Non-whites • Sometimes, but not always, more likely to apply • Blacks • Always more likely to apply • Twice the average probability of applying to VEEP high schools • English Learners • Generally less likely to apply • Are students with higher test scores, GPA or higher parental education being skimmed off? • Mixed evidence, weak effects

8. Who Exercises Choice – School Characteristics • Weak evidence that students base application decisions on academic characteristics of local and option schools • Distance is typically a barrier • At high school level, distance is strongest deterrent in Choice (open enrollment) program, which provides no busing

9. Outline • Introduction • Who Chooses to Leave and Why? • The Effect of School Choice on Integration • The Effect of Choice on Achievement • Policy Implications

10. Integration Results • Overall, choice programs… • Integrate along race/ethnicity and parental education lines • Segregate slightly along test score and English Learner (EL) lines • Decreasing Rank of integrating efficacy • VEEP, Magnets, Choice (no busing) • Choice actually lowers integration along race, parental education, test score and language status lines • Limited supply of places sharply reduces actual integration

11. Outline • Introduction • Who Chooses to Leave and Why? • The Effect of School Choice on Integration • The Effect of Choice on Achievement • Policy Implications

12. Overview • Lotteries represent a true quasi-experimental way of measuring impact of choice on choosers because selectivity bias is removed • Model reading and math test scores one, two and three years after the admissions lotteries for Fall 2001 • Two measures of math achievement per year, and three measures of reading achievement per year • Three types of school choice

13. How Lotteries Work in San Diego • Each application is assigned a random number • Applications to a given school and grade are sorted into priority groups (e.g. top priority goes to siblings), so potentially multiple lotteries per school/grade. • Table C.1 lists priority groups • Central administration admits in descending order of priority. Typically one priority group per grade undergoes a lottery

14. Preliminary Analysis for the Fall 2001 Admissions Lotteries • First, check which potential lotteries are “true” lotteries, in sense that proportion p winning admission obeys 0 < p < 1 • Generally a greater proportion of lotteries are “true” in grades above grade 3 • Elementary VEEP in particular: very small sample of students in true lotteries relative to other cases • Table C.2

15. Preliminary Analysis for the Fall 2001 Admissions Lotteries (II) • Second, for students in the subsample of lotteries that are “true”, need to check that lotteries are “fair”. • Should be no statistically significant difference in initial (spring 2001) test scores between lottery winners and losers • A regression of initial test scores on lottery dummies and a dummy for “winning” showed the latter to be insignificant

16. Preliminary Analysis for the Fall 2001 Admissions Lotteries (III) • Additional tests that lotteries are “fair”. • Repeated for 178 regression samples and null is rejected at roughly the size of the test, as expected (e.g. 0.6% of time is rejection at the 1% level) • Table C.3

17. Lottery Regression Models • Will focus on models of the “intent to treat”. • Let J denote number of true lotteries. • We will model the test score for student i in year t, where t is one of the post-lottery years 2002, 2003 and 2004. This student applies to lottery j, and in year t attends school s, so his score is denoted by Sijst.

18. Lottery Regression Models • Simplest model: • Model includes: • a set of dummy variables αj for the lotteries • a dummy variable WINijt and corresponding coefficient β indicating whether the student i, whose test score is modeled in year t, won lottery j, and • an error term consisting of an error component for school s in year t, ηst, and a white noise error term εijst:

19. Robustness Check: Estimate 4 Specifications • In addition to the previous model (1), add: • (2) Same as (1) but add spring 2001 test score • (3) Same as (2) but add square of spring 2001 test score • (4) Same as (3) but add personal characteristics • (5) Same as (4) but add classroom characteristics (teacher qualifications etc.) to see if we can explain any of the differences accruing to lottery winners

20. Preview of Results • Few differences across specifications • Will focus on model (2) which adds lagged test score to improve precision • Results for all four specifications appear in Appendix C

21. Effects of Winning a VEEP Lottery: CST Reading Achievement

22. Effects of Winning a Magnet Lottery: CST Reading Achievement

23. Effects of Winning an Open Enrollment (Choice) Lottery: CST Reading Achievement

24. Effects of Winning a VEEP Lottery: CST Math Achievement

25. Effects of Winning a Magnet Lottery: CST Math Achievement

26. Effects of Winning an Open Enrollment (Choice) Lottery: CST Math Achievement

27. Results of Choice on Achievement • In general, winning a lottery has no statistically significant effect on reading or math achievement one to three years after • Important exceptions • In some cases lottery winners fare slightly worse in first year, but recover to identical achievement • Students winning lotteries for magnet high schools seem to do better in math 2-3 years later • Effect sizes are meaningful: 0.18 and 0.23 in years 2 and 3 • But only for CST and not CAT/6

28. Charter SchoolsEffect on Achievement Gains • Relative to regular public schools • Charters do not systematically under- or overperform • A few exceptions. Most interestingly… • Often, elementary startup charters underperform in math and reading for 1 to 3 years, then perform at same level

29. Outline • Introduction • Who Chooses to Leave and Why? • The Effect of School Choice on Integration • The Effect of Choice on Achievement • Policy Implications

30. Conflicting Achievement Results • Choice affected some achievement results, but the more general finding was of no effect even three years after winning a lottery • Yet, school choice programs extremely popular in San Diego with 28% student share by 2003-2004 • Possible explanations: • Parents care about math and reading but lack information • Parents consider factors other than math and reading, such as socioeconomic climate, other achievement outcomes

31. Conclusions and National Implications • NCLB mandates that students in “failing” schools be given option to be bused to other schools • Question: Is the best way to help students at “failing” schools to move students from the school, or to reform the schools in question? • In San Diego, the NCLB Choice program piggybacked on VEEP and magnets • Effects likely similar, and small

32. Conclusions and National Implications • Additional aid to students in low-performing schools may be a better solution • Betts, Zau and King (PPIC, 2005) report some evidence that “within-school” reforms can boost achievement • Choice may provide better option in certain situations • But presumption that choice automatically confers large achievement gains is probably incorrect

33. Does School Choice Work? Effects on Integration and Student Achievement Julian R. Betts, Lorien A. Rice, Andrew C. Zau, Y. Emily Tang, and Cory R. Koedel

34. Supplementary Slides

35. Who Exercises Choice - Methodology • Look at applications to VEEP, magnet and Choice enrollment programs for 2001-2002 • Gives truer picture of demand for school choice than actual school switchers, because demand exceeds supply • Statistically model the probability that a student applies to VEEP; separate models for magnets and Choice (open enrollment)

36. Change in the Probability of Application for Local School Characteristics (Elementary)

37. Change in the Probability of Application for Local School Characteristics (Middle)

38. Change in the Probability of Application for Local School Characteristics (High)

39. Change in the Probability of Application for Option School Characteristics (Elementary)

40. Change in the Probability of Application for Option School Characteristics (Middle)

41. Change in the Probability of Application for Option School Characteristics (High)

42. Who Exercises Choice – Percent Change in Probability of Applying High School VEEP Magnet 3.2 Choice 2.7 2.2 1.7 1.2 0.7 0.2 -0.3 -0.8 -1.3 GPA Black Female Stanford 9 Asian Hispanic English High Parental Score Learner Education Relative to White, Male, non-English Learner, Low Parental Education

43. Black VEEP Applicants’ Change in % White (School Population) Potential versus Actual 50 45 40 35 30 Applications (potential exposure) 25 20 Lottery 15 Winners (potential) 10 Lottery Winners 5 Who Switch (actual) 0

44. Charter School Enrollment Share (%) in San Diego

45. Charter Schools Background • Charters vary widely in academic focus and remediation programs • Exempt from many district regulations and state laws • Other general characteristics • Students • 83% non-white (non-charters: 73%) • 66% eligible for free/reduced-price meals(non-charters: 56%) • Draw from a wider geographic area • Fewer teachers with long experience and advanced education • Often have additional costs such as rental of space, rental of school buses

46. Effects of Attending a Charter on Stanford 9 Scores

47. Effect of Attending a Startup or Conversion Charter Schoolon Stanford 9 Test Score Gain

48. Policy ImplicationsIntegration vs. Resegregation • Overall, school choice improves racial/ethnic and socioeconomic integration • Weak evidence that students with higher test scores or more highly educated parents more likely to apply • Effects small • Less clear for GPA • Does not support notion that choice programs “skim off the cream”

49. Qualifications of Integration Results • School choice programs could integrate more if supply constraints less severe • Notable integrative differences between Choice—which reduced integration along various lines—and VEEP and magnet programs • VEEP and magnets feature • Publicly provided busing • Regulation through geographic matching or preferences

50. Policy QuestionDoes Competition Force Schools to Improve? • Although not directly studied, student applications respond only weakly to average test scores at schools • If actual choices do not typically boost achievement, how can student departures induce schools that lose students to boost achievement?