Patrick J. Wolf University of Arkansas Advocates for Parental Choice Symposium Milwaukee, WI

1. Patrick J. Wolf University of Arkansas Advocates for Parental Choice Symposium Milwaukee, WI June 16, 2013 Latest Research Findings on School Vouchers

2. About Me • Long and fruitful relationship with Catholic education • Rigorous training in scientific research methods • “Wolf is a well-known advocate of vouchers. He bases his advocacy on research…” AERA Newsletter • Department of Education Reform doctoral program

3. Presentation Plan • DC voucher findings after 4+ years • Questions/discussion • Milwaukee voucher findings after 4 years • Questions/discussion • Florida and New York City findings • Discussion of voucher program design

4. Overview of the DC School Choice Incentive Act Program • “3-Sector Strategy” • Student Eligibility • Grade K-12, residents of DC • Family income ≤ 185% Federal poverty line • Scholarship • Up to $7,500 to cover tuition, school fees, and transportation • Renewable for up to 5 years • Order of Priority for Receiving a Scholarship • Students attending schools in need of improvement (SINI) • Students in non-SINI public schools • Students already attending private school

5. Legislative Mandate for Evaluation • Independent, using “… the strongest possible research design for determining the effectiveness of the Program.” • Key Outcomes to be Studied • Academic achievement • School safety • Retention, graduation, college admission • “...success… in expanding school choice options” (satisfaction) • Effects on schools in DC • Annual Reports to Congress (Spring 2005-Spring 2010)

6. Who Was Studied? • 2,308 lotteried public school applicants • 2,012 students still school-age 4+ years after randomization • Characteristics of participants: • Average household income below $18,000 at baseline • Only 6 percent have mothers with BA • SAT-9 scores: 33rd percentile in reading, 31st in math • 44 percent from SINI schools between 2003 and 2005 (highest priority group for scholarships)

7. Study Design: Randomized Control Trial • Eligible applicants assigned by lottery to be offered (“treatment” group) or not offered (“control” group) a scholarship • Each spring, evaluation team administered: • SAT-9 • Parent surveys • Student surveys (grades 4 and higher) • Principal surveys (public and private) • Three comparisons made: • Treatment versus control (pure experimental) • Scholarship users versus control (adjusts for decliners) • Private versus public school in year 4+ (Instrumental Variables)

8. Scholarship Use Over 4+ Years • 26% of treatment students used scholarship throughout the study • Participation drop-off (net): averaged 22% per year • Top reasons for dropping out of OSP • Child got into a charter school (22%) • Lack of space (for students transitioning from K-8) (19%) • Moved out of DC (15%) • Transportation problems (14%) • Students more likely to drop out of OSP: lower initial test scores, older grade levels, male, with special needs, more siblings

9. Schools Attended by “Analysis Sample” in 2008-09

10. Summary of Key Findings • Large increase in high school graduation rates • Suggestive evidence of small gains in reading overall • No impacts apparent in math • Half of subgroups show reading impacts • Parents more satisfied with schools, view them as safer • No impacts on student satisfaction and reports of safety

11. High School Graduation Rates, Overall Sample & SINI Subgroup, Parent Reports, 2008-09 IOT ITT

12. ITT & IOT Impacts of the OSP on Reading Achievement Overall, by Years After Application

13. ITT & IOT Impacts of the OSP on Math Achievement Overall, by Years After Application

14. Effect Sizes & Months of Schooling for Statistically Significant Reading Subgroup Impacts Based on 3 Analytic Methods * Statistically significant at the 95 percent confidence interval.

15. Parent and Student Reports of Safety and an Orderly School Climate, 2008-09 **Statistically significant at the 99 percent confidence level.

16. Parent and Student Reports of School Satisfaction, 2008-09 **Statistically significant at the 99 percent confidence level.

17. Highlights of Qualitative Research • Focus groups & interviews 2004-2008 • Findings: • Parents more satisfied if two-parent family & extensive search • Experience improved for families over time (stigma, financial policies, welcome) • Safety replaced by academics as key factor • Strict standards or accommodation?

18. Context • DCPS proficiency and DC NAEP scores increased by over 20 percent 2005-2009 • DCPS per-pupil spending of $17,542 • Reliable studies indicate graduation is far more important than achievement: • Increases earnings, health, longevity, & marriage • Decreases unemployment and incarceration

19. Political/Policy Developments • Program closed to new applicants in March of 2009 • 2010 Appropriations Law cut funding, capped program, mandated shoddy evaluation, added regs • 2010 Republicans capture House, new Speaker Boehner lists OSP reauthorization as a priority • Lieberman reauthorization language added to 2011 federal budget as final element of compromise • Obama Administration initially refuses to admit new students to program, then relents • Obama Administration 2014 budget proposes to zero-out program

20. Reauthorization Changes • Increases OSP appropriation to $20 million/year • Increases voucher max to 12k for high school and 8k for K-8 • Removes most regulations on private schools • Mandates new rigorous evaluation • Good for 5 more years

21. My 5 Minutes of Fame http://www.youtube.com/watch?v=5GTX-278Rpc&feature=player_embedded

22. Questions on DC OSP?

23. Milwaukee Parental Choice Program (MPCP) • Nation’s first urban school voucher program • Launched in 1990 with 341 students in 7 schools • Dramatically expanded in 1998 • Served 23,198 students in 106 schools in 2011-12 • In 2006, State of Wisconsin called on School Choice Demonstration Project to evaluate program over 5 years

24. MPCP Schools and Student Enrollment

25. Wisconsin 2005 Act 125 • Enacted on March 10, 2006 • Modified the main MPCP statute (119.23): • Raised enrollment cap to 22,500 • Added school accreditation requirements • Initiated standardized testing of Choice students in grades 4, 8, and 10 • Mandated that a representative panel take the Wisconsin Knowledge and Concepts Examinations (WKCE) • Required scores be sent to the SCDP, then on to the Legislative Audit Bureau

26. SCDP Longitudinal Evaluation • Comprehensive – examining participating effects on students, parents, and schools, plus systemic and community effects • Multi-method – quantitative and qualitative, with test scores, surveys, focus groups, site visits • Rigorous – careful matching on test and student characteristics for “apples-to-apples” comparisons whenever possible • Longitudinal – five years of data, fall 2006 through fall 2010, with focus on changes from baseline

27. Highlights of Quantitative Results • Positive effect of program on attainment • Some evidence of reading gains but none of math gains • Lots of school switching • Poor performing schools in both sectors being closed • Pressure of competition has increased MPS achievement slightly • Taxpayers save over $50 million/year

28. Effect of 2006 MPCP Attendance on 2010-11 Educational Attainment

29. Effect of 2006 MPCP Attendance on Student Achievement: 2007 to 2010

30. Program Expansion 2011 • Income limit increased to 300% of poverty • Permanent income qualification • No enrollment cap • Students can attend non-Milwaukee private schools • Schools can require top-ups from higher-income students • Testing of all Choice students in grades 3-8 & 10 using the state test • Regulations dropped regarding curriculum and teacher training • Program expanded to Racine

31. Program Expansion in 2013 • Maximum of 500 students outside Milwaukee/Racine (1000 beyond 2013) • Any school district, limited to 1% of enrollments • Family income < 185% of poverty • Maximum amount raised to $7,210 elementary and $7,856 high school in 2014-15 • Added individual income tax deduction for private school tuition

32. Questions/Discussion of MPCP?

33. Florida Tax Credit Scholarship Program & Evaluation • Launched in 2001 in response to court ruling • Served nearly 38,000 low-income students in over 1000 schools in 2011 • State Dept of Education longitudinal evaluation by David Figlio • Latest finding: program boosts reading achievement for students close to income cut-off

34. New York City Attainment Study • Privately-funded partial-tuition scholarship program expanded in 1997 • 1900 students entering grades 1-5 randomly assigned • Increased college enrollment rate for African-Americans by 7.1 percentage points if offered and 8.7 if used • No overall effect or effect for Hispanic students

35. The Question of Policy Design: What Should Arkansas Do? • Universal or targeted? • Voucher, tax-credit, or ESA? • Size of the scholarship? • What should it cover? • All top-off, some top-off, no top-off? • Apply admission criteria, first come, or lotteries? • Hold the public schools financially harmless or $$$ travels with the child? • Regulations on participating schools? • Required evaluation, testing, or financial reporting? • Programmatic supports for families?

36. For More Information Electronic versions of all DC and Milwaukee reports available at: http://www.uark.edu/ua/der/SCDP.html Patrick J. Wolf, Ph.D. Professor and 21st Century Endowed Chair in School Choice Department of Education Reform College of Education and Health Professions 201 Graduate Education Building University of Arkansas Fayetteville, AR  72701 Phone: 479-575-2084 FAX: 479-575-3196 pwolf@uark.edu