Jane Friesen, Mohsen Javdani and Simon Woodcock Simon Fraser University May 2009

1. Does public information about school quality lead to flight from low-achieving schools? Jane Friesen, MohsenJavdani and Simon Woodcock Simon Fraser University May 2009

2. Introduction Many jurisdictions are developing strategies to improve school quality at the K-12 level. School choice policies are increasingly popular: • increase competitive pressure on schools • improve disadvantaged students’ access to good schools School choice policies take two primary forms: • increase number of alternative choice schools • provide information about school-level achievement to encourage parents’ to choose “good” schools

3. Most recent evidence Direct provision of information about school-level achievement to parents increases probability that they choose higher-achieving schools Hastings et al. (2007) Hastings and Weinstein (2008)

4. Will public information have a similar effect? Information reaches a larger number of parents – preferred schools may hit capacity constraints • overall effect on choice may not scale up • disadvantaged children may be crowded out of access to good schools Children whose parents have poor access to media may not receive information Information is updated: • parents could be misled or confused when education authorities update public information if these measures are subject to substantial sampling variation.

5. Our questions • When released publicly, does news about achievement at a child’s school affect the likelihood of their changing schools? • Do some groups respond differently than others, particularly those who may have relatively poor access to private information (income, language)? • Does public information have persistent effects?

6. Outline of talk • Overview of approach and results • Empirical details • Implications

7. Overview – empirical approach • student-level longitudinal data • multiple cohorts of BC elementary school students • 1999-2004 our data span: the introduction of standardized testing, and the public dissemination of school-level results We measure the effect of public information on probability a student separates from her school following Grade 4 “within-school” estimator – we compare changes in behaviour in schools that get “good news” to changes in behaviour in schools that get “bad news”

8. Overview – public information in B.C. Spring 2000: first Foundation Skills Assessment (FSA) exams were written; reading, writing and numeracy; Grades 4 and 7; low-stakes Fall 2000: first FSA exams were released to schools, who were instructed to share the information with parents on request. October 2001: Results of the 1999/2000 and 2000/2001 FSA exams posted on the Ministry website. Each subsequent FSA result posted in the fall following the exam. June 2003: The Fraser Institute began issuing annual report cards on B.C. elementary schools. These include school scores based on FSA exam results, and rankings based on these scores. They receive very widespread media coverage.

9. Overview - results Public release of information about school-level achievement had a large effect on the inter-school mobility of some Grade 4 students. Response is particularly large among English-language parents in low-income neighborhoods English-language parents respond the first time that school-level achievement measures are placed in the public domain. Do not respond in subsequent years. Non-English language parents (especially Chinese) also respond strongly, but not until media provide widespread coverage of Fraser Institute’s school report cards.

10. Responses to public information 99,00,01 FI scores released 02 FI scores released 99 FSAs released 00 FSAs released 1999 2000 2001 2002 2003 English-language parents in low income neighborhoods respond Non-English-language parents respond

11. Institutional background: school access/choice Guaranteed access to the neighborhood public school May also choose a non-catchment neighborhood school, a public magnet program, or a private school. Access to non-catchment neighborhood school: • before 2003: required the permission of both the catchment school and the preferred school • after 2003: Open Boundaries policy allows students to attend any public school in the province provided space is available after catchment area students have enrolled Rules for admission to oversubscribed schools are set by province and individual school districts.

12. Empirical specification Linear probability model for whether a public school student separates from her school at the end of the Grade 4 school year 1999-2003, pooled School separations depend on: • individual characteristics • time-varying school characteristics • information • school fixed effects • year effects Standard errors clustered school-by-year.

13. Empirical specification • individual characteristics • gender, home language, special needs, own test scores, previous seps, French Immersion • census neighborhood chars, distance to school, distance to catchment school, # of schools in proximity to home • school characteristics • %ESL, %home lang,%abor, %special needs • school mean test scores (grade 4, current year)

14. Information variables Test scores are released in the school year following the school year in which tests are written i.e. information is based on one-year lag of test scores We allow information shocks to affect separations as: “news” interaction between information measure and indicator for year in which it is released “oldnews” Interaction between information measure and indicator for years subsequent to year of release

15. Empirical specification Effects of school and individual characteristics vary across information regimes. 1999 – no test scores available 2001, 2002 - school-level test score on Ministry’s website 2003, 2004 - school report cards widely publicized

16. Identifying assumption Previous years’ shocks affect separations only through their persistent effect on current test scores

17. Open boundaries Could be pent-up demand for separations from low-achieving schools Release of this pent-up demand under Open Boundaries could be confounded with first release of Fraser Institute scores We account for this by interacting school mean test scores with indicator for whether open boundaries in effect Interpretation of results before 2003: effect of information on seps under neighborhood enrollment Interpretation of results after 2003: effect of information on seps under Open Boundaries policy

18. Sample Characteristics, Grade 4 Students Enrolled in Lower Mainland Public Schools, 1999/2000-2003/2004

19. Effect of Information on Separation Probability, Grade 4 Students in Lower Mainland Public Schools, 1999/2000-2003/2004

20. Effect of Information on Separation Probability, Grade 4 Students in Lower Mainland Public Schools, 1999/2000-2003/2004

21. By Home Language and Quartile of Distribution of Mean Household Income

22. What have we learned? Some have worried that only well-off families can take advantage of school choice. This does not appear to be true. Families in low-income neighborhoods respond strongly to information.

23. What have we learned? Publicity associated with Fraser Institute rankings appears to play an important role in ensuring that information reaches non-English speaking families.

24. What have we learned? Annual updates of info don’t seem to confuse people – they don’t keep responding. On the contrary, annual releases may be important for informing newcomers to the province.

25. What have we learned? Public information about test scores provides information that parents use. Authorities should take care to ensure that information brings competitive pressure to bear on schools that are ineffective, rather than on schools that serve disadvantaged populations. Designing meaningful measures of school effectiveness is a challenge.