Is Education Reform in New Orleans Working? A Few Facts Swimming in a Sea of Unknowns Michael Schwam-Baird and Laura Mo

1. Is Education Reform in New Orleans Working? A Few Facts Swimming in a Sea of Unknowns Michael Schwam-Baird and Laura Mogg The Cowen Institute for Public Education Initiatives October 16, 2009

2. Situation Overview Claims made Regarding School Reform What the Data can tell us about School Reform in New Orleans Conclusions Presentation Outline

3. Situation Overview

4. NOPS was widely recognized as one of the lowest-performing school systems in the country. In addition the school system was poorly managed and had significant problems with corruption. Just before Katrina, NOPS was declared broke and its finances were handed over to a corporate turnaround firm. Voters in 2003 approved a constitutional amendment giving the state the power to intervene in chronically failing schools. With its new power, the state established the Recovery School District (RSD) as a mechanism for this intervention. The State of NOPS before Hurricane Katrina

5. The confusion and uncertainty after the storm were, to some, an opportunity to overhaul public education governance in New Orleans. The legislature, with Gov. Blanco’s support, passed Act 35, expanding the RSD’s authority to take over low-performing schools in New Orleans. With its new authority, the state took over more than 100 New Orleans public schools that were now considered failing. Hurricane Katrina and the Transformation of NOPS

6. Chartering was a way for the OPSB to re-open schools quickly as well as take advantage of federal charter school start-up grants. Despite having the authority to operate schools, the RSD viewed its role as overseeing charters. A rapid return of students to the city made RSD-run schools a necessity. The plan to charter schools was initially very controversial, though this has abated somewhat and the number and proportion of all public schools that are charters continues to grow. Charter Schools

7. School choice options prior to Katrina were limited. In a system with a significant number of charter schools, choice is inevitable After Katrina, both the RSD and OPSB were forced to institute choice for all schools out of necessity. City-wide school choice was eventually codified as official policy by the RSD and OPSB. School Choice

8. Charter and RSD-run schools were not subject to the teachers union contract and selected school-based personnel at the school site. Charter schools are not subject to state tenure and seniority laws. RSD-run schools’ work rules and a salary schedule are established at the district level. Many OPSB teachers are veterans from before Katrina while the RSD has used national organizations to attract inexperienced new teachers. Human Capital Policies

9. OPSB-run schools continue to be funded according to the prior year’s staffing needs. RSD-run schools are funded on a per-pupil basis according to a weighted student formula in amounts significantly more than before Katrina. All charter schools receive a set amount per pupil, regardless of the needs of the students they enroll. These schools have greater budget autonomy. Resource Allocation

10. There is disagreement about the extent to which these reforms have been implemented and the effects the reforms have had. It is difficult to assess these claims with the information available. Recent public opinion polls have shown that New Orleans voters see the changes to the school system as largely positive. The Effect of the Reforms

11. Claims Made Regarding School Reform

12. Choice has created an educational shopping mall. Parents will not choose low-performing schools and they will forced to close. (Charter advocates) Choice can be a strategy to meet the diverse needs of all children. (Community and political leaders) Many students are still unable to access the city’s better public schools creating a system of high and low performing schools. (Parent advocates & Union) Parents lack the information and resources to take advantage of school choice. (Parent advocates) School Choice/ Open Enrollment

13. Charter autonomy in hiring and other functions allows them to be higher quality schools. (Charter advocates) Charters have seen greater gains in student achievement than traditional public schools. (Charter advocates and leaders) Charter schools “cream” the best students, and access restrict access to some students. (Parent advocates and researchers) Charter schools enroll a smaller percentage of special education students than traditional public schools. (Parent advocates) Introduction of Charter Schools

14. The union’s resistance to change caused problems pre-Katrina. The elimination of collective bargaining provided a fresh start. (RSD and charter advocates) New teachers are motivated, willing to collaborate, and open to new strategies. (RSD) Evidence from national research finds that experienced teachers are more effective at raising student achievement. (Union) The RSD schools are seen as a “proving ground” for teachers who will either improve and move to other schools or leave the profession. (Union and teachers) Human Capital

15. The increase in per-pupil funds being spent by the RSD was necessary given the chronic under-funding of the district as well as the increased cost of educating children in poverty. (RSD) Many of these funds are one-time recovery dollars and the RSD will be unable to sustain its reforms once this money dried up. (Researchers and advocates) Charter schools, because of their ability to access private funding sources, are creating a system with large disparities in per-pupil spending between schools. (Researchers and parent advocates) Resource Allocation

16. What the Data on Public Education Can Tell Us about School Reform in New Orleans

17. System-wide Data

18. Significantly, proficiency rates grew in every testing area in the 3-year periods before and after Katrina.

19. On every state test, both before and after the storm, New Orleans outgrew the state in terms of the percentage of students scoring basic and above.

20. The growth rates in the three years after Katrina show that in three out of six tests the growth rate of public schools has increased significantly. In two of the six tests the growth rate grew by less than one percentage point. In 8th grade Math, the rate of growth has declined since Katrina. In four out of the six state tests, New Orleans outgrew the state at a faster pace after Hurricane Katrina. However, in two out of the six tests, New Orleans outgrew the state at a slower pace after the storm. City-wide Performance Conclusions

21. The growth in all areas of testing since the storm is certainly good news. In most areas, growth since the storm has outpaced growth before the storm. In a few areas, growth has not been greater. It is difficult to tie these trends to any particular aspect of reform. It’s not clear what the effects should be. We take these results to be generally positive, but not mind-blowing by comparison to growth before the storm. But you could make a lot of different arguments based on these numbers and others, and many do. What does it tell us?

22. Teachers by Years of Experience Before and After Katrina The proportion of teachers with 0-1 and 2-3 years of experience has increased significantly.

23. The proportion of new and less experienced teachers has grown dramatically. This does not seem to have had a generally negative effect on the school system, whose results have generally improved since the storm. Predictions of dire consequences from new teachers seem to have been unfounded. Predictions of radical improvement due to new teachers have yet to be proven. What does it tell us?

24. The percentages of students who were eligible for free or reduced lunch has changed only slightly since before the storm (77 percent in 2004-2005; 85 percent in 2008-2009). Within the category of students who are eligible for the school lunch program, there is a lot of room for variation. The proportion of families living in extreme poverty in New Orleans has declined according to the US Census Bureau (12.3% of families had less than $10K in income in 2005; 6.4% in 2007). This may mean that the proportion of very poor students, who are most likely to have educational challenges, has also declined. Student Population

25. Not much at the macro-level. These differences are suggestive because poverty is so closely correlated with achievement. But they are not a definitive explanation for higher achievement. What does it tell us?

26. Resources:Total Current Expenditures

27. Salaries per Pupil

28. Expenses per Pupil and Instruction Spending More resources are being spent on instruction post-Katrina, but most of this extra money is not being spent on teachers.

29. Schools are spending much more than before the storm on school operations. Some of these extra resources are being used for instructional purposes, though not necessarily on teachers. It is unclear how much extra resources might be responsible for better performance. The role of resources in schools is hotly debated given all of the other inputs that affect achievement. Extra resources are, however, a major difference between the pre-Katrina and the post-Katrina environment. If they are critical to sustaining improvements in education, then we should be concerned now that most of those extra resources are gone. Spending Conclusions

30. Data by School Type

32. The school type with the highest growth rate varies by grade-level and subject. No particular school type seems to outperform the others in terms of growth. Large differences in absolute scores between RSD-run schools and RSD charters in 2007 indicates that RSD charter students probably entered school with higher levels of preparation, giving some support to “creaming” claims among non-selective schools. Recent Stanford study does provide evidence that charter students as a whole outperform their peers in non-charter schools across Louisiana. The data for this is statewide and from before and after Katrina. School Type Performance Conclusions

33. Teachers by Years of Experience The RSD-run schools have relied heavily on new teachers, while the OPSB-run schools have a large population of veteran teachers.

34. No arrangement of experience levels seems to be particularly associated with better or poorer performance. Schools with large proportions of veterans have shown promising results, as have schools with high proportions of new teachers. Perhaps site selection is the more important variable? What does it tell us?

35. Students Eligible for Free and Reduced Lunch The real differences in eligibility between RSD charter schools, RSD-run schools, and even OPSB-run schools are probably relatively negligible

36. Student with Disabilities There is some evidence for the claims that students with special needs are more likely to end up in RSD-run schools by comparison to charter schools.

37. There are not great differences in the proportion of students eligible for free and reduced lunch among non-selective schools, though there could be some variation within income levels that this measure does not capture. There are important differences in the percentage of special education students educated by non-selective charters versus non-charters. What does it tell us?

38. Average Expenditures per Pupil RSD-run schools, a category that includes the district’s central office, are spending much more per student than the city’s charter schools, regardless of type. [Though they cannot be broken out the same way, OPSB-run schools and the OPSB central office also probably spend much more per-pupil than charters.]

39. Expenditures by Object in RSD-run and RSD Charter Schools Though RSD-run schools do spend more on teacher salaries than RSD charter schools, the biggest difference is in contracted services and supplies.

40. There is little evidence to support claims that charters have access to more resources than district-run schools. The RSD is spending a lot of money on operations. While contracted services and supplies may be able to be cut back, the extra money spent on teachers may be more painful to cut when federal money runs dry. What does it tell us?

41. Conclusion

42. Many of the claims made by advocates, union officials, researchers, and public officials are based on shaky evidence. This is due to both the politics of school reform and a lack of reliable data. Overall, student achievement in the city’s public schools has improved since the storm and, in many cases, at a faster rate than before the storm. However, achievement growth is not radically greater than before Katrina. The majority of New Orleanians report that the system of schools that has emerged since Katrina is better than what existed before. There is evidence from at least one study that students in Louisiana charter schools are outperforming their peers in district-run schools. What We Know

43. What effect are the many new teachers who have entered the system having on student achievement? Are some more effective than others? To what extent are demographic shifts in New Orleans caused by Katrina affecting aggregate scores across the system? How are the differences in student and family characteristics (including prior student achievement) affecting achievement patterns across different types of schools? How mobile are students? How often do special education students switch schools? What role are additional one-time resources playing? Will their eventual loss negatively impact the performance of public schools? Which reforms are driving achievement the most? With finite resources, this is important! Some of the Many Questions That Remain

44. Is Education Reform in New Orleans Working? A Few Facts in a Sea of Unknowns Cowen Institute October 16, 2009