Pennsylvania Charter Schools: What’s Working and What’s Not

1. Pennsylvania Charter Schools: What’s Working and What’s Not Christopher Nelson, Ph.D. The Evaluation Center Western Michigan University christopher.nelson@wmich.edu http://www.evaluation.wmich.edu

2. Purposes of the Talk • Identify key strengths and weaknesses • Identify key policy issues • Generate discussion of options for addressing weaknesses and shoring up strengths The Evaluation Center Western Michigan University

3. Outline • Background and data sources • Evaluative criteria • Key findings • Questions and discussion The Evaluation Center Western Michigan University

4. Background and Data Sources

5. Background on the Evaluation • Act 22 mandates evaluation after 5 years • Contract awarded through competitive bidding process • Current report is culmination of 4 years of field research and secondary data analysis • One of the longest sustained charter school evaluations • Evaluation report focuses mainly on aggregate trends and patterns The Evaluation Center Western Michigan University

6. Data Sources • Site visits to nearly all charter schools • Interviews • Focus groups • Observations The Evaluation Center Western Michigan University

7. Data Sources (cont’d) • Analysis of secondary data • PSSA • Demographic information • Financial data • Charter school annual reports • In-depth information on school missions and practices The Evaluation Center Western Michigan University

8. Data Sources (cont’d) • Documentation provided by schools • Mission statements • Internal evaluations • Interviews with district officials • Interviews with other stakeholder groups The Evaluation Center Western Michigan University

9. Evaluative Criteria

10. Evaluative judgments combine facts and values Thus, data alone cannot fully address questions of merit and worth Evaluative Criteria The Evaluation Center Western Michigan University

11. “Improve pupil learning . . .” “Provide expanded choices . . .” “Increase learning opportunities for all pupils . . .” “Encourage . . . different and innovative teaching practices . . .” “Create new professional opportunities for teachers . . .” “Hold charter schools accountable . . .” Evaluative Criteria . . .According to Act 22 The Evaluation Center Western Michigan University

12. Key Findings Student Achievement

13. Key FindingsStudent Achievement • Student achievement as an evaluative criterion • Key issues • Aggregate trends • Consistency of the charterschool effect • Correlates of success The Evaluation Center Western Michigan University

14. Charter school students have typically scored well below the average PA student Key FindingsStudent Achievement 150 point difference • But score levels say more about student composition than school effectiveness The Evaluation Center Western Michigan University

15. Charter school students have scored only slightly below those in demog. similar noncharter schools Key FindingsStudent Achievement 36 point difference • But matching on demographics might mask less tangible differences (e.g., “interventionist” parents) The Evaluation Center Western Michigan University

16. Average charter school student has gained 15 points per year, net of changes in student composition Key FindingsStudent Achievement Zero indicates parity with comparison schools The Evaluation Center Western Michigan University

17. Yet, there is considerable variation among schools – some positive Key FindingsStudent Achievement The Evaluation Center Western Michigan University

18. Yet, there is considerable variation among schools – others negative Key FindingsStudent Achievement The Evaluation Center Western Michigan University

19. Yet, there is considerable variation among schools Key FindingsStudent Achievement The Evaluation Center Western Michigan University

20. Key FindingsStudent Achievement – the Bottom Line • Achievement appears to be a source of modest strength • But considerable school-to-school variation • Not yet clear what the drivers of success are The Evaluation Center Western Michigan University

21. Key Findings Choice and Satisfaction

22. Key FindingsChoice and Satisfaction • Consumer satisfaction as an evaluative criterion • Key issues • Satisfaction with charter schools • Number of charter school options • Are charter school options any different than those in noncharter public schools? – the question of “innovation” The Evaluation Center Western Michigan University

23. Parents, students, and teachers are generally satisfied However, unable to make comparisons with noncharter schools Key FindingsChoice and Satisfaction The choice “halo” effect? The Evaluation Center Western Michigan University

24. Key FindingsChoice and Satisfaction • Typical charter school has a self-reported waiting list of 28% of current enrollment • Yet, schools report turnover rates of 0 - 40% per year • Dissatisfaction, counseling out, or realization of poor fit? The Evaluation Center Western Michigan University

25. The number of charter school options has grown considerably Key FindingsChoice and the “Supply” of Options The Evaluation Center Western Michigan University

26. Yet, those options remain concentrated in a few regions of the Commonwealth Only 18 of 67 counties (27%) have charter schools Of those, charter enrollment exceeds 1% in only 10 Key FindingsChoice and the “Supply” of Options The Evaluation Center Western Michigan University

27. Many charter schools offer new choices to particular education markets Non-traditional grade groupings (67%) Unique mission foci Non-traditional use of time and calendar Smaller schools & classes Cyber schools Key FindingsChoice and Innovation The Evaluation Center Western Michigan University

28. However, there is little evidence that charter schools are creating wholly new educational approaches Barriers to innovation Slack time Slack resources Challenges of start-up Consumer preferences Problems with CAB definition Key FindingsChoice and Innovation The Evaluation Center Western Michigan University

29. Key FindingsChoice and Innovation – the Bottom Line • Parents, students, and teachers appear to be quite satisfied with their charter schools • Number of charter school options has grown considerably but remains geographically concentrated • Many charter schools offer new options in particular markets, but few appear to be producing wholly new educational approaches The Evaluation Center Western Michigan University

30. Key Findings Equity and Access

31. Key FindingsEquity and Access • School choice . . . • Engine of desegregation? • Sorting machine? • Key issues • Types of communities served • Types of students selected (self-selected?) from these communities The Evaluation Center Western Michigan University

32. Key FindingsEquity and Access – School Location • Charter schools tend to be in areas with larger schools, lower test scores, and higher concentrations of low income and nonwhite students. The Evaluation Center Western Michigan University

33. Declining relative percentage over time Considerable school-by-school variation Key FindingsEquity and Access – (Self?) Selection More nonwhite students The Evaluation Center Western Michigan University

34. Slightly more low income students Relatively stable over time Considerable school-by-school variation Key FindingsEquity and Access – (Self?) Selection The Evaluation Center Western Michigan University

35. Fewer students with IEPs IEP students in charter schools tend to have less severe disabilities Considerable school-by-school variation Key FindingsEquity and Access – (Self?) Selection The Evaluation Center Western Michigan University

36. On the whole, charter schools have target communities of need In the aggregate, little systematic evidence of cream-skimming Possible exception of special education Outlier schools might raise some red flags Negative trend on race Some anecdotal evidence of cream-skimming Key FindingsEquity and Access – The Bottom Line The Evaluation Center Western Michigan University

37. Key FindingsEquity and Access – The Bottom Line • But, findings limited by limitations in using enrollment data • Family self-selection vs. school selection The Evaluation Center Western Michigan University

38. Key Policy Issues Achievement

39. What are the correlates of success? How can we reduce variability on the low end? What is the potential for success? Study correlates of success Facilitate identification and dissemination of best practices Key Policy Issues & OptionsAchievement The Evaluation Center Western Michigan University

40. Key Policy Issues & OptionsAccountability as Possible Correlate • Accountability lies at the heart of the charter concept • PA has done better than most states in reporting and technical assistance • Yet, there is still room for improvement The Evaluation Center Western Michigan University

41. Key Policy Issues & OptionsAccountability as Possible Correlate • Continue improvements to charter school annual report format • Make information public to facilitate market accountability • Continue providing technical assistance in developing clear, measurable goals The Evaluation Center Western Michigan University

42. Key Policy Issues & OptionsAccountability as Possible Correlate • Consider assessment of LEA capacity for oversight (wills, skills, & bills) • Consider the impact of “just-in-time” oversight by many LEAs • Consider targeted state oversight based on past performance & compliance The Evaluation Center Western Michigan University

43. Key Policy Issues Choice and Equity

44. Is choice good in itself or as tool for increasing achievement? Is innovation reasonable for charter schools? What counts as “innovation”? Possible trade-off between choice and equity Slippery relationship between satisfaction and achievement Key Policy Issues & OptionsChoice & Equity The Evaluation Center Western Michigan University

45. Slippery Relationship Between Achievement & Satisfaction The Evaluation Center Western Michigan University

46. Key Policy Issues Cross-Cutting Issues

47. Key Policy Issues & OptionsCross-Cutting Issues • Balancing achievement, choice, and equity – the challenges of multi-attribute evaluation systems • Cost-effectiveness • How do charter schools stack up against other investments in educational improvement? • Actual vs. potential charter school performance • Addressing questions of fact and value The Evaluation Center Western Michigan University

48. Final Thought • “By providing important data and raising key policy issues, this report has sought to make a sound contribution to the debate over the Commonwealth’s charter schools” The Evaluation Center Western Michigan University