Measuring Charter Quality

1. Measuring Charter Quality Eric Paisner, NAPCS Anna Nicotera, NAPCS Lyria Boast, Public Impact

2. Who is evaluating schools? • State Education Agencies (SEAs) • Local Education Agencies (LEAs) • Charter School Authorizers • Charter Organizations • Private/media organizations (e.g. Great Schools, US News and World Reports)

3. Why measure school quality? • Hold schools accountable for results • Identify schools for support, intervention, or closure • Inform students, parents, and communities • Provide a consistent set of metrics that policymakers and community members can use to compare school performance

4. Trends • Summative ratings • Multiple measures • Student growth models • Expanded proficiency metrics • College and career readiness measures • Student and parent engagement

5. Multiple Measures Summative Ratings

6. Summative Ratings Multiple Measures

7. What is included in most rating systems? Rating systems of all types generally include data related to five broad categories: • Student Growth • Proficiency • Subgroup Performance • College and Career Readiness (high schools) • Student and Parent Engagement

8. Wisconsin – Sample 2011-12 School Report Card

9. Student Growth • Student growth models assess how much students are learning each year. • In 2011-12, 22 states used growth models to evaluate schools. The most common growth models used to evaluate schools are: • Student Growth Percentiles • Value-added analysis • Value tables

10. Student Growth Growth models require two or more years of student-level assessment results It is important to ask whether “typical” growth is “adequate” to bring students to grade level. Growth can be difficult to assess for high school students when there are no annual assessments

11. Proficiency • NCLB AYP designations report the percentage of students meeting or exceeding proficiency. • Additional methods used to assess proficiency include: • Comparison to district or state performance • Targets for advanced proficiency • Evaluation of students at different proficiency levels – achievement index • Controls for differences in student population

12. Wisconsin Composite Index

13. Example of Proficiency Index The Louisiana School Performance Score (SPS) includes an index score based on how many students are in each proficiency level.

14. Subgroup Performance New approaches include: • Creation of consolidated “supergroups” to avoid double-counting students that belong to more than one subgroup • Focus on lowest-performing students instead of students in demographic subgroups • Use of “achievement gap” metrics that calculate gaps between different student groups

15. College and Career Readiness • Extended grad rates • Diploma quality • Advanced coursework • College readiness exams • Industry certification • College remediation • College attendance • Dual credits • Availability of postsecondary data continues to improve across states • Common data points include:

16. Student and Parent Engagement • Some rating systems have incorporated measures of engagement, including: • Parent and student surveys • Student retention rates • Student attendance rates Engagement measures can be difficult to quantify in meaningful ways.

17. Challenges of cross-state comparison • Absence of a national rating system • Different assessments and proficiency benchmarks in each state • Range of growth models used in different states; access to student-level data required to calculate student growth • Collection and access to college and career readiness data inconsistent across states

18. Discussion Topics Is the time right for a national measure of charter school quality? What data elements should be part of a national measure of charter school quality? What are the biggest hurdles to developing and implementing a national measure of charter school quality?

19. Discussion