Commissioner’s Performance-Based Accountability Task Force: A Proposal for a Multi-level System

1. Commissioner’s Performance-Based Accountability Task Force: A Proposal for a Multi-level System Deb Wiswell &Scott Marion January 29, 2010

2. Adequacy Accountability System • Two approaches for demonstrating that the school is providing the opportunity for an adequate education • Input System—Based on a subset of the school approval standards • Performance System—what this committee is required to design • A school has to demonstrate that it is providing an opportunity for an adequate education by meeting the requirements of at least one of the systems, but not both. Commissioner's Performance Based Accountability System Task Force. 1/29/10

3. A Multi-Level Performance Accountability System for NH • Level One • A very limited set of common (across the state) indicators and metrics • Level Two • Locally determined goals, targets, and indicators Commissioner's Performance Based Accountability System Task Force. 1/29/10

4. Level One Overview: State System • A very limited set of common indicators and metrics • Applied consistently across all schools in the state • Focused on unarguable outcomes, e.g., NECAP, graduation rate, postsecondary assessments, attendance Commissioner's Performance Based Accountability System Task Force. 1/29/10

5. Level One: A K-8 Sketch • Inclusion Factors • Status Measures • Growth Measures • Gap Analyses Commissioner's Performance Based Accountability System Task Force. 1/29/10

6. Level One: A High School Sketch • Inclusion Factors • Status Measures • Gap Analyses • Postsecondary Indicators Commissioner's Performance Based Accountability System Task Force. 1/29/10

7. Level One: State System Indicators • All schools • Test participation • NECAP index scores (status)—reading, writing, math, science • NECAP status gaps—reading, writing, math, science • Attendance/truancy • Additional indicators for K-8 Schools • NECAP growth percentiles (growth)—reading, math • NECAP growth gaps—reading, math • Additional indicators for High Schools • Graduation rate • Postsecondary assessments (AP, SAT, ACT)? Commissioner's Performance Based Accountability System Task Force. 1/29/10

8. Inclusion: Common Indicators for K-8 & HS • Test Participation—required level of participation (e.g., 95%) to meet performance requirements • Weighted average across all state tests given in the school • School must average 95% participation to meet adequacy standard • Attendance/Truancy—percent of students absent fewer than 15 days • Actual percentage will get folded into the overall adequacy determination • Committee and other stakeholders, informed by data, will have to determine what is “good enough” Commissioner's Performance Based Accountability System Task Force. 1/29/10

9. Level One: Achievement Factors (K-8 & HS) • Status—a weighted (by # of tested students) average of index scores in reading, mathematics, writing, and science • Status Gaps—effect size differences (standardized difference using NECAP scale scores) for reading and mathematics comparing students receiving FRL with those not receiving FRL • What about science and writing? • What about other groups (e.g. , SWD) Commissioner's Performance Based Accountability System Task Force. 1/29/10

10. Level One (K-8): Growth Factors • Student growth percentiles in reading and mathematics • Specific details being worked out by AYP task force • Growth gaps—are FRL students growing at an equally appropriate rate as non-FRL students? • Specific details being worked out by AYP task force Commissioner's Performance Based Accountability System Task Force. 1/29/10

11. Level One: Postsecondary Readiness (HS) • Graduation rate—Using the new federally required graduation rate • Dropout rate—Using NH method for calculating dropout • Other postsecondary measures??? Commissioner's Performance Based Accountability System Task Force. 1/29/10

12. Level One: Putting it together • Aggregate indicators within major classifications, e.g., status, growth, postsecondary • Determine “adequate” for each class of indicators • This allows for more informative feedback • Aggregate these “adequate” determinations across major classifications • Use a profile or related approach to determine overall adequacy for Level One • Obviously, the devil is in the details, but once we get general agreement on the overall plan, we can begin filling in some of the details Commissioner's Performance Based Accountability System Task Force. 1/29/10

13. Level Two: Locally-determined system • A very limited set (e.g., 2-5) of district/school-determined goals, targets, and indicators • For example, “increase the % of students achieving their NWEA growth targets to 90% by 2015” • The school results related to such goals and targets would count in the performance-based accountability system Commissioner's Performance Based Accountability System Task Force. 1/29/10

14. Level Two: Establishing the Goals & Targets • Schools will be required to identify at least two, but no more than five (for these purposes) goals for which they want to be held accountable • Goals must be measureable with identifiable measurement approaches • Goals could (should) stretch over several years, but yearly measureable targets must be established • Committee could require that the goals could be: • Academic only • A mix of academic, social, emotional, and physical, but requiring at least one academic • Any goals that the school chooses that can be measured Commissioner's Performance Based Accountability System Task Force. 1/29/10

15. Level Two: Justifying the Goals • The school must provide a rationale for selecting each individual as well as the set of goals • This rationale should be constructed in terms of a “theory of action,” i.e., a logical flow that describes how focusing (and measuring) this particular goal and associated targets will lead to the ultimate goal of improving student achievement • For example…. • Goal: Increase the number/percent of students taking and passing (earning a 3) AP exams • Target: In 2010, we will increase by 5% the percentage of seniors who have taken at least one AP course with no drop in the percentage of students earning a 3 or better Commissioner's Performance Based Accountability System Task Force. 1/29/10

16. For example (continued)… • Rationale: Our school team has determined that one way to ensure the postsecondary preparation of students leaving our school is to increase the rigor of our HS courses. The AP program provides a vehicle for accomplishing and measuring our goals • Theory of action: • The focus on AP, will lead our school to ensure that our prerequisite course are aligned with AP expectations. • This, in turn, will provide students with more preparation and interest in the particular subjects to increase their likelihood of enrolling in AP. • Counselors ensure that essentially all students are steered toward this academic pathway • The district will support this goal by creating an “AP fund” to pay for tests for students unable to pay Commissioner's Performance Based Accountability System Task Force. 1/29/10

17. Approving the goals • The school will be required to have the goals approved by the district leadership and the board • The school will be required to publish the goals, targets, indicators, and results • NH DOE will have to approve the goals (or just the process by which the goals were established?) and the plans for determining targets and measures Commissioner's Performance Based Accountability System Task Force. 1/29/10

18. Measuring the goals and targets • Setting the yearly targets • Are the targets ambitious enough? • Was the target-setting process appropriate and inclusive? • Measures • Are the measures/indicators appropriate for the purpose? • Are the measures of appropriate technical quality? • E.g., is the measurement error larger than the yearly target? Commissioner's Performance Based Accountability System Task Force. 1/29/10

19. Support and Resources • NH DOE will have to: • Provide guidelines on appropriate process and outcome indicators • Approve the goals and indicators process (but not necessarily the specific goals) • Approve the use of the specific measures • Evaluate the results for each year • We will have to design a process for simplifying this evaluation • Integrate the results of levels one and two • All without any additional resources??? Commissioner's Performance Based Accountability System Task Force. 1/29/10

20. Considerations • Level One—can meet SB 180 requirements • Level Two • Should it be required or left optional? • Theory of action supports requirement • “unfunded mandate” argues for optional • If optional, it all rests on Level One • Combining evaluations from Level One and Two? • Relatively equal weight? • Level two as extra credit or tie-breaker? • Level Two can be a powerful vehicle and incentive for having schools engage in systematic school improvement efforts • Other considerations, concerns, questions, etc? Commissioner's Performance Based Accountability System Task Force. 1/29/10