Accountability: Where Are We Going?

1. Accountability: Where Are We Going? Martha Musser, Coordinator NYS Education Department

2. This may be the Dawning of the Age of Accountability • Contract for Excellence • NCLB Reauthorization • NYC and other local accountability initiatives

3. Contract for Excellence • Governor has proposed a Contract for Excellence that calls for an enhanced State accountability system, including: • New accountability standards based on State assessments and other indicators of progress, such as graduation rates or college attendance and completion rates. • Growth model by 2008-09. • Value-added model by 2010-2011 based on new or revised state assessments. • Expanded SURR system, resulting in the identification of up to 5% of State school’s by 2011-2012 for restructuring or reorganization.

4. Contract for Excellence: Plans for Intervention • School Review Teams conduct resource, program and planning audits of SINI and SRAP schools and assist all SINI and SRAP schools in development of improvement plans. • Joint School Intervention Teams, whose members are either appointed by Commissioner or educators from the district, review and recommend plans for reorganizing or reconfiguring schools that are to be closed. • Distinguished Educators assist low-performing schools and districts. • The services of all the above are a charge to the school district.

5. Contract for Excellence: Now it’s Personal • Commissioner shall define in regulation deficient district performance. • School districts hiring a superintendent after the effective date of regulation must include a provision in superintendent’s contract that after two years of deficient district performance the superintendent will fully cooperate with a distinguished educator. • After four years of deficient performance, school board must seek to remove a superintendent or provide a rationale to the Commissioner for why board should not take such action. • After six years of deficient district performance, the Commissioner shall commence action to remove the board or board members.

6. Distinguished Educators and Deficient District Performance • Consult with Commissioner on removal of superintendent. • Serve as ex officio, non-voting members of Board of Education. • Review all school and district improvement plans and either endorse or make recommendations to the Board of Education for change. • Boards of Education must make changes as required by Distinguished Educators unless upon petition by Board of Education Commissioner decides there is a compelling reason not to implement the modification.

7. Status, Growth and Value- Added Models Defined Contract for Excellence requires SED to implement growth and value-added models and meet any Federal requirements for such models.

8. Status Models Defined Status or improvement models, the current requirement for measuring Adequate Yearly Progress (AYP) under No Child Left Behind, measure progress by tracking the improvement at the same grade levels within the school over time. (This year’s grades 3-5 compared to last year’s grades 3-5.)

9. Growth Models Defined Growth models generally refer to accountability models that assess the progress of a cohort of individual students over time with the intent of measuring the progress these students have made (Performance in fourth grade compared to performance in third grade).

10. Value-Added Models Defined Value-added models generally refer to a specific type of growth model in which student demographic data or other statistical controls are used to attempt to analyze the specific effects of a particular school, program, or teacher on student learning. These models ask whether the school has increased the measured achievement of students more than expected based on data from similar schools.

11. Purpose of Status, Growth, and Value Added Models • Status model: Determine whether an increasing percentage of students are gaining proficiency over time. • Growth model: Determine how much progress groups of students are making over time. • Value-added model: Determine success of schools, programs, or teachers by measuring student growth over time while controlling for non-school variables that impact on student performance.

12. Value-Added Models: The Holy Grail of Accountability? • Much controversy about value-added models. • Some claim that good value-added models exist (Sanders), statistical models not based on vertical scales. • Some claim that the state of the psychometric art does not currently support vertical scales. • Value-added models are not easy: • Less sophistication is required to make accurate determinations about status then growth. • Acquisition of knowledge is not linear. • Value-added models are heavily dependent upon the robustness of their assumptions. • Value-added models break down more quickly as groups become smaller.

13. Growth Model System Requirements Growth Model System Requirements: • Annual assessments in successive grades • A unique student identification system • Vertically aligned or scaled assessments • Grade by grade standards • A minimum of two years of assessment data per student • Value-added systems require additional data on those demographic or resource factors for which the model seeks to control

14. NY and Growth Models • NYS Grades 3-8 Testing Program uses Vertically Moderated Standards (VMS) • Student progress is measured from grade-to-grade relative to proficiency in meeting the standards (rather than in terms of change in scale scores). • VMS is applicable to some but not all growth model designs. • NY will by the end of 2006-07 have only two years of test data, limiting our ability to create growth trajectories for individual students. • SED’s goal is to select a growth model architecture by April.

15. NCLB: Status Report • Revised State standards and Annual Measurable Objective’s (AMOs) established. • 05-06 Accountability Decisions Released. • Development of elementary/middle level attendance standards in progress. • Regents continuing deliberations regarding raising high school graduation standards. • Regents engaging in NCLB reauthorization advocacy.

16. State Standards State Standards for 06-07 are: • 155 for Grade 3-8 ELA and math • 165 for HS ELA and math State Standards for 07-8 are: • 160 for Grade 3-8 ELA and math • 170 for HS ELA and math State standard is used to determine which schools must do Local Assistance Plans (LAPs) and which schools and districts may be high performing.

17. Original Annual Measurable Objectives for2002–03 to 2013–14 School Year Elementary-Level Middle-Level Secondary-Level ELA Math ELA Math ELA Math 2002-03 123 136 107 81 142 132 2003-04 123 136 107 81 142 132 2004–05 131 142 116 93 148 139 2005–06 138 149 126 105 154 146 2006–07 146 155 135 117 159 152 2007–08 154 162 144 129 165 159 2008–09 162 168 154 141 171 166 2009–10 169 174 163 152 177 173 2010–11 177 181 172 164 183 180 2011–12 185 187 181 176 188 186 2012–13 192 194 191 188 194 193 2013–14 200 200 200 200 200 200

18. Revised Annual Measurable Objectives for2005–06 to 2013–14 School Year Elementary & Middle-Level Secondary-Level ELA Math ELA Math 2005–06 122 86 154 146 2006–07 122 86 159 152 2007–08 133 102 165 159 2008–09 144 119 171 166 2009–10 155 135 177 173 2010–11 167 151 183 180 2011–12 178 167 188 186 2012–13 189 183 194 193 2013–14 200 200 200 200

19. Calculating AMOs • In 2000-2001 and 2001-2002 combined, 20% of NY’s public school students were enrolled in school’s with PI’s below: • Grade 4 ELA: 123 • Grade 8 ELA: 107 • Grade 4 Math: 136 • Grade 8 Math: 81 • In 2004-05, the percent of students below the 2004-2005 AMO’s were: • Grade 4 and 8 ELA combined: 13.3% • Grade 4 and 8 math combined: 4.2% • 2005-2006 Grade 3-8 ELA results: • AMO at 13.3% = 122 • AMO at 20.0% = 133 • 2005-2006 Grade 3-8 math results: • AMO at 4.2% = 86 • AMO at 20.0% = 132

20. Limited English Proficient (LEP) Students • All LEP students in grade K–12 must take the NYSESLAT annually. • LEP students in grades 3 through 8 enrolled in U.S. schools (not including Puerto Rico) for less than one year (enrolled on or after January 2, 2006) were not required to take the NYSTP ELA assessment in January 2007. For such students who did not take the ELA assessment, valid scores on the NYSESLAT Reading/Writing and Speaking/Listening components will meet the ELA participation requirement. • The eligible LEP students must be identified in the repository using Program Service 0242. • NYSESLAT performance levels will not be used in calculating the Performance Index. LEP students meeting the criteria to use the NYSESLAT in lieu of the ELA will not be included in the Performance Index calculation.

21. 2002 Graduation-Rate Cohort Definition • This cohort will be used to determine if the district or school meets the graduation-rate requirements for the 2006–07 school year. The 2002 graduation-rate cohort consists of all students, regardless of their current grade status, who were enrolled in the school on October 6, 2005 (BEDS day) and met one of the following conditions: • first entered grade 9 (anywhere) during the 2002–03 school year (July 1, 2002 through June 30, 2003); or • in the case of ungraded students with disabilities, reached their seventeenth birthday during the 2002–03 school year.

22. 2002 Graduation-Rate Cohort Definition (cont’d) • The State will exclude the following students when reporting data on the 2002 cohort: • students who transferred to another high school (excluded from the high school graduation-rate cohort) or district (excluded from the district graduation-rate cohort) or criminal justice facility after BEDS day 2005; • students who transferred to an approved alternative high school equivalency preparation (AHSEP) or high school equivalency preparation (HSEP) program (CR 100.7) after BEDS day 2005 and met the conditions stated on the next slide; • students who left the U.S. and its territories after BEDS day 2005 and before August 30, 2006; and • students who died after BEDS day 2005 and before August 30, 2006.

23. 2002 Graduation-Rate Cohort(Transfers to GEDRemoved from Cohort) Students will be removed from the cohort for the school and district from which they transferred to an AHSEP or HSEP program if the final enrollment record shows that on June 30, 2006 the student a) earned a high school equivalency diploma; or b) was enrolled in an AHSEP or HSEP program. Students will be removed from the school cohort if the enrollment records showed that the student transferred to a different high school and was working toward or earned a high school diploma. Students will be removed from the district cohort if the enrollment records show that the student transferred to a high school in a differentdistrict and was working toward or earned a high school diploma.

24. 2003 Graduation-Rate Cohort Beginning with the 2003 graduation-rate cohort (used for accountability in 2007-08): • students are included in the cohort based on the year they first enter grade 9 (or for ungraded students, the year they turn 17). • students who have spent at least five months in a district/school during year 1, 2, 3, or 4 of high school are part of the district/school cohort unless they transfer to another diploma-granting program.

25. Inclusion Rules for the 2003 Graduation-Rate Cohort A student will be included in the district/school cohort if the student’s last enrollment record in the district or school shows: • that the student was enrolled for at least five continuous (not including July and August) months and the ending reason was not one of the following: transferred to another New York State district or school, died, transferred by court order, or left the U.S. • fewer than five month’s enrollment and an ending reason indicating that the student dropped out or transferred to a GED program and the student’s previous enrollment record in that district/school (assuming one exists): • indicates that the student dropped out or transferred to a GED program, and • that the student was enrolled in the district/school for at least five months.

27. Student Enrollment by Accountability Status

28. Statewide Status of SINI and SRAP Schools 2005-06 vs. 2006-07 Note: A school’s status would improve or stay the same if the school made AYP in the area(s) of identification which gave the school its status. A school’s status becomes worse if it fails to make AYP in one or more of the area of identification which gave the school its status.

30. NCLB Reauthorization: Regents have developed positions regarding: • Schools and districts with multiple federal designations • Growth Models • Targeted interventions • AYP Methodologies • Additional time to meet graduation standards • ELL testing issues • Implementation of choice and SES • Safe Schools

31. NCLB Reauthorization: What we are Hearing • Serious discussions about NCLB are beginning but reauthorization more likely to occur in 2009 than 2007, very unlikely to occur in 2008. • Key issues being discussed: • Full funding • Growth models • N size and confidence intervals • Required interventions, including SES and choice • Assessments: LEP, SWD, high school • Highly Qualified Teachers vs. High Quality Teaching • National Standards • Science Assessments

32. Other Accountability Initiatives: NYC • Children First: • Valued Added Model. • More Ragu than KISS. • Each school receives letter grade (A-F) on school progress report. • Progress report consists of four categories: • School Environment (15%) • Student Performance (30%) • Student Progress (55%) • Additional Credit

33. Children First Progress Report • School’s grade is based upon how well the school performs relative to other City schools (1/3 of grade) and to a peer group of approximately 40 schools with similar demographics (2/3 of grade). • State assessments used to measure performance of students over time. • Additional credit given to improved performance among low achieving students and various disaggregated groups. • Factors such as parent, student, and teacher surveys; credit accumulation; PSAT scores used in determining school grade.

34. Challenges for SED • Growing Pains • Testing Times • Vision 2020 • A Cart Before the Horse?

35. Challenges Ahead for School Districts Immediate: • Single Grade 3-8 Performance Index makes schools and districts responsible for more disaggregated groups. • New standard setting for grade 3-8 assessments may challenge middle schools even more. • Changes in testing practices for LEP students require that bilingual education programs emphasize rapid acquisition of English as well as fluency in the native language. • New graduation standards will raise expectations for middle schools and force high schools to reveal “hidden students.” Longer term: • If Governor and Legislature show school districts the money, they are also going to demand that districts show results.

36. Challenges Ahead: Lists, Lists, Lists How do we make all of these work together: • SURR • SINI, DINI • SRAP, DRAP • IDEA Districts • Title III AMAO’s • Persistently Dangerous Schools • High School Initiative • Highly Qualified Teachers

37. This may be the Dawning of the Age of Accountability Key Questions: • How do we design accountability models that compel movement from awful to adequate without impeding the movement from good to great? • How do we move from beating the odds to changing the odds?

38. More Information Ira Schwartz, Coordinator Office of School Improvement and Community Services (NYC) ischwart@mail.nysed.gov 718 722-2796 Accountability PowerPoint for 2006-07: http://www.emsc.nysed.gov/irts/accountability/2005-06/accountability-rules-Nov2006_files/frame.htm Manuals for NYS Student Identifier System and the Repository System: http://www.emsc.nysed.gov/irts/SIRS/home.shtml