Demonstrable Improvement for Schools in Receivership

1. Demonstrable Improvement for Schools in Receivership January 9, 2019 Ira Schwartz Associate Commissioner, Office of Accountability

2. Goals for This Presentation Provide overview of Demonstrable Improvement (DI) Process. Describe selection of Demonstrable Improvement indicators. Explain DI Process for current and newly identified Receivership schools.

3. 1. Demonstrable Improvement Process Overview

4. Demonstrable Improvement Process All schools in newly identified for Receivership are designated as Struggling. • No new Persistently Struggling schools. Struggling Schools must annually make Demonstrable Improvement (DI) after a two-year period, and annually thereafter, or they will be placed in Independent Receivership. • Schools that make Demonstrable Improvement continue under Superintendent Receivership. In deciding whether Demonstrable Improvement has been made, Commissioner shall consider: • Performance on Metrics (Indicators). • Number of Years Schools have been Identified. • Superintendent’s Successful use of Receivership Powers to Implement the School’s Plan.

5. What is Demonstrable Improvement? For schools operating under Superintendent Receivership: • Demonstrable Improvement (DI) is the process used to determine whether an Independent Receiver shall be appointed to a school in Superintendent Receivership. • DI does not determine whether a school is removed from Receivership. • Removal from Receivership occurs at the end of the school year in which a school under Superintendent Receivership is removed from Comprehensive Support and Improvement status.

6. DI Methodology: Guiding Principles One or more indicator(s) have been established for each Metric specified in legislation. For each indicator, a school can make progress by achieving a school specific Progress Target. Progress Targets increase over the three-year period. Most indicators are based on student performance – these are classified as Tier 1 indicators. Other indicators are based on implementation of programs and/or processes – these are classified as Tier 2 indicators.

7. DI Methodology: Guiding Principles The State Education Department will select and assign some of the school indicators, and the School Receiver in consultation with the Community Engagement Team shall select some. Selected indicators will be based primarily on where the school’s performance needs the most improvement. A Superintendent Receiver may seek to have local measures approved by the Commissioner. Schools should avoid selecting indicators that measure the same performance as another indicator or indicators that only apply to a small percentage of school’s students. The result of the process shall be a judgment that a) the school made Demonstrable Improvement, b) the school did not make Demonstrable Improvement unless there are shown to be extenuating or extraordinary circumstances, or c) the Commissioner shall review the totality of the school's record to make a determination as to whether or not the school made Demonstrable Improvement.

8. DI Methodology: Revision The existing DI methodology has been utilized from the 2015-16 school through the 2017-18 school year. Beginning with the 2018-19 school year, NYSED has revised the methodology to a) align the DI indicators with the ESSA indicators, b) update and streamline the annual Progress Targets, and c) differentiate the targets for current vs. new Receivership schools. Implementation of the revised methodology is differentiated for schools that are continuing in Receivership, referred to as Cohort 1 Schools, vs. newly identified Receivership schools, referred to as Cohort 2 Schools (discussed in more detail in a separate section). Schools will be assigned and asked to select new indicators based on the revised methodology. However, DI determinations based on the revised methodology will first be made using 2019-20 school year results.

9. 2. Selecting Demonstrable Improvement Indicators

10. Selecting Indicators Schools are measured on indicators selected both by NYSED and the district. There are two indicator types: Level 1: • Indicators that are mostly performance-based (e.g., Performance Index, Growth, Graduation Rate). • Selected by the State based on a school’s baseline performance. • A Level 1 indicator will be assigned when a school performs either below the State Measure of Interim Progress (MIP) or state average for the indicator, whichever is applicable. Level 2: • Indicators selected by the individual school from a state-approved list. • A school may choose a Level 2 indicator if the school performs either below the State Measure of Interim Progress (MIP) or state average for the indicator, whichever is applicable. • Local indicators may be added by districts. All local indicators for Cohort 1 schools must be resubmitted to the Department for approval.

11. Selecting Indicators: Level 1 • A minimum of 5 Level 1 Indicators (7 for schools serving grades 3-12) and a minimum of 5 (or 7) Level 2 Indicators. • Total number of Level 1 indicators is determined by school’s performance on each indicator. • Schools are assigned all Level 1 indicators on which they fall below the current year target for the indicator. • After year 1, if a school’s Level 1 indicator falls below the State MIP or State average, then that indicator will be assigned to the school as an additional Level 1 indicator for the 2019-20 and 2020-21 school years. • If the number of such indicators is less than 5 (or 7 for schools serving grades 3-12): • First, “alternative” Level 1 indicators are assigned. These are indicators where school’s current baseline exceeds 1st year State MIP, but does not exceed 3rd year State MIP. • If multiple “alternative” Level 1 indicators are available, the State will assign the indicator with the largest gap between current performance and the Year 3 MIP. • Second, if total available Level 1 indicators and Level 1 alternative indicators <5 (or <7 for grades 3-12 schools), Level 2 indicators may be used as Level 1 indicators.

12. Selecting Indicators: Level 1 Elementary and Middle Schools 1 Core Subject and Weighted Average Index calculations use the same formula in the numerator. For the denominator, Weighted Academic Achievement uses the greater of: (1) # of continuously enrolled students who are tested or (2) 95% of continuously enrolled students with or with test scores. The Core Subject Performance denominator is the # of continuously enrolled students who are tested. School will be assigned either the Core Subject or Weighted Average Indicator for a subject, not both. 2 Baseline MGP based on 3-year average.

13. Selecting Indicators: Level 1 High Schools

14. Selecting Indicators: Level 2 Schools can select a maximum of 10 Level 2 indicators. Level 2 indicators are categorized into two tiers: • Tier 1 = Focused on student and teacher outcomes. (List on slide 15.) • Many of these indicators are based on accountability subgroups (i.e., English language learners, low-income students, students with disabilities, and racial/ethnic subgroups). • Tier 2 = Focused on organizational and school process changes. • For Cohort 1 Schools these indicators are each weighted at 1% of the Level 2 component of the DI Index. (List on slide 16.) The following Level 2 indicators have been eliminated: • Post-graduation plans of completers (Indicators #83 and #84). • Classes taught by teachers without appropriate certification (Indicator # 87). • Percent of newly hired teachers with state-provided growth ratings of effective or above (Indicator #96).

15. Selecting Indicators: Level 2, Tier 1 † These Level 2 indicators are based on the accountability subgroups. Shading indicates New or Revised indicators.

16. Selecting Indicators: Level 2, Tier 2 Tier 2 Indicators: • Focused on organizational and school process changes. • For Cohort 1 Schools each indicator is weighted at 1% of the Level 2 component of the DI Index. • For Cohort 2 Schools these indicators are weighted in the same way as other Level 2 indicators.

17. Computing the DI Index Schools get credit on an indicator for meeting or exceeding the annual Progress Target. Schools get “full credit” or “no credit” for meeting or not meeting target or goal. No partial or extra credit. DI Index is a weighted average of Level 1 and Level 2 indicators. Level 1 indicators weighted at 50% of the DI Index and Level 2 indicators weighted at 50% of the DI Index, regardless of the number of Level 1 and Level 2 indicators. Each indicator within Level 1 and Level 2 shall be weighted equally (exception is Tier 2 indicators for Cohort 1 Schools). DI Index ranges from 0% to 100%, with cutpoints as follows:

18. Computing the DI Index: Cohort 2 Example

19. Demonstrable Improvement Determinations In this example, because the school’s Demonstrable Improvement Index is at or above 40% but below 67%, the Commissioner reviews the entire performance of the school. After the review, the Commissioner determines the school has made Demonstrable Improvement. Note: Progress targets become more rigorous in Years 2 and 3.

20. Demonstrable Improvement Determinations NYSED will make preliminary determinations for each indicator as to whether school met the progress target. Districts may appeal the preliminary determinations for any indicator. NYSED will use school’s DI Index and review of additional data for schools with DI Index at or above 40% but below 67% to make preliminary determinations. Commissioner will consult with stakeholders who may appeal DI preliminary determination. Commissioner makes final determination as to whether the school made DI.

21. Additional Information Once selected, indicators remain in place for three years. If a determination cannot be made in the current year about an indicator, the weights for that year will be adjusted. If a determination cannot be made in a future year about an indicator, another indicator may be selected. After year 1, if a school’s Level 1 indicator falls below the 2018-19 target, then that indicator will be assigned as an additional Level 1 indicator to the school for 2019-20 and 2020-21 school years. After year 1, the Department will be review the progress targets for 2019-20 and 2020-21 school years. If any are determined to be insufficiently or overly rigorous, the Department reserves the right to adjust them.

22. 3. DI Process for Cohort 1 & Cohort 2 Schools

23. Demonstrable Improvement Cohort 1 Schools: Current Receivership Schools that are identified as CSI schools under the ESSA accountability system. Cohort 2 Schools: Current Priority Schools not in Receivership that are identified as CSI schools under ESSA. Differentiation between Cohorts 1 and 2: • 2018-19 school year determinations. • Indicators available for selection. • Progress targets.

24. DI Determinations for 2018-19: Cohort 1 Cohort 1 Schools: DI Determinations for Cohort 1 Schools will be made using the existing methodology and indicators selected for these schools. New Progress Targets will be determined and provided to these schools. • If necessary, changes may be made to DI indicators assigned to these schools in a manner consistent with current methodology. Additionally, Cohort 1 Schools will be assigned new Level 1 indicators and will have the opportunity to select new Level 2 indicators based on the revised methodology for DI determinations beginning with the 2019-20 school year.

25. DI Determinations for 2018-19: Cohort 2 Cohort 2 Schools: DI Determinations for Cohort 2 Schools will not be made for the 2018-19 school year. Schools will receive a DI index for informational purposes only. Schools will be assigned new Level 1 indicators and will have the opportunity to select new Level 2 indicators based on the revised methodology for DI determinations beginning with the 2019-20 school year. Schools that select a DTSDE indicator will not receive a determination for that indicator in the 2018-19 school year.

26. DI Determinations for 2019-20 and 2020-21 Cohort 1 and Cohort 2 Schools: DI Determinations for both Cohorts will be made using the revised methodology and the new indicators selected for these schools. More rigorous Progress Targets will be used in each subsequent year.

27. How a School Exits Receivership Cohort 1 and Cohort 2 Schools: Schools will be removed from Receivership at the end of the school year in which the school is not identified as a Comprehensive Support and Improvement School. The first opportunity for removal will be as of June 2021 based on 2018-19 and 2019-20 school year results. Making DI does not determine whether a school is placed into or removed from Receivership. Other Schools: For current schools in Receivership that are not identified as CSI based on 2017-18 school year results, removal will occur at the end of the 2018-19 school year.

28. Timeline for Indicator Selection Cohort 1 Schools: For the 2018-19 school year, schools will be assigned Level 1 indicators and will be allowed to make revisions to Level 2 indicators with applicable goals and targets no later than for 4 weeks following notification regarding indicators, using the existing methodology. Cohort 1 and Cohort 2 Schools: For 2019-20 and subsequent school years, schools will be assigned Level 1 indicators and allowed to select Level 2 indicators for use for with applicable targets no later than 6 weeksfollowing identification.

29. Next Steps Participate in NYSED Webinar for schools, which will occur once information on assigned Level 1 indicators and eligible Level 2 indicators have been provided to districts. Discuss selection of Demonstrable Indicators with the school’s Community Engagement Team. Submit the file with the selected indicators to:accountinfo@nysed.gov. Call (718) 722-4553 or send an e-mail toaccountinfo@nysed.govfor technical assistance. Submit questions to: accountinfo@nysed.gov.

30. Thank You

31. Appendix: Progress Targets for New Indicators Level 1 Indicators: Level 1 Indicators are automatically assigned to the school by NYSED if the school’s baseline falls below the State Measure of Interim Progress (MIP) for that indicator (if a State MIP is available) or below the State Average (if a State MIP is not available). Level 2 Indicators: Schools have autonomy to select Level 2 Indicators. However, the Indicators must be eligible for selection. An Indicator is eligible for selection if the State MIP is available and the school baseline falls below the State MIP or, if State MIP is not available, the school baseline falls below the state average for that indicator. Cohort 1 schools have more rigorous progress Targets than Cohort 2 schools for Level 2 indicators where State MIP is not available.

32. Progress Targets: Level 1 Indicators Level 1 Indicators:

33. Progress Targets: Level 2 Indicators Level 2 Indicators:

34. Progress Targets: Comparison with Measures of Interim Progress Example of Expected Annual DI Progress for Indicators for which End Goals are Available: Compared with MIP Calculations.