Accountability & Assistance Advisory Council Meeting

1. Accountability & Assistance Advisory Council Meeting May 8, 2018 Best Western Royal Plaza Hotel Marlborough, MA

2. AGENDA Welcome & introductions Review of discussion norms Weighting of accountability indicators Proposed amendments to state accountability regulations Update on assistance model Revising DESE’s District Standards & Indicators Next Steps

3. Welcome & introductions 01

4. Welcome & introductions • Council co-chairs: Meg Mayo-Brown & Valerie Annear

5. Review of discussion norms 02

6. Norms & protocols • Promote risk-taking by not assigning specific comments to individual members in meeting summaries & minutes • Maintain respect before, during, & after meetings • Seek clarification & check understanding to avoid mistakenly attributing ideas to an individual or organization represented on the council • Keep improvements in student learning at the core of the discussions. Students should drive the conversation • Stay engaged in the issues • Actively address implicit biases & instill cultural proficiency in discussions • Encourage discussion from all voices of members. Be additive, not repetitive • Let members know if pre-reading or other pre-meeting assignments require more than 30 minutes of preparation time • Make it fun!

7. Weighting of accountability indicators 03

8. System highlights • Additional accountability indicators • Provide information about school performance & student opportunities beyond test scores • Normative & criterion-referenced components • Accountability percentiles & progress toward targets • Focus on raising the performance of each school's lowest performing students • In addition to the performance of the school as a whole • Discontinuation of accountability & assistance levels 1-5 • Replaced with accountability categories that define the progress that schools are making & the type of support they may receive from the Department • Districts classified based on district-level data • No longer based on the performance of a districts lowest performing school

9. Massachusetts’ accountability indicators – non-high schools

10. Massachusetts’ accountability indicators – high schools

11. Weighting decisions to be made • Three weighting decisions to be made • Weighting of indicator categories in the school percentile calculation • Weighting of “meeting target” points • Weighting of “all students” vs. “lowest performing” students

12. Considerations for weighting achievement & growth • The current ratio of achievement & growth is 3 (achievement) to 1 (growth) • Impact of increasing weight of growth in system: • Could increase differentiation between similarly-achieving schools • Increases the value of a normative measure (there will always be a 1st percentile & a 99th percentile) • Decreases the value of grade 3 assessment results (no SGP for students in grade 3) • Decreases value of science assessment in system (no SGP for science)

13. Considerations for weighting achievement & growth • All indicators need to be included in the weighting • Progress towards English language proficiency only applies to a subset of schools, & weighting needs to be flexible • Ratio between achievement & growth can be held constant between non-high schools & high schools but actual weightings will differ • ESE intends to apply the same weighting rules to both the normative & criterion-referenced components of the system • Recommendation is to maintain current ratio (3:1) • Board to make final decision this spring

14. Proposed weighting of indicators in non-high schools

15. Proposed weighting of indicators in high schools & middle/high/K-12 schools

16. Normative component • Accountability percentile 1-99, calculated using all available indicators for a school • Compares schools administering similar statewide assessments • Used to identify the lowest performing schools in the state • Same calculation used at the subgroup level to identify low-performing subgroups

17. Criterion-referenced component • Focus on closing the achievement gap by raising the “achievement floor” • Gap-closing can occur as a result of a decline in performance by the high-performing group • In addition to meeting targets for the school as a whole, the performance of the lowest performing students in each school will be measured • Every school has a group of lowest performers • Identified from cohort of students who were enrolled in the school for more than one year

18. Criterion-referenced component • Targets set for each accountability indicator, for the school as a whole & for the lowest performing students in each school *Minimum group size for each indicator = 20 students

19. Criterion-referenced component • Points assigned based on progress toward target for each indicator, for both the aggregate & the lowest performing students • Proposed model has equal intervals between categories of performance against meeting targets • Recommendation is to maintain these intervals until multiple years of data allow for impact analysis of unequal weighting of points

20. Criterion-referenced component calculation – non-high school

21. Criterion-referenced component calculation – high school

22. Categorization of schools • Schools will no longer be placed in a vertical hierarchy of levels 1-5 • Number of schools that will be placed into a category based upon a relative standing will be cut in half from previous system • Approximately 90 percent of schools could be categorized based on their own performance against targets • Most schools will have 50 percent of its categorization based on students that have been in the school for at least two years • Category labels are primarily tied to the level of required assistance or intervention • Stronger emphasis on schools commended for success

23. Categorization of schools • Notes: • Category names not finalized • School percentiles & performance against targets will be reported for all schools

24. Categorization of districts • Districts will be classified based on the performance of the district as a whole • No longer categorized based on performance of lowest performing school • District accountability percentiles will not be calculated • Classified based on criterion-referenced component • Adjustments made for low graduation rates & low assessment participation • Board may designate a district as underperforming or chronically underperforming

25. Categorization of districts • Notes: • Category names not finalized • Performance against targets will be reported for all districts

26. Accountability reports • Accountability reports published for each district & school (fall 2018) • Reports will include: • Overall classification • Including reason(s) for classification (e.g., low graduation rate, low-performing subgroup) • Criterion-referenced target percentage • Accountability percentile (schools only) • Data related to performance on each accountability indicator for each subgroup meeting the minimum group size (20 students) • All students • Lowest-performing students • High needs students • English learners • Students with disabilities • Economically disadvantaged students • Major racial/ethnic subgroups

27. Proposed amendments to state accountability regulations 04

28. State accountability regulations • 603 CMR 2.00 – Accountability & Assistance for School Districts & Schools • Current regulations describe the process & measures used to place schools & districts into accountability & assistance levels 1-5 • Do not align with Massachusetts’ federally approved Every Students Succeeds Act (ESSA) state plan • Do no reflect changes in how we report MCAS data • Next-Generation MCAS achievement results reported using a different measure than the legacy MCAS tests

29. State accountability regulations • Proposed amendments align the regulations to our approved ESSA state plan • Describe the general framework of the accountability system & the assistance provided to schools & districts identified as needing support from the Department • Clarify the Board of Elementary & Secondary Education’s role in reviewing & approving the system

30. ESE Turnaround Assistance Redesign: New Opportunities 2018 & beyond 05

31. Current Model • Focusing on L3, L4, & L5 districts with 42% of the state’s students

32. Redesigned Regional Support Structure Features Vision: All students in the Commonwealth, especially the historically marginalized, learn in dynamic and responsive educational environments, graduate with the tools and agency to achieve their visions for success, and contribute as responsible members of their communities.

33. Redesigned Regional Support Structure Staffing • Provide content support, in partnership with assistance leads, to schools and districts around math, literacy, ELL, SWD, SEL, and educator effectiveness. • Support regional teams in vetting and selecting content-focused partners to support schools and districts.

34. Discussion Question How should the redesign Statewide System of Success measure its impact? What outcomes should we be paying attention to as we measure our effectiveness?

35. Revising DESE’s District Standards & Indicators 06

36. CONTENTS Overview of DESE’s District Standards & Indicators Revision process Discussion

37. Overview: District Standards & Indicators

38. What they are and how they are used • DESE’s definition of effective district policy and practice • Incorporate DESE’s expectations for schools • Framework for district reviews • Guidance for district reflection and planning

39. Revision process

40. Guiding principles • Same 6 standards with revised/updated indicators • Goals for revision: • High-leverage • Explicit focus on equity • Aligned with current ESE priorities/expectations • Realistic/practical • Succinct • Coherent/consistent • Flexibility as to extent of revisions

41. Gathering feedback • To date: • Initial revisions & iterative feedback • Input from across & beyond DESE (standard-specific) • Going forward: • Broader review of complete document • Research focus

42. Discussion

43. For your consideration… • What are your recommendations for improving/clarifying the Standards & Indicators? • What are some ways that the Standards & Indicators (and/or supporting documents) might be useful for districts? • How can we better promote districts’ use of the Standards & Indicators?

44. Next steps 07