Top-to-Bottom Ranking & Priority/Focus/Reward Designations

1. Top-to-Bottom Ranking &Priority/Focus/RewardDesignations Understanding the

2. Why the TTB Ranking? • Move from metric as designation only (“stick”) to leveraging the metric as a diagnostic tool for schools • Resist urge for “more data” until we understand the metrics available; avoids “dying in data” • Focus of this presentation: • Overview of calculations • How to interpret results (for schools and districts)

3. Breaking it Down

4. The Top-to-Bottom List • Statewide ranking of ALL schools that meet ranking criteria • Bottom 5% = Priority (PLA) Schools • 10% schools with largest achievement gaps = Focus Schools • Also used for Reward School status: • Use top 5% from overall ranking = Reward Schools • Use top 5% improving schools = Reward Schools • Use Beating the Odds Schools = Reward Schools

5. Top to Bottom(TTB) Ranking • Three main components by subject: • Achievement • Improvement in achievement over time • Achievement gap between the top scoring 30% of students versus the bottom scoring 30% of students • Each component tells schools something about their overall performance and can be used for diagnostics

6. TTB Ranking • Graduation rates are included in the statewide Top-to-Bottom Ranking. • Schools with a graduation rate have it included in the following two ways: • Graduation Rate • Improvement in graduation rate over time

7. Who Receives a Ranking? • Schools with 30+ full academic year (FAY) students over the last two years in at least two state-tested content areas; school must be OPEN at time of list generation • Application  Some schools do not receive a ranking if they: • Have too few FAY students • Only have one year of data

8. Tested Grades & Subjects • Reading and Mathematics: Grades 3-8 and 11 • In grades 3-8, testing every year allows us to figure out student performance level change (our current “growth” metric) in reading and math • Students can either significantly improve, improve, maintain, decline or significantly decline • Writing: Grades 4 & 7 • Science: Grades 5 & 8 • Social Studies: Grades 6 & 9

9. What About Reconfigured Schools? • A school must change by four or more grades in order to get a new code • Example: A K-2 building becoming a K-6 building. • New codes NOT granted when a school is reopened as a charter, for example • If not, the school retains the old code and continues to have data “point” at it from all students for whom that code is their feeder school • There is no “phase reset” like there was in AYP • If school population changed by 51%, could request a phase reset—still got AYP calculations, but sanctions delayed • Under Priority/Focus interventions, would simply have a customized intervention.

10. How Is the Top to Bottom Ranking Calculated • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

11. How do we get Standardized Scale Scores for Each Student? • Step #1: Take each student’s score on the test they took and compare that score to the statewide average for students who took that same test in the same grade and year • This creates a student-level z-score for each student in each content area • Compare • MEAP to MEAP • MEAP-Access to MEAP-Access • MME to MME • MI-Access • Participation to Participation • Supported Independence to Supported Independence • Functional Independence to Functional Independence

12. What do we do with those Standardized Scores? • Step #2: Once each student has a z-score for each content area (based on the test they took), we take all of the students in a each school, and rank order the students within the school. • Z-scores will have come from different tests, and compare students to statewide average for that grade, test, and subject • But they can now be combined for the school • Step #3: Add up all z-scores and take the average. This is now the average standardized student scale score. • Step #4: Define the top and bottom 30% subgroups, based on that rank ordering.

14. Average Z-score (average standardized student scale score): 0.28 (sum all z-scores, divide by 15)

15. Top 30% Bottom 30%

16. How Is the Top to Bottom Ranking Calculated? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

17. What is Important to Show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Step #1: Achievement How well did the school do in that subject? Positive number = better than average Near zero = average Negative number = worse than average Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

18. What is Important to Show Schools? Step #2: Improvement Is the school improving in that subject? Positive number = greater rate of improvement than average Near zero = average improvement Negative = slower rate of improvement than average; can also mean they are declining • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

19. What is Important to Show Schools? Raw value is also meaningful: Positive number: More students improving than declining Negative number: More students declining than improving • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

20. What is Important to Show Schools? Step #3: Achievement Gap Is the gap in that subject between top 30% and bottom 30%: (positive number) = smaller gap than average (negative number) = larger gap than average (near zero) = average gap • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

21. Once you have Looked at each Component, Discuss: • What’s the overall pattern? • Low achievement? • Declining achievement? • Large gaps? • Where are the actionable areas? • Which subjects need the most attention? • Is everyone doing poorly (small gap, low achievement) or are some students doing well and others falling behind (decent achievement, but large gap)

22. What is Important to Show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Reward Schools (for improvement) Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Focus Schools Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

23. Weighted Performance Level Change (PLC) • A weighted composite of individual student performance level change is used to calculate improvement in grades 3-8 reading and mathematics • Rewards large improvements more heavily, rewards maintenance of proficiency if a student was already proficient

24. How is the Top-to Bottom Ranking Calculated? • For science, social studies, writing, and grade 11 all tested subjects Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Four-Year Achievement Trend Slope School Performance Achievement Trend Z-Score School Content Area Index Content Index Z-score 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

25. How is the Top-to-Bottom Ranking Calculated? • For graduation rate Two-Year Average Graduation Rate School Graduation Rate Z-Score 2/3 Four-Year Graduation Rate Trend Slope School Graduation Rate Trend Z-Score School Graduation Rate Index Grad Index Z-score 1/3

26. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

27. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

28. How is the Top-to-Bottom Ranking Calculated? • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate) Slope = 2.3%

29. How is the Top-to-Bottom Ranking Calculated? • Calc an overall rank for a school with a grad rate School Mathematics Std Index 18% School Reading Std Index 18% Overall Standardized School Index Overall School Percentile Rank School Science Std Index 18% School Social Studies Std Index 18% School Writing Std Index 18% School Graduation Rate Std Index 10%

30. How is the Top-to-Bottom Ranking Calculated? • Calculating an overall ranking for a school without a graduation rate School Mathematics Std Index 20% Look at each subject index. Help schools understand which subjects are strong/weak for them. Positive number: better than average Negative number: below average Near zero: near average School Reading Std Index 20% Overall School Standardized Index Overall School Percentile Rank School Science Std Index 20% School Social Studies Std Index 20% School Writing Std Index 20%

31. How is the Top-to-Bottom Ranking Calculated? • Calculating an overall ranking for a school without a graduation rate and without a writing score School Mathematics Index 25% School Reading Index 25% Overall School Standardized Index Overall School Percentile Rank School Science Index 25% School Social Studies Index 25%

32. Which years of data are in the ranking?

33. For elementary and middle schools • Michigan tests in the fall. • These fall tests reflect the learning of students in the previous school year. SY 2008-2009 SY 2009-2010 SY 2010-2011 Fall 2010 Testing Fall 2008 Testing Fall 2009 Testing Fall 2011 Testing

34. For High Schools • Michigan tests in the spring • The spring test (MME and MI-Access) measures what students have learned from grades 9, 10 and grade 11 prior to the MME testing.

35. What do the 2012 Rankings reflect? • For elementary/middle schools: • Performance on the MEAP and MI-Access tests in fall 2011 (which represents learning from school year 2010-2011) and before • For high schools: • Performance on the MME and MI-Access tests in spring 2012 (which represents learning from school year 2011-2012 prior to testing) and before.

36. Resources Available • Complete list of all schools and their ranking • Individual school look-up to see your school’s results • Overview presentation with voice over • FAQ • Business rules by which the rankings were calculated • Complete data file and validation file You can access these resources at www.mi.gov/ttb You can also request individual assistance by calling the Evaluation, Research and Accountability unit at 517-373-1342, or emailing mde-accountability@michigan.gov

37. Priority Schools Identifying

38. Schools in the bottom 5% of the Top to Bottom ranking. • MDE will ensure that the number of schools identified as Priority schools is equal to at least 5% of the state’s Title I schools as Priority schools. • In addition to the bottom 5%, MDE will also add any school with a graduation rate of less than 60% for three consecutive years, and any Tier I or Tier II school using SIG funds to implement a turnaround model. What are priority schools?

39. Based on the Top-to-Bottom (TTB) ranking methodology which includes data from • Achievement results • Improvement results • Achievement gap results • All tested subjects are included in the ranking where possible for a school. • A ranking is determined for all open schools with 2 years of data for • 30 or more students • two or more tested subjects How are they determined?

40. For a school to exit priority school status, it has to receive a Green, Lime, Yellow or Orange on the Accountability Scorecard at the close of its third year in the priority school intervention. • A school must either meet aggressive proficiency targets (toward 85% of students proficient by 2022), or must have demonstrated significant improvement. • The proficiency and/or improvement gains must be demonstrated all nine traditional ESEA subgroups as well as in the new “bottom 30%” subgroup. How does a school EXIT Priority School status?

41. This means that a priority school who achieves a Green, Lime, Yellow or Orange on the Accountability Scorecard and exits priority status has: • Met all interim measurements of progress for priority schools (approved plan, leading and lagging indicators). • Met proficiency and/or improvement targets on average as a school. • Increased the proficiency rate of all traditional subgroups • Increased the proficiency rate of their very lowest performing students. How does a school EXIT Priority School status?

42. Focus Schools Identifying

43. What is a Focus School? • Schools with the largest achievement gaps in scale score between the top 30% of students and bottom 30% of students within a school • Focus schools may have high average performance overall, but have a significantly large gap, suggesting struggles addressing low achieving students

44. What is the purpose of Focus School designation? • Identifying Focus Schools is a critical component to closing achievement gaps within schools and statewide • Focus schools highlight where changes in teaching and learning practices need to be undertaken to respond to the learning needs of low-achieving students. • These changes are difficult, and both accountability and support need to be differentiated.

45. How was our school identified as a Focus School? • Top-to-Bottom list includes a component that examines the gap in achievement scores between top 30% and bottom 30% of students within a school • Gaps are standardized between all students using a common assessment within a school, and then averaged for the school • Gaps are calculated for all subject areas and combined

46. How were Focus schools identified (continued)? • Average gap is then standardized and ranked among all schools • 10% with largest gaps are identified as Focus Schools

47. Common concerns about this metric Z-scores compare schools • Are Focus schools only high-achieving schools? • Are Focus schools only high socioeconomic status schools? • Is the bottom 30% subgroup in Focus schools actually high performing? • Are schools more likely to be Focus schools if they have [fill in the blank group] kids?

48. Not just high achieving...

49. Not just high (or low) socioeconomic...

50. Bottom 30% are not high achieving ... Across all subject areas and grade levels, the bottom 30% subgroup consistently had average achievement z-score below zero, and most of them are between -0.5 and -1.5. (This example: E/MS Reading) Top 30% Bottom 30%