2012 Top to Bottom Ranking

1. 2012 Top to Bottom Ranking Understanding How the Ranking is Calculated

2. Why use the TTB Diagnostically? • Move from metric as designation only (“stick”) to leveraging the metric as a diagnostic tool for schools • Resist urge for “more data” until we understand the metrics available; avoids “dying in data” • Focus of this presentation: • Overview of calculations • How to interpret results (for schools and districts)

3. Top to Bottom List • Statewide ranking of ALL schools • Bottom 5% = Priority/PLA schools (starting SY 2012-2013) • Also used for Focus and Reward: • Use achievement gap component only  Focus schools • Use top 5% overall  Reward schools • Use improvement component only  Reward schools

4. Top to Bottom (TTB) Ranking • Three main components by subject: • Achievement • Improvement in achievement over time • The largest achievement gap between two subgroups calculated based on the top scoring 30% of students versus the bottom scoring 30% of students Each component tells schools something about their overall performance and can be used for diagnostics

5. TTB Ranking • In addition to the achievement components, student graduation is included in the statewide top-to-bottom ranking for schools with a graduation rate in the two following ways: • Graduation Rate • Improvement in graduation rate over time

6. Who receives a ranking? Schools with 30+ full academic year (FAY) students over the last two years in at least two state-tested content areas; school must be OPEN at time of list generation Application  Some schools do not receive a ranking if they: • Have too few FAY students • Only have one year of data

7. Tested Grades and Subjects • Reading and Mathematics: Grades 3-8 and 11 • In grades 3-8, testing every year allows us to figure out student performance level change (our current “growth” metric) in reading and math • Students can either significantly improve, improve, maintain, decline or significantly decline • Writing: Grades 4 & 7 • Science: Grades 5 & 8 • Social Studies: Grades 6 & 9

8. What about Reconfigured schools? • A school must change by four or more grades in order to get a new code • Example: A K-2 building becoming a K-6 building. • New codes NOT granted when a school is reopened as a charter, for example • If not, the school retains the old code and continues to have data “point” at it from all students for whom that code is their feeder school • There is no “phase reset” like there was in AYP • If school population changed by 51%, could request a phase reset—still got AYP calculations, but sanctions delayed • Under Priority/Focus interventions, would simply have a customized intervention.

9. What is a Z-Score? Quick Reference for Z-Scores

10. Why do We Use Z Scores? • Z-scores are a standardized measure that helps you compare individual student (or school) data to the state average data (average scores across populations). • Z-scores allow us to “level the playing field” across grade levels and subjects • Each Z-score corresponds to a value in a normal distribution. A Z-Score will describe how much a value deviates from the mean. What do you need to know: Z-scores are used throughout the ranking to compare a school’s value on a certain component to the average value across all schools.

11. Z-Score “Cheat Sheet” Student z-score = (Student Scale Score) – (Statewide average of scale scores) Standard Deviation of Scale Score School z-score= (School Value) – (Statewide average of that value) Standard deviation of that value Z Score Summary PowerPoint and Business Rules- http://www.michigan.gov/mde/0,4615,7-140-37818_56562---,00.html

12. Z-Score “Cheat Sheet” • Z-scores are centered around zero • Positive numbers mean the student or school is above the state average • Negative numbers mean the student or school is below the state average State Average …Worse than state average Better than state average…. 1 -3 -2 0 2 3 -1

13. Z-Score Examples • Your school has a z-score of 1.5. You are better than the state average. Z-score of 1.5 State Average …Worse than state average Better than state average…. 1 -3 -2 0 2 3 -1

14. Z-Score Examples • Your school has a z-score of .2. You are better than the state average, but not by a lot. Z-score of 0.2 Z-score of 1.5 State Average …Worse than state average Better than state average…. 1 -3 -2 0 2 3 -1

15. Z-Score Examples • Your school has a z-score of -2.0. You are very far below state average. Z-score of -2.0 Z-score of 0.2 Z-score of 1.5 State Average …Worse than state average Better than state average…. 1 -3 -2 0 2 3 -1

16. How Is the Top to Bottom Ranking Calculated • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

17. How do we get Standardized Scale Scores for Each Student? • Step #1: Take each student’s score on the test they took and compare that score to the statewide average for students who took that same test in the same grade and year • This creates a student-level z-score for each student in each content area • Compare • MEAP to MEAP • MEAP-Access to MEAP-Access • MME to MME • MI-Access • Participation to Participation • Supported Independence to Supported Independence • Functional Independence to Functional Independence

18. What do we do with those standardized scores? • Step #2: Once each student has a z-score for each content area (based on the test they took), we take all of the students in a each school, and rank order the students within the school. • Z-scores will have come from different tests, and compare students to statewide average for that grade, test, and subject • But they can now be combined for the school • Step #3: Add up all z-scores and take the average. This is now the average standardized student scale score. • Step #4: Define the top and bottom 30% subgroups, based on that rank ordering.

20. Average Z-score (average standardized student scale score): 0.28 (sum all z-scores, divide by 15)

21. Top 30% Bottom 30%

22. How Is the Top to Bottom Ranking Calculated • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

23. What is important to show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Step #1: Achievement How well did the school do in that subject? Positive number = better than average Near zero = average Negative number = worse than average Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

24. Step #2: Improvement Is the school improving in that subject? Positive number = greater rate of improvement than average Near zero = average improvement Negative = slower rate of improvement than average; can also mean they are declining What is important to show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

25. Raw value is also meaningful: Positive number: More students improving than declining Negative number: More students declining than improving What is important to show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

26. Step #3: Achievement Gap Is the gap in that subject between top 30% and bottom 30%: (positive number) = smaller gap than average (negative number) = larger gap than average (near zero) = average gap What is important to show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

27. Once they have looked at each component, Discuss: • What’s the overall pattern? • Low achievement? • Declining achievement? • Large gaps? • Where are the actionable areas? • Which subjects need the most attention? • Is everyone doing poorly (small gap, low achievement) or are some students doing well and others falling behind (decent achievement, but large gap)

28. What is important to show Schools? • For grade 3-8 reading and mathematics Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Reward Schools (for improvement) Two-Year Average Performance Level Change Index School Performance Level Change Z-Score Content Index Z-score School Content Area Index 1/4 Focus Schools Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

29. Weighted Performance Level Change • A weighted composite of individual student performance level change is used to calculate improvement in grades 3-8 reading and mathematics • Rewards large improvements more heavily, rewards maintenance of proficiency if a student was already proficient

30. How Is the Top to Bottom Ranking Calculated • For science, social studies, writing, and grade 11 all tested subjects Two-Year Average Standardized Student Scale (Z) Score School Achievement Z-Score 1/2 Four-Year Achievement Trend Slope School Performance Achievement Trend Z-Score School Content Area Index Content Index Z-score 1/4 Two-Year Average Bottom 30% - Top 30% Z-Score Gap School Achievement Gap Z-Score 1/4

31. How Is the Top to Bottom Ranking Calculated • For graduation rate Two-Year Average Graduation Rate School Graduation Rate Z-Score 2/3 Four-Year Graduation Rate Trend Slope School Graduation Rate Trend Z-Score School Graduation Rate Index Grad Index Z-score 1/3

32. How Is the Top to Bottom Ranking Calculated • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

33. How Is the Top to Bottom Ranking Calculated • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate)

34. How Is the Top to Bottom Ranking Calculated • Calculating a four-year slope (e.g., graduation rate) • Plot the school’s graduation rate for the last four years • Plot a linear regression line through the points • Calculate the slope of the line (gives the school’s annual improvement rate) Slope = 2.3%

35. How Is the Top to Bottom Ranking Calculated • Calculating an overall ranking for a school with a graduation rate School Mathematics Std Index 18% School Reading Std Index 18% Overall Standardized School Index Overall School Percentile Rank School Science Std Index 18% School Social Studies Std Index 18% School Writing Std Index 18% School Graduation Rate Std Index 10%

36. How Is the Top to Bottom Ranking Calculated • Calculating an overall ranking for a school without a graduation rate School Mathematics Std Index 20% Look at each subject index. Help schools understand which subjects are strong/weak for them. Positive number: better than average Negative number: below average Near zero: near average School Reading Std Index 20% Overall School Standardized Index Overall School Percentile Rank School Science Std Index 20% School Social Studies Std Index 20% School Writing Std Index 20%

37. How Is the Top to Bottom Ranking Calculated • Calculating an overall ranking for a school without a graduation rate and without a writing score School Mathematics Index 25% School Reading Index 25% Overall School Standardized Index Overall School Percentile Rank School Science Index 25% School Social Studies Index 25%

38. Which years of data are in the ranking?

39. For elementary and middle schools • Michigan tests in the fall. • These fall tests reflect the learning of students in the previous school year. SY 2008-2009 SY 2009-2010 SY 2010-2011 Fall 2010 Testing Fall 2008 Testing Fall 2009 Testing Fall 2011 Testing

40. For High Schools • Michigan tests in the spring • The spring test (MME and MI-Access) measures what students have learned from grades 9, 10 and grade 11 prior to the MME testing.

41. What do the 2012 Rankings reflect? • For elementary/middle schools: • Performance on the MEAP and MI-Access tests in fall 2011 (which represents learning from school year 2010-2011) and before • For high schools: • Performance on the MME and MI-Access tests in spring 2012 (which represents learning from school year 2011-2012 prior to testing) and before.

42. Resources to Understand my Ranking

43. Resources Available • Complete list of all schools and their ranking • Individual school look-up to see your school’s results • Overview presentation with voice over • FAQ • Business rules by which the rankings were calculated • Complete data file and validation file You can access these resources at www.mi.gov/ttb You can also request individual assistance by calling the Evaluation, Research and Accountability unit at 517-373-1342, or emailing mde-accountability@michigan.gov