New York State Growth Model for Educator Evaluation 2011–12

1. New York State Growth Model for Educator Evaluation2011–12 July 2012 PRESENTATION as of 7/9/12

2. Today’s Agenda • Background • The What, Why, and How of Growth Models and Measures • Using Growth Measures for Educator Evaluation • What Data Will Be Available and When?

3. Background

4. Evaluating Educator Effectiveness

5. The What, Why, and How of Growth Models and Measures

6. By the End of This Section…. • You should be able to: • Explain why the state is measuring student growth and not achievement • Describe how the state is measuring growth compared to similar students • Define a student growth percentile and mean growth percentile

7. Prior Year Performance for Students in Two Teachers’ Classrooms ─ Proficiency

8. Current Year Performance of Same Students ─ Proficiency

9. Prior and Current Year Performance for Ms. Smith’s Students

10. Student A’s Current Year Performance Compared to “Similar” Students If we compare student A’s current score to other students who had the same prior score (450), we can measure her growth relative to other students. We describe her growth as a “student growth percentile” (SGP). Student A’s SGP is the result of a statistical model and in this example is 45, meaning she performed better in the current year than 45% of similar students. High SGPs ELA Scale Score Student A 450 Low SGPs 2011 2012

11. Comparing Performance of “Similar” Students Given any prior score, we see a range of current year scores, which give us SGPs of 1 to 99. Current Year Score Prior Year Score

12. SGPs for Ms. Smith’s Students

13. Student Growth Percentiles: True or False? • A student with an SGP of 50 performed better than 50% of similar students. • A student with an SGP of 80 must be proficient. • A student with an SGP of 20 grew less than a student with an SGP of 60. • The highest SGP that a student can receive is 99. • A student with an SGP of 80 grew twice as much as a student with an SGP of 40.

14. From Student Growth to Teachers and Principals To measure teacher performance, we find the mean growth percentile (MGP) for his or her students. To find an educator’s mean growth percentile, take the average of SGPs in the classroom. In this case: Step 1: 45+40+70+60+40=255 Step 2. 255/5=51 Ms. Smith’s mean growth percentile (MGP) is 51, meaning on average her students performed better than 51% of similar students. A principal’s performance is measured by finding the mean growth percentile for all students in the school.

15. Which Students Count in a Teacher’s or Principal’s MGP for 2011–12? Student has valid test scores for at least 2011–12 and 2010–11 Student meets continuous enrollment standard for 2011–12 Student growth is attributed to the teacher and the school Yes Yes No No Student scores do not count for 2011–12 Expected for 2012–13: students weighted by duration of instructional linkage

16. From Student Growth to Teachers and Principals In order for an educator to receive a growth score, he or she must have a minimum sample size of 16 student scores in ELA or mathematics across all grades taught. Examples: A teacher has a self-contained classroom with 8 students who take the 4th grade ELA and math assessments; this teacher would then have 16 student scores contributing to his or her growth score. A teacher has a class with 12 students in varied grades (4th, 5th, 6th) who take the ELA and math assessments for their respective enrolled grade level; this teacher would then have 24 student scores contributing to his or her growth score. If an educator does not have 16 student scores, he or she will not receive a growth score from the state and will not receive information in the reporting system. Educators likely to have fewer than 16 scores should use student learning objectives (SLOs).

17. MGP Lower Limit Upper Limit Confidence Range MGPs and Statistical Confidence 87 • NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be provided. • An educator’s confidence range depends on a number of factors, including the number of student scores in their MGP and the variability of student performance in the classroom.

18. Pause and Reflect: Mean Growth Percentiles • We talked about: • How to find a mean growth percentile (MGP) • How to interpret an MGP • What students are counted in an MGP • How many student scores are needed to provide an MGP • How a measure of statistical confidence (upper and lower limits of a 95% confidence range) will be provided with MGPs and why

19. Expanding the Definition of “Similar” Students • So far we have been talking about “similar” students as those with the same prior year assessment score • We will now add two additional features to the conversation: • Two additional years of prior assessment scores • Remember—a student MUST have current year and prior year assessment score to be included • Student-level factors • Economic disadvantage (ED) • Students with disabilities (SWDs) • English language learners (ELLs)

20. Adjustments for Three Student-Level Factors in Measuring Student Growth TeacherInstruction Student performance Disability Other factors (12–13) Language proficiency Economic disadvantage

21. Student A’s Current Year Performance Compared to “Similar” Students If we compare student A’s current score to other students who had the same prior score (450), we can measure his or her growth relative to other students. We describe that growth as a student growth percentile (SGP). Student A’s SGP is the result of a statistical model and in this example is 45, meaning student A performed better in the current year than 45% of similar students. High SGPs ELA Scale Score Student A 450 Low SGPs 2011 2012

22. Expanding the Definition of “Similar” Students to Include Economically Disadvantaged—An Example Now if student A is economically disadvantaged, we compare student A’s current score to other students who had the same prior score (450) AND who are also economically disadvantaged. In this new comparison group, we see that student A now has an SGP of 48. ELA Scale Score High SGPs Student A 450 Low SGPs 2011 2012

23. Further Information on Including Student Characteristics in the Growth Model The following slides were developed using sample data from 2010–2011. The “combined” MGPs on the charts have been calculated at the educator level (combining all grades and subjects). Not all districts provided data linked to teachers for grades 4–8 ELA/Math in 2010–11.

24. Teacher MGPs after Accounting for Economic Disadvantage Taking student-level characteristics into account helps ensure educators with many students with those characteristics have a fair chance to achieve high or low MGPs. For example, note that for teachers with any percent of economically disadvantaged students, teacher MGPs range from 1 to 99. NOTE: Beta results using available 2010–11 data.

25. Teacher MGPs after Accounting for SWD NOTE: Beta results using available 2010–2011 data.

26. Teacher MGPs after Accounting for ELL Percent of ELL Students in Class NOTE: Beta results using available 2010–2011 data.

27. “Similar” Students: A Summary Reported to Educators Used for Evaluation Reported to Educators

28. One Last Feature of the Growth Model…. The New York growth model accounts for measurement error in computing student growth percentiles. All testscontain measurement error, with greater uncertainty for highest and lowest achieving students

29. State Growth Model Summary Growth model for 2011–12 only for grades 4–8 ELA/Math for teachers and principals

30. By the End of This Section…. • You should be able to: • Explain why the state is measuring student growth and not achievement • Describe how the state is measuring growth compared to similar students • Define a student growth percentile and mean growth percentile

31. Using Growth Measures for Educator Evaluation

32. By the End of This Section…. • You should be able to: • Explain how growth ratings and scores will be obtained, using illustrative data

33. Growth Ratings and Score Ranges The growth scores and ratings are based on an educator’s combined MGP.

34. Distribution of 2010–11 Teacher-Level MGPs Percent of MGPs Distribution of Mean Student Growth Percentiles (Teacher Level) For illustrative purposes only Number of Teachers 50 1 99 MGP NOTE: Beta results using available 2010–2011 data.

35. MGPs and Statistical Confidence MGP 87 Lower Limit Upper Limit Confidence Range • NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be provided. • An educator’s confidence range depends on a number of factors, including the number of student scores included in his or her MGP and the variability of student performance in the classroom.

36. HEDI Classification Approach for Teachers (using 2010–11 sample data) • Effective requires MGPs within 1 standard deviation of the average MGP of 51. • MGPs between 40 and 61 will earn Effective ratings. • Well Above Average (Highly Effective) requires • MGP of 62 or higher • AND confidence range above 51. (If not, rating is Effective.) • Well Below Average (Ineffective) requires • MGP of 39 or lower • AND confidence range must be less than 51. (If not, rating is Developing.)

37. From MGPs to Growth Ratings: Teachers Mean Growth Percentile Confidence Range Growth Rating Yes Yes Highly Effective: Results are well above state average for similar students Mean Growth Percentile ≥62 Lower Limit > 51 No Yes Yes Effective: Results equal state average for similar students Yes Yes Mean Growth percentile 40–61 Any Developing: Results are below state average for similar students Mean Growth Percentile ≤39 Upper Limit < 51 No Ineffective: Results are well below state average for similar students

38. HEDI Classification Approach for Principals (using 2010–11 sample data) Same methodology as for Teachers. Slightly different cut scores. • Effective requires MGPs within 1 standard deviation of the average MGP of 50. • MGPs between 43 and 57 will earn Effective ratings. • Well Above Average (Highly Effective) requires: • MGP of 58 or higher • AND confidence range above 50. (If not, rating is Effective.) • Well Below Average (Ineffective) requires • MGP of 42 or lower • AND Confidence Range must be less than 50. (If not, rating is Developing.)

39. From MGPs to Growth Ratings: Principals Mean Growth Percentile Confidence Range Growth Rating Yes Yes Highly Effective: Results are well above state average for similar students Mean Growth Percentile ≥ 58 Lower Limit > 50 No Yes Yes Effective: Results equal state average for similar students Yes Yes Mean Growth percentile 43–57 Any Developing: Results are below state average for similar students Mean Growth Percentile ≤ 42 Upper Limit < 50 No Ineffective: Results are well below state average for similar students

40. Well Above Average (62) Average (51) Well Below Average (39) MGP 99 MGP 1 MGP MGP MGP MGP MGP MGP MGP Illustrating Possible Teacher Growth Ratings

41. Illustrating Possible Teacher Growth Ratings Well Above Average (62) Average (51) Well Below Average (39) MGP 99 MGP 1 MGP MGP MGP MGP MGP MGP MGP

42. Illustrating Possible Teacher Growth Ratings Well Above Average (62) Average (51) Well Below Average (39) MGP 99 MGP 1 MGP MGP Ineffective MGP Highly Effective Developing MGP MGP Developing Effective

43. Illustrating Possible Teacher Growth Ratings Well Above Average (62) Average (51) Well Below Average (39) MGP 99 MGP 1 MGP Effective MGP Effective

44. Illustrative Results: Teachers(Using 2010–11 sample data) Points available within each HEDI category will be assigned based on educator MGP

45. Illustrative Results: Principals(Using 2010–11 sample data) Points available within each HEDI category will be assigned based on educator MGP

46. By the End of This Section…. • You should be able to: • Explain how growth ratings and scores are obtained

47. What Data Will Be Available and When

48. Data — What to Expect When Mid-July Early fall Mid-August Test scores finalized and teacher linkage data final submission Growth scores provided to districts Online reporting system available

49. Data — What to Expect in August Data Elements (for teachers and schools) • Unadjusted mean growth percentiles (Unadjusted MGPs) • Adjusted mean growth percentiles (Adjusted MGPs and upper and lower limits based on confidence range for these adjusted MGPs) • Percent of students above the State median: this will be provided at the teacher and school level, and can be used as a local measure in APPR • Number of student scores included • Growth rating (HEDI) • Growth score (0–20) Breakdowns (by teacher and school) • MGPs by subject, grade, and overall (not HEDI) • Can be used with SLOs as part of the Comparable Measures or Locally Selected Subcomponent • Overall MGPs for subgroups — ELL, SWD, Economic Disadvantage, High- and Low-Achieving • Subgroup scores will not be included on reports if there are fewer than 16 student scores

50. One Teacher’s Information — August Sample data for illustration only