Assessment & Accountability

1. Wyoming Department of Education Assessment & Accountability Statewide System of SupportMeeting October 15, 2014 Deb Lindsey, Division Administrator

2. Assessment Topics • Assessment program requirements • Assessment transition, spring 2013 to spring 2015, PAWS • PAWS and ACT standard-setting • Growth: student growth percentiles, median growth percentiles, adequate growth percentiles, what??!!

3. Current Assessment Program • ESEA requires annual testing in grades 3-8, plus once in high school, of ELA/reading and math • ESEA also requires science testing once in each grade span, elementary, middle and high • Wyoming statute requires a writing test in grades 3, 5 and 7 during a window that’s different from the reading and math tests • Wyoming statute also requires the ACT Explore, Plan and ACT Plus Writing in grades 9, 10, and 11; the ACT at grade 11 serves as the ESEA-required accountability assessment

4. Current Assessment Program • ESEA requires that all assessments measure performance on the state’s adopted content standards • ESEA and IDEA permit adoption of alternate content standards for students with significant cognitive disabilities (1%); these are to be directly aligned to the “primary” content standards • Students with significant cognitive disabilities must participate in alternate assessments aligned to the alternate standards (extensions of the “regular” standards)

5. PAWS Design Changes PAWS blueprint necessitated new proficiency level descriptors, standard-setting and cut scores before 2014 results were released PAWS CCSS PAWS PAWS CCSS CCSS 2013 2014 2015

6. Standard-Setting Context

7. Standard-Setting Context

8. 2014 Standard-Setting • Conducted for PAWS, SAWS (gr 5 & 7), and ACT • Plans approved by WDE and its Technical Advisory Committee (TAC) • Participants included Wyoming teachers; for ACT, also included administrators and higher ed representatives • PAWS-SAWS judgments were standards-based; ACT judgments were empirically-based • Consistent with experiences of other states, different expectations result in lower proficiency rates

9. 2014 Results • Baseline data from new test based on different expectations in 2012 WY Content and Performance Standards in ELA and Math • Results are not directly comparable to prior years because of the content shift and the “break” in scale -- but can use concordance tables to roughly compare 2013 to 2014 • Focus is on continuous improvement; student performance will improve over time as districts continue to implement new standards • Primary purpose of assessment in WY: support improved teaching and learning, school and program improvement, and measuring performance indicators under W.S. 21-2-204

10. 2015 Assessment • No changes to state statute in 2014 Legislative Session • Continue PAWS, grades 3 – 8, with current blueprints & summer 2014 cut scores • Continue SAWS, grades 3, 5, & 7; use summer 2014 cuts for 5 & 7 (grade 3 cuts to be set in summer 2015) • Adopt new ALT beginning in 2015 • Continue with ACT Explore, ACT Plan, and ACT Plus Writing at grades 9 – 11; use new Wyoming ACT scale for cut scores needed for WAEA and federal reporting

11. WAEA Indicators • Elementary and Middle • Achievement • Growth • Equity • High School • Achievement • Equity • College Readiness Wyoming School Rating System

12. What is growth? • PAWS shows how each student is achieving relative to state standards • Is John proficient in 6th grade mathematics? What percent of his peers are proficient? • Growth measures change in an individual student’s performance over time, using scale scores from one year to the next • How much did John improve in mathematics from 5th grade to 6th grade relative to his academic peers?

13. Why measure growth? • Measures progress for students at all performance levels • A student can achieve at a low level but still improve relative to his academic peers • Another could achieve well but not improve much from year to year • Considered to be more fair in school accountability systems • Provides evidence of improvement even among those with low achievement • Gives high achieving students and schools something to strive for beyond proficiency

14. Growth Across Years Advanced Below Basic Proficient Basic Grade 5 Performance Levels: PAWS Reading Basic Proficient Advanced Below Basic Grade 4 Performance Levels: PAWS Reading

15. Growth Across Years Advanced Below Basic Proficient Basic Grade 5 Performance Levels: PAWS Reading Basic Proficient Advanced Below Basic Grade 4 Performance Levels: PAWS Reading

16. Growth Models Have Shown… • Growth not correlated with proficient status • Growth gaps can narrow among student groups • Many low-achieving students not growing fast enough to catch up • Many proficient students not keeping up • High-achieving schools can show low growth and low-achieving schools can show high growth

17. Growth Terms Student Growth Percentiles (SGP) – a normative measure that compares students with other like-performing students across the state. A SGP produces a relative percentile score (such as 70th percentile) that tells the student that s/he scored equal to or better than 70% of students who had scores like him in the previous year in the state. Median Growth Percentiles (MGPs) are used to summarize SGPs across classes, grades, and schools.

18. Growth Terms Adequate Student Growth Percentiles (AGP) – a criterion-referenced measure relative to proficiency that measures how far away from proficiency a student is and answers: “how much growth does a student have to make to reach proficiency in three years or by the end of 8th grade?” A student can make 70th percentile growth and still not meet AGP goals. AGPs are NOT reported in 2014 because of the change in cut scores but will return with the 2015 SPRs. When SGP => AGP, then the student is on track.

19. A student’s PAWS scores can be plotted from one year to the next 500 450 425 400 Mathematics PAWS scaled score 350 300 250 3 5 6 4 Grade

20. A student’s PAWS scores can be plotted from one year to the next 500 450 455 425 400 Mathematics PAWS scaled score 350 300 250 3 5 6 4 Grade

21. The fourth grade scores of students with the same third grade score can differ and form a distribution 500 450 455 425 400 Mathematics PAWS scaled score 350 300 250 3 5 6 4 Grade

22. Comparing the example student’s score to students with similar score histories yields a percentile 500 82nd 450 50th 400 Mathematics PAWS scaled score 350 300 250 3 5 6 4 Grade

23. The fifth grade growth percentile is calculated relative to students with similar score histories at both grades three and four 500 Other students whose scores diverged from the example student are no longer considered to have a similar score history 450 46th 400 Mathematics PAWS scaled score 350 300 250 3 5 6 4 Grade

24. SGP Features A student growth percentile compares the student’s current PAWS score with the scores of students throughout the state with similar score histories

25. SGP Features SGP: 1-99

26. Example 1 What does Suzie’s 5th grade scale score tell you? If you have the scale score ranges with each performance level, then you’ll know which proficiency category she’s in.. What can you tell from Suzie’s growth percentile of 70? At fifth grade, Suzie outperformed 70 percent of students with similar score histories. Can you calculate Suzie’s growth percentile just by knowing her previous years’ scores? No, because we do not have the distribution of scores from students with similar score histories.

27. Example 3 Why does Jamar but not Dante, have a student growth percentile? Should Zachary’s teacher be concerned about his performance, given his scale score and growth percentiles? Jamar has two consecutive years’ worth of data; Dante does not. Zachary is achieving at a high level but his progress relative to other students in the state who also have this score history, is low. This is concerning because all students, including high achieving ones, can and do achieve high growth.

28. Rules of Thumb Typical student growth percentiles are between about 40 and 60 WY School Performance Reports >=60 Exceeding, >=45 to <60 Meeting, <45Below Students or groups outside this range has higher or lower than typical growth Differences of fewer than 10 SGP points are likely not educationally meaningful

29. Growth is distinct from achievement A student can achieve at a low level but grow quickly, and vice versa Each student is compared only to his statewide academic peers, not to all students statewide Others with a similar test score history All students can potentially grow at the 1st or 99th percentile Growth is subject-, grade-, and year-specific Different academic peer groups for each subject, grade, and year Therefore, the same change in scaled scores can yield different student growth percentiles The percentile is calculated on the change in achievement, not the absolute level Differs from more familiar norm-referenced measures Key Concepts

30. Median Student Growth Percentile Median SGP for the 6th grade class Imagine that the list of students to the left are all the students in your 6th grade class. Note that they are sorted from lowest to highest SGP. The point where 50% of students have a higher SGP and 50% have a lower SGP is the median. The Median Growth Percentile (MGP) is used in the Wyoming school rating system under the Wyoming Accountability in Education Act (WAEA), Enrolled Act 65.

31. Questions? Deb Lindseydeb.Lindsey@wyo.gov307-777-8753