2013-14 Wyoming School Accountability

1. 2013-14 Wyoming School Accountability Presentation for DIRECTORS OF SPECIAL EDUCATION (12/17/13) Michael Flicek, ed.d. Education Accountability Consultant

2. WAEA School Performance Levels Exceeding Expectations Meeting Expectations Partially Meeting Expectations Not Meeting Expectations

3. Indicators used to Identify School Performance Level • Schools with grades 3 through 8 • Achievement • Growth • Equity • Schools with grades 9 through 12 • Achievement • Readiness • Equity

4. Achievement – Grades 3-8 • Assessments used in 2014 • PAWS reading – Grades 3-8 • PAWS math – Grades 3-8 • PAWS science – Grades 4 & 8 • SAWS – Grades 3, 5 &7

5. Achievement – Grade 11 • Assessment used in 2014 • ACT Subject-Area Tests • Reading • Mathematics • Science • Combined English/Writing

6. 2014 Standard Setting PAWS & ACT Subject Area Tests New student performance levels to be established Alignment with Wyoming State Standards (i.e. CCSS) Expected to be more rigorous than current performance levels

7. Some Business Rules • Minimum n for all indicators = 10 • When fewer than 10 students look back is applied • Look back 1 year first, then a second year when needed • Small school review when fewer than 2 indicators meet minimum n • Full Academic Year Only • October 1st to mid point of testing window

8. Grade 3 – 8 Model

9. Illustration of Computation of a Grade 3-8School Achievement Score

10. Professional Judgment Panel (PJP) • A representative group of 27 to 30 people • Representing groups prescribed by statute • Selected by the State Board of Education

11. PJP Major Tasks • Determined the cut points for school scores on each indicator that determine if schools are: • Exceeding Targets • Meeting Targets • Below Targets

12. Student Growth – Grades 4-8 • Growth in reading and in math • Each student will have a student growth percentile (SGP) • Same grade in school • Similar test scores in previous years • Scores from 1 to 99 • Excellent indicator for special education program improvement

13. School Growth – Grades 4-8 • For school accountability • For accountability • School median SGP (MGP) • The SGP that ½ of students at the school scored above and ½ of students at the school scored below • For school & program improvement • Special Education median SGP (MGP) • District overall • Within a school • Suppression rules applied to reports

14. Equity – Consolidated Subgroup • Current subgroups performance will continue to be reported • For Wyoming accountability, however, a consolidated subgroup will be used • The consolidated subgroup = all students who scored below proficient on the previous year’s reading test and/or math test

15. Equity – Grades 4-8 • Adequate Growth Percentile (AGP) • The SGP on this year’s test, the student needs to be on track for being proficient within 3 years or by the end of grade 8 • These are computed for all students in grades 4 through 8 with more than one previous PAWS test • School’s Equity Score • The percent of students at the school who are considered on track to become proficient within 3 years (i.e., for whom their SGP equaled or exceeded their AGP) • Another excellent indicator for special education • What % of special education students are on-track for proficiency

16. High School Model

17. High School Achievement Indicator

18. Alternative to Percent Proficient • Fluctuations in scale scores from year-to-year (i.e., documentation on WDE website) • Percent proficient cut-scores subject to substantial fluctuation unrelated to school performance • Alternative standardized score approach • More stable • More sensitive to changes due to school performance

19. Student Standardized Scores • Identify a baseline year (2013) • Compute statewide mean score and standard deviation for that year • For example, on the math subject area test: • Mean = 19.6 • Standard deviation = 4.5 • Compute a standardized score for each student • (state mean – student score)/ state standard deviation • Assume student score is 21 • (21 – 19.6) = 1.4 • 1.4/4.5 = 0.31 • “The student scored 31% of a standard deviation above the baseline mean score” • These scores will be negative when the student score is below the state baseline mean score

20. School Achievement Score • The mean student standardized score at the school multiplied by 100 and rounded to a whole number • The mean is for all subject area tests combined • Each school score will be a whole number • Most school will have scores will be between -100 and +100 • “A school with a score of 28 had a mean student score that was 28% of a standard deviation above the baseline year state mean score” • A mean of the special education subgroup would be helpful

21. High School Equity Indicator Policy objective: To encourage a focus on improving performance of the most high-risk students

22. High School Consolidated Subgroup • Criteria for consolidated subgroup membership • Current grade 11 students with • Grade 10 PLAN scores on 2013 subject area tests • Below 17 on the math test (bottom 37% of scores) and/or • Below 16 on the reading test (bottom 33% of scores) • Schools were notified on November 5th in a WDE Assessment Update about these criteria for the 2014 high school consolidated subgroup

23. Equity Score Computation • Illustration: student standardized score computation • State ACT math baseline mean score = 19.6 • State ACT math baseline year standard deviation = 4.5 • Assume a student’s ACT math score is 17 • (17 – 19.6) = -2.6 • -2.6/4.5 = -0.58 • This student’s score was 58% of a standard deviation below the state baseline year mean score • High School Equity Score = the median student standardized score for the consolidated subgroup for reading and math combined

24. Readiness – Grades 9-12 • Performance on ACT Suite (Explore, Plan & ACT) • Graduation index • Grade 9 credits earned • Hathaway eligibility

25. ACT Suite – Average Index Score for all Tested Students • Aligned with Hathaway Scholarship eligibility cut-points • Each student receives an index score • The average of the index scores for all students will be the school score

26. Graduation Index • The index points were established by the PJP

27. Grade 9 Credits Earned The percent of students who completed grade 9 with one fourth of the credits required to receive a diploma

28. Hathaway Scholarship Eligibility Index • The school’s score will be the mean of the student points for the • graduating class at the school • The school receives an index score for each graduate • The average of the index scores for all students will be the school score

29. Illustration of Total School Readiness Score

30. Decision Table for Schools with Three Indicators for Grades 3-8

31. Performance Level Descriptions(For Schools with Grades 3-8) EXCEEDING EXPECTATIONS: This category is reserved for schools considered models of performance. These schools demonstrated high achievement and exceeded target on at least one other performance indicator – equity or growth – while meeting target on the other indicator. MEETING EXPECTATIONS: Schools in this category demonstrated performance that met or exceeded target on multiple performance indicators. These schools typically had acceptable or better levels of achievement, student growth, and/or in promoting equity for students with below-proficient achievement. PARTIALLY MEETING EXPECTATIONS: Schools in this category performed below target on multiple performance indicators or were below target in achievement while failing to exceed target in the other indicator(s). Many schools in this category showed acceptable performance in promoting equity based on growth for students with below-proficient achievement and/or met target for student growth from year to year. NOT MEETING EXPECTATIONS: Schools in this category had unacceptable performance on all indicators. For schools in this category, improvement is an urgent priority. These schools have low levels of achievement, demonstrate below-target growth, and fall short of producing academic improvement for below-proficient students that will move them toward proficiency. PJP 2013 Version

32. Decision Table for Schools with Two Indicators for Grades 3-8

33. Decision Table for Schools with Three Indicators for High Schools

34. Performance Level Descriptions(For High Schools) EXCEEDING EXPECTATIONS: This category is reserved for schools considered models of performance. These schools demonstrated high achievement and exceeded target on at least one other performance indicator – equity or readiness – while meeting target on the other indicator. MEETING EXPECTATIONS: Schools in this category demonstrated performance that met or exceeded target on multiple performance indicators. These schools typically had acceptable or better levels of achievement, student readiness, and/or in promoting equity for students with below-proficient achievement. PARTIALLY MEETING EXPECTATIONS: Schools in this category demonstrated either unacceptable levels of achievement or were below target on improving the achievement of below-proficient students and on graduation rate and tested readiness. Many schools in this category showed acceptable performance in promoting equity based on growth for low achieving students and/or met target for student readiness. NOT MEETING EXPECTATIONS: Schools in this category had unacceptable performance on all indicators. For schools in this category, improvement is an urgent priority. These schools have low levels of achievement, fall short of targets on graduation and tested readiness, and have large achievement gaps that show little or no improvement. PJP 2013 Version

35. Decision Table for High Schools with Two Indicators

36. Participation Rate Impact • Schools with grades 3-8 • All schools had participation rates of 98% or higher in 2013 • One small school was docked a performance level for having less than 95% participation when a prior year was included in an attempt to meet the minimum n requirement • This school went from “meeting” to “partially meeting”

37. High School Participation Rate • 11 of 84 high schools had participation rates on the ACT suite of between 90% and <95% • 3 of these were already “not meeting expectations” • 4 dropped from “meeting” to “partially meeting” • 4 dropped from “partially meeting” to “not meeting”

38. Participation Rate “Not Met” • 12 high schools had less than 90% participation rate on the ACT suite of tests • 8 of these were already “not meeting” • 3 dropped from “partially meeting” to “not meeting” • 1dropped from “meeting” to “not meeting”

39. The Result Grades 3-8 did well on participation rate 13 of 84 high schools (16%) had lower performance levels because of poor participation rate on the ACT suite of tests An additional 10 of 84 high schools (12%) had less than 95% participation rate on the ACT Suite but were already “not meeting” 28% of high schools had participation Rate Problems.

40. Contact Information • Michael Flicek • mikefli@msn.com • 307-259-3963