Wyoming School Performance Rating Model Report to: Wyoming State Board of Education

1. Wyoming School Performance Rating ModelReport to:Wyoming State Board of Education Dr. Michael Flicek Education Consultant October 8, 2013

2. WAEA School Performance Levels Exceeding Expectations Meeting Expectations Partially Meeting Expectations Not Meeting Expectations

3. Indicators used to Identify School Performance Level • Schools with grades 3 through 8 • Achievement • Growth • Equity • Schools with grades 9 through 12 • Achievement • Readiness • Equity

4. Achievement – Grades 3-8 • Assessments used in 2013 • PAWS reading – Grades 3-8 • PAWS math – Grades 3-8 • PAWS science – Grades 4 & 8 • Assessment to be added in 2014 • SAWS – Grades 3, 5 &7

5. Achievement – Grade 11 • Assessment used in 2013 • ACT Subject-Area Tests • Reading • Mathematics • Science • Assessment to be added in 2014 • Writing (to begin after spring 2014 testing)

6. Illustration of Computation of a School Achievement Score

7. Professional Judgment Panel (PJP) • A representative group of 27 to 30 people • Representing groups prescribed by statute • Selected by the State Board of Education

8. PJP Major Tasks • Determined the cut points for school scores on each indicator that determine if schools are: • Exceeding Targets • Meeting Targets • Below Targets

9. Grade Bands for Achievement with PJP Established Cut Scores • Grade 3 through 6 • Low cut = 75 • High cut = 86 • Grades 7 and 8 • Low cut = 68 • High cut = 80 • Grade 11*See Oct. 7, 2013, Data Preparation Report by Flicek & Paul • Low cut = 63 • High cut = 78

10. Data Preparation • WAEA Data Model • PJP version contained original ACT student proficiency cut-scores • Replaced by adjusted ACT student proficiency cut-scores in current version • Used equipercentile linking to correct cut-scores for the adjusted ACT student proficiency cut-scores

11. Grade 11 Achievement Cut Scores

12. Impact of Corrected Data andCorrected Cut Scores

13. Student Growth – Grades 4-8 • Growth in reading and in math • Each student will have a student growth percentile (SGP) • Same grade in school • Similar test scores in previous years • Scores from 1 to 99

14. School Growth – Grades 4-8 • For accountability • School median SGP (MGP) • The SGP that ½ of students at the school scored above and ½ of students at the school scored below • PJP Established Cut Scores were: • Low cut = 45 • High cut = 60

15. Equity – Consolidated Subgroup • Current subgroups performance will continue to be reported • For Wyoming accountability, however, a consolidated subgroup will be used • The consolidated subgroup will be all students who scored below proficient on the previous year’s achievement test • Minimum n size for subgroup accountability = 15

16. Equity – Grades 4-8 • Adequate Growth Percentile (AGP) • The SGP on this year’s test, the student needs to be on track for being proficient within 3 years or by the end of grade 8 • These are computed for all students in grades 4 through 8 with more than one previous PAWS test • School’s Equity Score • The percent of students at the school who are considered on track to become proficient within 3 years (i.e., for whom their SGP equaled or exceeded their AGP) • Equity cut scores were: • Low cut = 40 • High cut = 55

17. Equity – Grade 11 Policy objective: To encourage a focus on improving performance of the most high-risk students This will be applied to schools which had at least 15 students assessed on the ACT that were at the school for a full academic year

18. High School Equity 2013 • The percent of not proficient test scores in reading and math on the 2012 PAWS was computed • The percent of not proficient test score in reading and math on the 2013 ACT subject area test was computed • The change in the percent of not proficient test scores from 2012 to 2013 was computed

19. High School Equity 2013 • The schools were placed into one of three (approximately equal) categories for percent improvement • Decrease in % Not Proficient (Exceeding Targets) (decrease of -3.2% or more) • Minimal Change in % Not Proficient (Meeting Targets) (-3.2 to +3.4) • Increase in % Not Proficient (Not Meeting Targets) (increase of 3.4 or more)

20. Correction • Reading and Math only • ACT datawith adjusted student performance levels was used • 16 schools exceeding targets • 20 schools meeting targets • 17 schools not meeting targets

21. High School Equity in 2014 • A consolidated subgroup will be established based upon Plan scores from 2013 for the 2013-14 grade 11 students • The change in performance of this subgroup from the 2013 PLAN to the 2014 ACT will become the metric • This metric has been shown to perform in a manner similar to the 3-8 equity indicator

22. Readiness – Grades 9-12 • In 2013 • Performance on ACT Suite (Explore, Plan & ACT) • Graduation index • Indicators to be added in 2014 • Grade 9 credits earned • Hathaway eligibility

23. ACT Suite – Average Index Score for all Tested Students • Aligned with Hathaway Scholarship eligibility cut-points • Each student receives an index score • The average of the index scores for all students will be the school score

24. Graduation Index 2013 • The index points were established by the PJP

25. Graduation Index 2014 • The index points were established by the PJP.

26. Grade 9 Credits Earned - 2014 The percent of students who completed grade 9 with one fourth of the credits required to receive a diploma

27. Hathaway Scholarship Eligibility Index - 2014 • The school’s score will be the mean of the student points for the • graduating class at the school • The school receives an index score for each graduate • The average of the index scores for all students will be the school score

28. Computation of Total School Readiness Score for 2013 • Total Readiness Cut Scores • Low Cut = 68 • High Cut = 80

29. Illustration of Computation of Total School Readiness Score for 2014

30. Decision Table for Schools with Three Indicators for Grades 3-8

31. Performance Level Descriptions(For Schools with Grades 3-8) EXCEEDING EXPECTATIONS: Schools in this category, which is reserved for schools considered models of performance, have demonstrated high growth overall, have average to high levels of achievement (proficiency rates) overall, and excel in promoting equity based on growth for students with prior below proficient performance. MEETING EXPECTATIONS: Schools in this category have demonstrated acceptable levels of achievement and growth overall and are showing acceptable progress in promoting equity based on growth for students with prior below proficient performance. PARTIALLY MEETING EXPECTATIONS: Schools in this category have demonstrated either acceptable levels of growth or acceptable levels of achievement overall. Schools in this category may or may not show acceptable performance in promoting equity based on growth for students with prior below proficient performance. NOT MEETING EXPECTATIONS: This category is reserved for schools with unacceptable performance on many or most indicators. For schools in this category improvement is a priority. These schools have low levels of achievement overall and demonstrate low to average growth overall and fall short of producing growth for below proficient students that will move them toward proficiency.

32. Number of Wyoming Schools with Grades 3-8 with Each Pattern (2013)

33. Decision Table for Schools with Two Indicators for Grades 3-8

34. Number of Wyoming Schools with Grades 3-8 With Each Pattern (2013)

35. Overall Impact for Grade 3-8 Schools

36. Decision Table for Schools with Three Indicators for High Schools

37. Performance Level Descriptions(For High Schools) • EXCEEDING EXPECTATIONS: Schools in this category, which is reserved for schools considered models of performance, have demonstrated average to high levels of achievement (proficiency rates) overall, have high performance on graduation rates and other readiness indicators and have narrow and/or improving achievement gaps for students with below proficient performance. • MEETING EXPECTATIONS: Schools in this category have demonstrated either high levels of achievement overall or high performance on graduation rates and other readiness indicators and are showing acceptable performance in promoting equity based on the magnitude and/or improvement of the achievement gap for students with below proficient performance. • PARTIALLY MEETING EXPECTATIONS: Schools in this category have demonstrated either acceptable levels of achievement overall or acceptably performance on graduation rates and other readiness indicators. Schools in this category may or may not demonstrate acceptable performance for promoting equity based on the size of the achievement gap or improvement in the achievement gap for students with below proficient performance. • NOT MEETING EXPECTATIONS: This category is reserved for schools with unacceptable performance on many or most indicators. For schools in this category improvement is a priority. These schools typically have low levels of achievement fall short of expectations on graduation and other readiness indicators and have large achievement gaps that show little improvement.

38. Number of Wyoming High Schools with Each Pattern (2013)

39. Decision Table for High Schools with Two Indicators

40. Number of Wyoming High Schools with Each Pattern (2013)

41. Overall Impact for High Schools (PJP Version)

42. Overall Impact for High Schools (Corrected)

43. Participation Rate Impact • Schools with grades 3-8 • All schools had participation rates of 98% or higher in 2013 • One small school was docked a performance level for having less than 95% participation when a prior year was included in an attempt to meet the minimum n requirement • This school went from “meeting” to “partially meeting”

44. High School Participation Rate • 11 of 84 high schools had participation rates on the ACT suite of between 90% and <95% • 3 of these were already “not meeting expectations” • 4 dropped from “meeting” to “partially meeting” • 4 dropped from “partially meeting” to “not meeting”

45. Participation Rate “Not Met” • 12 high schools had less than 90% participation rate on the ACT suite of tests • 8 of these were already “not meeting” • 3 dropped from “partially meeting” to “not meeting” • 1dropped from “meeting” to “not meeting”

46. The Result • Grades 3-8 did well on participation rate • 13 of 84 high schools (16%) had lower performance levels because of poor participation rate on the ACT suite of tests • An additional 10 of 84 high schools (12%) had less than 95% participation rate on the ACT Suite but were already “not meeting” • 28% of high schools had participation Rate Problems.

47. Final High School Impact (PJP)

48. Final Impact High Schools (Corrected)

49. Final Wyoming Result

50. Contact Information • Michael Flicek • mikefli@msn.com • 307-259-3963 • John Paul • John.paul@wyo.gov • 307-777-8771