Getting Up to Speed on Value-Added - An Accountability Perspective

1. Getting Up to Speed on Value-Added - An Accountability Perspective Presentation by the Ohio Department of Education

2. Agenda • Value-Added and the Accountability System • 2007 – Overall Value-Added Results • Analysis of Individual Districts/Schools • AYP Growth Model • Data Availability

3. Value-Added and the Accountability System

4. Former Accountability System Architecture

5. Change in Improvement Measure:Implementing Value-Added • Up to 2007, Performance Index Growth has been used • Starting in 2008, Value-Added data will be used for districts and schools with tested grades 4-8

6. Ohio’s Value-Added Measure • Measures the contribution of a school or district to the progress of its students on test scores • Requires a Scaled Score metric • Current Alternate Assessments are not measured on a Scaled Score

7. Ohio’s Value-Added Measure • Scores are measured in “Normal Curve Equivalent” gains • “0” gain represents the typical or “expected” gain • “Value-Added” is based on the 2006 – 07 distribution of scores • Scores also use a measure of precision (1 “Standard Error”) to help describe the Value-Added classification

8. Ohio’s Value-Added Measure There are three classification “bands” using “gain score” and 1 Standard Error • + Above expected growth (Green) •  Met expected growth (“one year of growth in one year of time”) (Yellow) • - Below expected growth (Red)

9. Ohio’s Value-Added Measure • Scores calculated for: • Grades (4-8) • Subjects (Reading and Math only) • Grade and subject composites • School composite • District composite • Only District and School composite scores will be used for ratings

10. Gain Score, Standard Error and Classifications Relative to Value-Added Standard Exceed (+) Exceed (+) Meet ( ) Meet ( ) Not Meet (-) Not Meet (-)

11. Value-Added Will Affect Ratings • 2007-08 first year VA can change LRC designation • Reward / sanction • Reward enough growth • Penalize insufficient growth

12. Impact on Designation Above expected gain is rewarded Each rating category is rewarded based on aboveexpected gain, including an “Excellent with Distinction” rating At least two years of aboveexpected gain 12

13. Below expected gain results in lower ratings Each rating category is impacted by belowexpected gain Rating is lowered if you have three years of belowexpected gain Academic Emergency will not be lowered Impact on Designation 13

14. New 2007-08 Accountability System Architecture Including VA Impact

15. Impact on Designation - Examples District A: PI = 85 (Effective Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating ? 15

16. Impact on Designation - Examples District A: PI = 85 (Effective Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating - Excellent 16

17. Impact on Designation - Examples District B: PI = 77 (Academic Watch Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating - ? 17

18. Impact on Designation - Examples District B: PI = 77 (Academic Watch Range) Met AYP VA – Above Expected in 2007 and 2008 Final Rating - Continuous Improvement (Why?) 18

19. Proposed Report Card Graphic *Used in LRC rating

20. 2007 – Overall Value-Added Results

21. Value-Added 2007 Data Scores based on four years of data: • 2006-07 Grades 4-8 results (Reading, Math, Writing, Science and Social Studies) • 2005-06 Grades 3-7 results (Reading and Math in addition to Grade 4 Writing) • 2004-05 Grade 3 results (Reading and Math); Grades 4-5 results (Reading only) • 2003-04 Grade 3 results (Reading only)

22. Achievement Test Results: NCE Means based on 2007 results

23. Relationship of Gains to Grade Level

24. Achievement Test Results: NCE Means based on 2007 results

25. Relationship of Gains to Grade Level

26. Value-Added 2007 Composite Results

27. Relationship of LRC Designation to District Typology *Note: Urban 21 are a subset of All Urban category

28. Relationship of Composite VA Gains to District Typology *Note: Urban 21 are a subset of All Urban category

29. Relationship of VA Gains to Achievement

30. Relationship of VA Gains to Math Achievement

31. Relationship of VA Gains to Reading Achievement

32. Performance (Status) by % Poverty

33. Performance (VA Gain) by % Poverty

34. Performance (Status) by % Minority

35. Performance (VA Gain) by % Minority

36. Analysis of Individual Districts/Schools

37. Value-Added Gains and Performance Index High Performance – Low Value-Added

38. Value-Added Gains and Performance Index Average Performance – Low Value-Added

39. Value-Added Gains and Performance Index Very Low Performance and Value-Added

40. Value-Added Gains and Performance Index - Very Low Performance – “Green” VA But is it enough?

41. Value-Added Gains and Performance Index “Green”- But will it be enough to improve?

42. Value-Added Gains and Performance Index Low Performance – High Value-Added

43. Value-Added Gains and Performance Index High Performance – High Value-Added

44. Value-Added Gains and Performance Index High Performance – High Value-Added

45. Within District VariabilitySchool level VA Gains vs. Performance

46. AYP Growth Model

47. Changes in 2008 • New Goals • Making minimum N uniform • Include a Growth Model criterion

48. New Goals

49. Uniform Minimum N Size • Minimum size for evaluation was: • 30 for all groups except students with disabilities • 45 for students with disabilities • Minimum N size starting with 2007-08: • 30 for all groups • Minimum N size change for 2007-08 report card data to meet federal requirement

50. Meeting Adequate Yearly Progress • Up to 2006-07, districts and schools could meet AYP achievement in one of three ways: • By meeting or exceeding all AYP targets; • By meeting or exceeding AYP targets with a two-year average of previous and current year’s reported data; • Via the AYP safe harbor provision –district/school achieves a 10% reduction in the percentage of non-proficient students from the previous year and also meets graduation or attendance rate goal.