Next-Generation Accountability

1. Next-Generation Accountability Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association November 22, 2011

2. The Challenge • Design a school accountability system that: • Sets a high proficiency standard (where proficiency is based on career and college ready standards) AND • Rewards schools for achieving growth with students, regardless of starting point • Moves Michigan toward a higher level of preparation for career and college • Fair and equitably applied

3. Where are we now? Assessing Michigan’s Current Situation

4. College going rates • Statewide: • 71% of 2008-2009 graduates enrolled in an IHE • 73% of those who enroll earn at least one year’s worth of credits • Gives a total of 52% of 2008-09 graduates who earned at least one year’s worth of credits • By individual school: • Median = 63% • 25th percentile: 40% • 75th percentile: 75%

5. 2.5 2 1.5 Density 1 .5 0 0 .2 .4 .6 .8 1 Percentage of Graduates Enrolled at an IHE

8. 100 80 60 Percent Enrolled in an IHE 40 20 0 0 20 40 60 80 100 2008 MME Math Percent Proficient (based on new cut scores) Relationship between new cut scores and college going

9. Takeaways • Michigan students are going to college • Even if students are not proficient on new cut scores on the MME, they are enrolling in college. Question: Will those students be successful? Will they pursue challenging majors?

10. Achievement Gap • Since 2001, schools have been held accountable on overall student performance… AND the performance of the nine traditional subgroups • Put the focus on achievement of all students, as defined by demographic characteristics • Caveat: IF you had a sufficient number of students!

11. Economically disadvantaged gaps: Math

12. Ethnicity gaps: Math

13. Economically disadvantaged gaps: Reading

14. Ethnicity gaps: Reading

15. Career and College Readiness in Our Schools: Math 95th percentile 50th percentile

16. CCR: Reading 50th percentile 95th percentile

17. Tension: Is it really important that our students be career and college ready? • Evidence points to the importance of higher education. • Competitive job market, especially in states like Michigan. • Fast-growing occupations (health care, technology) require higher education/specific training • Educational inflation

18. Next Generation Accountability Charting a New Path

19. Necessary Components • Focused consequences and interventions for schools most in need • Achievement gap • Differentiated accountability = differentiated interventions • Fair and equitable • Ambitious AND achievable goals

20. Focused Consequences and Interventions • Priority Schools • Lowest 5% of the Top to Bottom list • Priority schools = PLA Schools • Aligns federal and state accountability • Priority schools must enter a three year cycle of school improvement, with the most highly targeted interventions

21. PLA Schools: Anecdotal Evidence • Two “cohorts” of PLA schools: 2010 and 2011. • 2010 schools: first year of implementation • 2011 schools: planning • 2010 schools: • About half experienced an increase in percent proficient and increased their improvement rate • More meaningful data in another 1-2 years

22. PLA Schools: Anecdotal Evidence • What are they saying? • ERA Unit doing PLA data visits • Being named a PLA school was extremely difficult • However, it has fundamentally altered the way the schools are approaching achievement • Pushing a fundamental redesign • Impetus to address crucial issues • Innovative strategies (i.e. flipped learning)

23. Achievement Gap as Central Focus • Achievement gaps have not closed to the extent that we need • Proposal: • Focus on the bottom 30% of students, regardless of demographic, not the traditional subgroups • Puts the attention firmly on the lowest achieving students • By improving that group, increase school’s overall achievement, and improvement rate

24. Achievement Gap • Pros • All schools have a subgroup • At least 700 schools have no subgroup under AYP traditional subgroups • Unmask low performance in high performing subgroups • Asks that all schools consider their lowest performing students • Schools cannot mask low-performance with overall high performance

25. Achievement Gap • Cons: • Concern that we will lose focus on demographic subgroups. • In the lowest 30% subgroup—approximately 70% of that group are also a member of one or more traditional subgroups. • High-achieving schools do not like it • People think that “lowering the ceiling on our highest achieving students” will help the schools

26. Achievement Gap: Focus Schools • Need to identify the schools with the largest achievement gaps. • Using the bottom 30% subgroup, would rank the bottom 10% in terms of largest gap. • Using traditional subgroups in a ranking (normative) setting is complicated: • Not all schools have a subgroup • Comparing schools with the same subgroup • Unfairly focuses on students with disabilities • Still allows for “masking”

27. Achievement Gap: Our Belief • If Michigan is serious about raising the achievement of ALL students, then the bottom 30% is the correct way to go. • Distributing accountability to traditionally high achieving schools and asking them to achieve those same results with all students is appropriate. • Michigan cannot leave students behind any longer.

28. Differentiated Accountability For All Schools • Priority schools = 5% and Focus Schools = 10%--so what about the other 85%? • Need a more nuanced system than pass/fail AYP. • Need to integrate performance for all students, bottom 30%, and all subjects (not just reading and mathematics)

29. Use this system to set a proficiency goal with improvement • Proficiency target = AMO • Set for each school as the increase in percent proficient necessary for that school to reach the overall target proficiency • Improvement target • If school does not meet proficiency target, can meet an improvement target • Set as the increase in percent proficient demonstrated by a high-improvement school in the base year

30. Example with Data • End proficiency target: 85% • School is at 10% proficient now • Need to improve 75% in 10 years, or 7.5% per year. • Proficiency target in year 1: 17.5% proficient • If does not meet it, must have improved by 3.5% (which is improvement rate for school at the 90th percentile in base year)

31. Bottom 30% Subgroup as Accountable Subgroup • Only one “accountable” subgroup now (still report on nine traditional subgroups) • Need to meet a proficiency target for the bottom 30% subgroup (unlikely…) OR the improvement target

32. Notes on this system • Offers differentiated AMOs by school • Keeps a clear proficiency target in the system • Proficiency target is actually an improvement target as well • Many of the increases demanded of schools will be greater than we have historically seen, so need the improvement (safe harbor) target

33. Questions • What is an “ambitious and attainable” end goal? 100%? 85%? 70%? How do you determine this? • Should meeting the target based on improvement be equivalent to meeting it based on straight proficiency? • Should we reset each year?

34. Participation • Necessary to keep a firm participation target in the system • Schools will begin to “game” on who they assess if no clear participation target

35. “Green” School

36. “Yellow” School

37. “Red” School

38. Rules for Colors • Need to be green on all indicators • This makes “green” a more rare indicator; it means that there are no areas of concern • To be red, need to be red on all five academic indicators; makes it a more rare indicator • Yellow—largest category—can have some red, some green; is indicative of “intervention” needed; use colors within to target • Final color is not the key determiner for consequences; priority/focus status is more critical

39. Questions? • How to determine the final colors? • Balancing public desire for “one” rating with internal knowledge that “one” rating is difficult. • Other indicators that should be included?

40. Contact Information • Venessa A. Keesler, Ph.D. • Evaluation, Research and Accountability • Office of Psychometrics, Accountability, Research and Evaluation (OPARE) • keeslerv@michigan.gov