State Special Education Outcomes: Results of a 2009 Survey of States

1. State Special Education Outcomes: Results of a 2009 Survey of States Jason Altman – NCEO Mari Quenemoen – NAAC TASH Annual Conference – Nov. 19, 2009

2. Introduction • It is important that state large-scale assessment systems be both inclusive of and fair to all populations including students with disabilities • A 2009 survey explored these issues as a whole and paid special attention to state practices in assessing students who may participate in an alternate assessment option

3. Survey • All 50 states returned the 12th survey of state directors of special education and state directors of assessment by the National Center on Educational Outcomes (NCEO) • The survey was disseminated to respondents electronically in the winter of 2009 • NCEO returned surveys for verification and, upon receipt, entered, reviewed, and analyzed the data • Results of the eleventh survey were presented at the 2008 TASH annual conference

4. Presentation Purpose • The purpose of this presentation is to provide a snapshot of new initiatives, trends, accomplishments, and emerging issues in the large scale assessment of students with disabilities • Presenters will pay special attention to issues surrounding the testing of students with disabilities who may be assessed using an alternate assessment

5. Apples to Oranges or Apples to Apples

6. General Issues • States reported that assessment validity and test design/content were areas of much success • Issues related to English language learners with disabilities and the performance of urban schools were often selected as challenging • States appear to have a very mixed viewpoint on the AA-MAS (7 respondents in each category) • On the other hand, respondents appear to be strongly entrenched on the successful side of the ledger as related to reporting and monitoring

7. Regular Assessment IssuesAccommodations • More than four in five states reported monitoring accommodations use in 2009

8. Regular Assessment IssuesAccommodations • Most respondents also reported that accommodations cause some difficulty on test day in schools and districts within their state (n = 41)

9. Alternate Assessments Based on Modified Academic Achievement Standards • In April 2007, new No Child Left Behind regulations on AA-MAS were finalized • States have the option of developing alternate assessments based on modified achievement standards (AA-MAS) • At the time of the 2007 survey of states we found that five states already had an assessment in place, 32 states were considering modifying an existing grade-level assessment and another twenty-five were considering developing a new assessment • In the past two years, some states have further refined their motivations for moving in this assessment direction, while other states were instead making efforts to improve the assessments that they already offer

10. Alternate Assessments Based on Modified Academic Achievement Standards • Since 2007, 14 states have decided not to develop of AA-MAS, while 12 states were still in the process of making their decision • Of the 24 that have developed, or were developing, an AA-MAS, nine have already given the assessment and analyzed the data, one for the first time in 2008-09

11. Alternate Assessments Based on Modified Academic Achievement Standards • States that were developing or have developed their AA-MAS were three times more likely to modify an existing grade-level test rather than design an entirely new test

12. Alternate Assessments Based on Modified Academic Achievement Standards • Respondents have been asked on the past two surveys about the changes they envisioned making when modifying existing tests • Findings were similar to those found in 2007 save for a less frequent response to use of non-traditional items (keeping in mind that many more states have decided not to develop the AA-MAS at this time

13. Alternate Assessments Based on Modified Academic Achievement Standards • States reported a variety of strategies and methods in determining content targets and blue prints for their AA-MAS

14. Alternate Assessments Based on Alternate Achievement Standards • Seven states (HI, ID, MS, NE, NH, NV, and UT) and three unique states (Guam, PR, and RMI) are currently in the process of revising their AA-AAS • Trend analysis from 2005 and 2003 data

15. AA-AAS Topics • Test formats • Content alignment • Scoring methods • Rubric analysis • Methods for determining achievement levels • Scorers

16. AA-AAS Formats • 40% of states use a portfolio or body of evidence (40% of these are standardized) • 36% use a standardized set of performance events/tasks (44% of which require the submission of evidence) • 16% use a multiple choice test

17. AA-AAS Content Alignment • Complete transition to alignment with academics • Extended/expanded academic content standards (27 states) or grade level academic content standards (16) • No states align AA-AAS to functional skills, and IEP teams no longer determine the content of the test for individual students

18. AA-AAS Scoring Procedures

19. Outcomes Measured by Rubrics • Primarily skill/competence, level of assistance, and alignment to academic content • Trend away from scoring other factors

20. Outcomes Measured by Rubrics (2005 and 2009)

21. Outcomes Measured by Rubrics ( States and Unique States 2009)

22. Methods for Determining Achievement Levels

23. Who Scores the AA-AAS?

24. Thank you! For more information… National Alternate Assessment Center University of Kentucky 859-257-7672 http://www.naacpartners.org Mari Quenemoen: mari.quenemoen@uky.edu National Center on Educational Outcomes University of Minnesota612-626-1530http://www.nceo.info Jason Altman: altma014@umn.edu