1 / 36

Snapshots: What We Know about State Assessment Practices

Snapshots: What We Know about State Assessment Practices. Martha L. Thurlow National Center on Educational Outcomes THURL001@umn.edu www.nceo.info. Assessment Options. General Assessments General Assessments with Accommodations

kaloni
Download Presentation

Snapshots: What We Know about State Assessment Practices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Snapshots:What We Know about State Assessment Practices Martha L. Thurlow National Center on Educational Outcomes THURL001@umn.edu www.nceo.info

  2. Assessment Options • General Assessments • General Assessments with Accommodations • Alternate Assessments based on Grade-Level Achievement Standards • Alternate Assessments based on Alternate Achievement Standards • Alternate Assessments based on Modified Achievement Standards

  3. General Assessments • Participation • Performance • Assessment Development • Narrowing the Achievement Gap

  4. Participation • Public reporting of participation provides one view – most often NCLB view (students count only if in school for one year) • Reporting to the Secretary of Education, as required by law gives another view – based on total population of students with IEPs

  5. Reading Assessment Participation Rates in Elementary School: Percent Participation is of IEP Enrollment (Includes Regular and Alternate Assessment) 98 100 100 2005-2006 99 107 100 100 99 99 99 md 99 md 99 md 99 99 99 99 99 99 99 95 100 100 100 99 98 100 100 92 98 99 98 100 99 100 98 100 100 99 99 99 99 100 99 AS 100 100 94 BIE 95 98 100 CNMI 98 DC 95 FSM Key md 96 GU > 105% (n=1 regular states and 0 unique states) Palau 100 PR 95% - 105% (n=45 regular states and 7 unique states) 99 RMI < 95% (n=1 regular states and 2 unique states) 100 VI md =missing data (n=3 regular state and 1 unique state) 84 From NCEO analysis of APR data – Thurlow et al., 2008

  6. Reading Assessment Participation Rates in High School: Percent Participation is of IEP Enrollment (Includes Regular and Alternate Assessment) 91 100 98 2005-2006 98 97 89 97 98 96 93 md 96 84 98 97 95 96 99 97 95 96 md 91 86 97 99 94 98 96 98 90 98 84 96 96 98 100 92 100 100 93 97 100 97 98 97 AS 97 100 98 BIE 89 62 CNMI 87 75 DC 72 FSM Key 46 GU 84 > 105% (n=0 regular states and 0 unique states) Palau 100 PR 95% - 105% (n=35 regular states and 3 unique states) 96 RMI < 95% (n=13 regular states and 6 unique states) md VI md =missing data (n=2 regular states and 1 unique state) 80 From NCEO analysis of APR data – Thurlow et al., 2008

  7. Participation Reporting Practices Changing Situations Under Which Students are Counted as Nonparticipants (# of States): 20072005 Did not take assessment 47 35 Did not complete enough items to score 17 6 Accommodations produced invalid results 18 8 Tested at a lower grade than enrollment 16 2 Based on data from the 2007 NCEO Survey of States

  8. Performance • Public reporting of performance provides one view – most often NCLB view (students count only if in school for one year) • Reporting to the Secretary of Education, as required by law gives another view – based on total population of students with IEPs National Center on Educational Outcomes (NCEO)

  9. Reading Assessment Proficiency Rates in Elementary School: Percent Proficient of IEP Enrollment (Regular and Alternate Assessment) 46 61● 45 2005-2006 19 70 33* 62 51 53● 65● 48● md 31 37 md 69 35 23● 51● 27 60 27 44 26 50 32 47 65 21 50 54 70● 23 50 55 51 65 21 59 21 49 29 20 53● 45 58 AS 58● 31 6 BIE 38● 17 CNMI 14 30 DC 17 FSM Key – Percent Proficient * State had >105% participation on assessments md GU 50% – 100% (n=22 regular states and 0 unique states) 2 Palau 40% – 49% (n=7 regular states and 0 unique states) 0 30% – 39% (n=7 regular states and 2 unique states) PR  State had >1% proficient on alternate assessment based on alternate achievement standards 20% – 29% (n=10 regular states and 0 unique states) 33 RMI 10% – 19% (n=2 regular states and 3 unique states) 0 VI 0% –9% (n=0 regular states and 4 unique states) 16 From NCEO analysis of APR data – Thurlow et al., 2008 md = missing data (2 regular states and 1 unique state

  10. Reading Assessment Proficiency Rates in High School: Percent Proficient of IEP Enrollment (Regular and Alternate Assessment) 39 44● 29 2005-2006 15 11 45 20 42 32 17 55● 19 44 32 40 58 20 32 56● 40 md 24 24 31 33 26* 50 9 45 26 12 68● 24● 14 15 20 81 9 21 19 39 25 17 8 38 72● AS 57● 16 30● BIE 16● 13 CNMI 5 19 DC 7 FSM * State had >105% participation on assessments Key – Percent Proficient 16 GU 50% – 100% (n=8 regular states and 0 unique states) 2 Palau 40% – 49% (n=7 regular states and 0 unique states) 0 30% – 39% (n=9 regular states and 0 unique states) PR  State had >1% proficient on alternate assessment based on alternate achievement standards 20% – 29% (n=11 regular states and 0 unique states) 19 RMI 10% – 19% (n=10 regular states and 5 unique states) md VI 0% –9% (n=4 regular states and 4 unique states) 8 From NCEO analysis of APR data – Thurlow et al., 2008 md =missing data (n=1 regular state and 1 unique state

  11. Assessment Development How States Addressed Universal Design in General Assessment Development Process (# of States): Included in RFP for test development 39 Expert review 27 Think aloud methods as part of field test 2 Statistical analyses 10 “UDA” NOT addressed in test development 1 Based on data from the 2007 NCEO Survey of States

  12. Assessment Development Bias/Sensitivity Reviews – Representation of Disability Perspective Disability No Disability Representation Representation Content Review 31 3 Bias Review 33 2 Based on data from the 2007 NCEO Survey of States

  13. Narrowing the Achievement Gap Perceived Factors Related to Positive Achievement Based on data from the 2007 NCEO Survey of States National Center on Educational Outcomes (NCEO)

  14. Positive Trends in Performance Data Availability for Positive Trend Factors Based on data from the 2007 NCEO Survey of States

  15. General Assessments with Accommodations • Numbers Using Accommodations • Accommodations Policies • Hints & Tips for States

  16. Accommodations • States are required to report on the number of students using accommodations during the general assessment • Not all states yet report these data; some report data for different types of accommodations than others

  17. Reading Assessment Accommodation Rates in Elementary School: Percentage of Students with IEPs Taking the Regular Reading Assessment with Accommodations 32 54 63 2005-2006 40 8 72 47 55 52 md 22 40 md 71 md 42 53 79 77 47 52 61 52 41 49 63 36 39 59 80 69 61 11 24 33 71 66 57 69 35 67 52 47 45 20 62 AS 1 73 94 BIE 64 59 CNMI 65 73 DC Key 71 FSM ≥ 75% (n=3 regular states and 1 unique states) md GU 67 Palau 50% - 74% (n=25 regular states and 5 unique states) md PR 26% - 49% (n=13 regular states and 1 unique states) 59 RMI < 25% (n=6 regular states and 0 unique states) md VI 43 md =missing data (n=3 regular states and 3 unique states) From NCEO analysis of APR data – Thurlow et al., 2008

  18. Reading Assessment Accommodation Rates in High School: Percentage of Students with IEPs Taking the Regular Reading Assessment with Accommodations 18 53 43 2005-2006 36 5 51 44 53 47 md md 37 md 70 57 45 42 76 77 53 md 38 73 49 23 41 27 35 59 79 65 43 5 46 0 59 46 81 59 22 66 56 50 62 0 61 AS 0 100 59 BIE 39 22 CNMI 28 42 DC Key 48 FSM ≥ 75% (n=4 regular states and 1 unique state) md GU 61 Palau 50% - 74% (n=17 regular states and 2 unique states) md PR 26% - 49% (n=17 regular states and 3 unique states) 46 RMI < 25% (n=8 regular states and 1 unique state) md VI md =missing data (n=4 regular states and 3 unique states) 53 From NCEO analysis of APR data – Thurlow et al., 2008

  19. Accommodation Policies Change States with Read Aloud Question Changes in Policy – comparing 2003 to 2005 and 2007 Number of States No changes since 2003 19 One change since 2003 19 Two changes since 2003 12 From NCEO Data Viewer at http://data.nceo.info

  20. Alternate Assessments based on Grade-Level Achievement Standards (AA-GLAS) • Number of States Providing This Option • Issue – A Misunderstood Option?

  21. Percentage of Students with IEPs Participating in AA-GLAS a Small percentage (< 0.5) participated n the grade level(s) and content area(s) represented as 0%. From NCEO analysis of APR data – Thurlow et al., 2008

  22. Issue: AA-GLAS Misunderstood 2005-062006-07* Kansas Indiana Louisiana Massachusetts Massachusetts Montana Minnesota North Carolina Mississippi Virginia North Carolina Washington Texas West Virginia Virginia Wisconsin

  23. Alternate Assessments based on Alternate Achievement Standards • Participation • Performance

  24. Reading Assessment Based on Alternate Achievement Standards Participation Rates in Elementary School: Percent Participation is of IEP Enrollment (both Alternate and Out-of-level) 12 20 6 2005-2006 34 3 8 6 12 21 7 23 6 md 5 md 5 7 8 10 6 5 7 3 6 6 9 8 6 5 6 6 9 8 5 4 8 6 8 6 5 4 6 12 18 7 6 AS 4 50 0 BIE 6 4 CNMI 4 22 DC 7 FSM Key md GU 8 Palau >9% (n=10 regular states and 2 unique states) 18 PR 0% − 9% (n=38 regular states and 7 unique states) 4 RMI md =missing data (n=2 regular states and 1 unique state) 0 VI 6 From NCEO analysis of APR data – Thurlow et al., 2008

  25. Reading Assessment Based on Alternate Achievement Standards Participation Rates in High School: Percent Participation is of IEP Enrollment (both Alternate and Out-of-level) 19 31 8 2005-2006 4 8 5 11 8 22 9 22 7 6 4 23 9 8 6 3 8 8 7 7 5 8 10 7 6 6 5 0 11 8 11 20 4 6 10 9 7 6 11 5 0 10 15 AS 47 0 24 BIE 13 3 CNMI 2 6 DC 4 FSM Key 0 GU 2 >9% (n=17 regular states and 0 unique states) Palau 0 PR 0% − 9% (n=33 regular states and 9 unique states) 9 RMI md =missing data (n=0 regular states and 1 unique state) md VI 9 From NCEO analysis of APR data – Thurlow et al., 2008

  26. States Exceeding the 1% Cap Delaware* Michigan Oregon** Florida Mississippi Texas Georgia North Dakota Virginia Louisiana* Ohio Wisconsin** Massachusetts** * High School Only ** Elementary School Only From NCEO analysis of APR data – Thurlow et al., 2008

  27. Alternate Assessments based on Modified Achievement Standards • Number of States Providing This Option • Criteria for Participation • Nature of Assessments National Center on Educational Outcomes (NCEO)

  28. Number of States with AA-MAS • July 2007 – 5 states indicated that they had an AA-MAS (all were developed prior to the April, 2007 regulation: • Kansas • Louisiana • North Carolina • North Dakota • Oklahoma • In addition, Maryland provided information on its Web site about the development a AA-MAS. From NCEO Synthesis Report 67 – Lazarus et al., 2007

  29. States Considering AA-MAS Based on data from the 2007 NCEO Survey of States

  30. Criteria for Participation Eligibility Criteria in 6 States with AA-MAS* * Includes Maryland From NCEO Synthesis Report 67 – Lazarus et al., 2007

  31. NCEO Survey Showed States Identifying Many Categories As Most Likely to Participate in AA-MAS Based on data from the 2007 NCEO Survey of States

  32. Nature of Assessments Type of Assessment/Questions in 6 States with AA-MAS From NCEO Synthesis Report 67 – Lazarus et al., 2007

  33. Nature of Changes to Items in 6 States with AA-MAS From NCEO Synthesis Report 67 – Lazarus et al., 2007

  34. Accommodations Incorporated into Design in 6 States with AA-MAS From NCEO Synthesis Report 67 – Lazarus et al., 2007

  35. Characteristics of Assessment in 40 Planning to Develop and AA-MAS Characteristic Number of States Include only multiple choice questions 11 Reduce total # of items or time to complete 25 Simplify vocabulary 26 Use non-traditional items or formats 16 Use shorter or fewer reading passages 29 Use fewer items or fewer answer choices 23 Based on data from the 2007 NCEO Survey of States

  36. Stay Tuned for More Snapshots Visit NCEO’s Web site for more and continuing information on assessment options: www.nceo.info

More Related