South Carolina Alternate Assessment (SC-Alt)

1. South Carolina Alternate Assessment (SC-Alt) Advisory Committee Meeting October 17, 2012

2. Advisory CommitteeRole and Purpose

3. American Institutes for Research (AIR)SC-Alt Contractor

4. A.I.R. Staff • DeeAnn WagnerProject Director • Jennifer Chou Project Manager • Lynnett Wright Alternate Assessment Specialist

5. 2012Post Administration Discussion

6. Teacher Comments “Tasks are too high for all of my students. The test is very burdensome to assemble.” “My students all did great. Very easy to organize and put together for testing.”

7. 2012Student Participationand Performance

8. SC-Alt 2012 Participation by Primary Disability

9. SC-Alt 2011 to 2012 Participation Change

10. ELA Performance for All Students2007 - 2012

11. Math Performance for All Students2007 - 2012

12. Science Performance for All Students2007 - 2012

13. Social Studies Performancefor All Students in 2008 - 2012

14. ELA Scores for SC-Alt 2012 by Disability Group

15. Math Scores for SC-Alt 2012 by Disability Group

16. Second Rater Administration

17. Procedures Used for MonitoringSC-Alt Scoring Fidelity Special procedures are used to monitor test administration effectiveness and scoring consistency A videotaping procedure was used for this purpose for the 2007-2011 administrations The “second rater procedure” was piloted in 2011 and was fully implemented to replace the videotaping for the 2012 administration

18. Second Rater Procedures The second rater procedure involves having a second trained test administrator (a “second rater”) observe an SC-Alt administration and score the student’s responses on a separate second rater answer folder The second rater scores the student responses independently without consulting or comparing scores with the regular test administrator

19. Second Rater Procedures(cont.) The scoring by the regular test administrator is used as the official scoring for the student The scoring of the second rater is compared to the scoring of the regular test administrator by the scoring contractor to calculate measures of scoring consistency

20. Second Rater Pilot Results Results of the pilot in 2011 indicated that the second rater procedure yielded comparable results to the videotaping procedure The second rater procedure was also overwhelmingly preferred over videotaping by test administrators and DTCs-Alt

21. Advantagesof the Second Rater Procedure The second rater procedure eliminated the problems associated with using video cameras and recording media Teachers reported the procedure to be less stressful than videotaping Teachers reported the procedure was less distracting to students than videotaping.

22. Second Rater Procedures for 2012 • The teachers were sampled to conduct second rater administrations in the fall prior to test administrator training, allowing the DTC-Alt to assign and train a second rater for each administration • Each sampled teacher was required to conduct a second rater administration with one student for the mathematics administration only

23. Second Rater Sampling Procedures All districts were sampled Sampling was implemented by teacher and student Teachers were sampled according to proportions of students in their district Approximately 1/3 of teachers and 10% of students were sampled

24. Second Rater Qualifications • Must meet the test administrator criteria: • Certified teacher • Administrator (e.g., school administrator, district level special education consultant, or other administrator) • Related services personnel • Must participate in test administration training

25. Second Rater Results Previous results have indicated consistently high rates of scoring agreement at all three form levels (elementary, middle, and high school) For 2012, the average item agreement statistics for mathematics second rater samples were: Elementary form – 97.7%, Middle School form – 97.6%, and High School form – 97.0%. These results are consistent with the scoring consistency results for previous years and confirm a high level of scoring consistency at all three levels.

26. Second Rater Results Full implementation of the second rater procedure in 2012 was very successful Fewer problems were reported than with videotaping and the identification and training of second raters was not a problem for districts The return rate for complete and scorable second rater materials was significantly greater than for videotaping (94% vs. 79% for 2011)

27. 2013 Second Rater Procedures Teachers will be sampled to conduct second rater administrations from the 2012 SC-Alt data file and the names of the teachers will be provided to districts in November prior to regional test administrator training Second rater administrations for 2013 will be conducted in science only.

28. Teacher Comments I liked having the second-rater features of the test. Great change!” “Second-rater had trouble with monitoring eye gaze with non-familiar students.” Note: This was also problematic with the videotape raters. Appreciate SPQ and the security affidavit in the answer folder and revised answer worksheet.

29. 2013Administration

30. Changes for 2013 • Green border instead of green arrow to indicate correct response. • Updated “Try 2” statement • Change in TA “Say” statement • “Tell (show)” changed to “Show (tell)” • “Secure Materials” statement added to response options and other print manipulatives

31. Task Sharing Plan Share tasks and training materials with New Mexico Tasks will be reviewed by Content and Bias Committee in SC Tasks will be field-tested in SC Results in increasing bank of tasks so that we can replenish at a more rapid pace

32. Issues for Committee Consideration • Response Options • Purpose of Response Cards • Tense of verbs • Other Response Methods to Consider • Open ended • Other • Use of pictures/picture symbols in ELA

33. Participation Guidelines for SC-Alt Background and New Guidance on Eligibility Criteria

34. Participation Ratesof SWD in SC-Altand PASS Is Better Identification Guidance Needed ?

35. SC-Alt Participation 2007 – 2012

36. Summary of Changes in Rates of Participation • The overall number of students tested in 2012 increased by 256, which was a 8.1% increase over 2011. The increases for the previous two years were 7.9 and 6.6% respectively. • The steady increases since 2007 have been largely due to the increases for Autism and Mild ID. • The percentages of increase from 2007 to 2012 have been 93% for Autism and 62% for Mild ID, compared to an increase since 2007 of 15.7% for all other students.

37. Previous Efforts to ImproveIEP Team Testing DecisionsHave Been Ineffective • The increases for 2012in suspect categories of disabilities have occurred in spite of collaboration with the OEC and districts over the past two years.

38. 2012 Rates of Participation • In 2012, the number of Mild ID students approximately equaled the number of Moderate ID students for the first time. • Increases in Autism and Mild ID continued to be major contributors for 2012, however, there were also substantial increases for DD and OHI. • The number of LD students has continued to increase during the last three years and now constitutes 2.4% of SC-Alt students, contrary to what would be expected based on LD identification criteria.

39. Statewide Percent of All Studentsby Disability Group Taking the SC-Alt • Severe Intellectual Disability – 100% • Moderate Intellectual Disability – 96.5% • Autism – 36.9% • Mild Intellectual Disability – 30.8% • Traumatic Brain Injury – 26.7% • Orthopedic Disability – 23.8% • Developmental Delayed – 17.5% • Other Health Impaired – 2.8%

40. The 1% Rule for AA-AAS Assessments • The number of alternate assessment students counted as Proficient for AYP may not exceed 1% of all students tested (i.e., regular and alternate tested students) • 1% rule calculations are made at the district and state level, not at the school level

41. Statewide Calculations for Participation in SC-Alt • The percent of all tested students (SC-Alt and PASS) who were tested with SC-Alt in 2012 was 0.9% • 81.6% of SC-Alt students taking ELA scored Proficient (Achievement Levels 3 and 4) • The percent of all students (SC-Alt and PASS) scoring Proficient on SC-Alt ELA was 0.8% (significantly below the NCLB Reg of <= 1.0%)

42. Variation in District SC-Alt Identification • The percent of all students tested with SC-Alt was less than 1% for most districts • The percent of all students (SC-Alt and PASS) scoring Proficient on SC-Alt ELA was less than 0.8% for most districts • 27% of districts had high rates of participation related to problematic identification decisions

43. Variation in District SC-Alt Identification • The number of districts exceeding the 1% Proficient rule was 22 in 2012 and 23 in 2011, and almost all were the same districts • One-half of the districts exceeding 1% had percent proficient rates of 1.5% or higher • The percent of all students tested with SC-Alt in these districts ranged from 1.2% to 3.8%

44. Districts Exceeding the 1% Rulewith the Highest RatesNumber Tested and Percent of All Grade 3-8 Students:

45. Contrasts in District SC-Alt IdentificationNumber Tested and Percent of All Grade 3-8 Students:

46. Participation Guidelines Guidance for IEP Teams on Determining Participation in theSouth Carolina Alternate Assessment (SC-Alt) Manual Features

47. Definition ofSignificant Cognitive Disability Characterized by ability scores on both verbal and non-verbal scales that are at least 2 ½ ─ 3 standard deviations below the mean Deficits in adaptive behavior skills (skills that enable people to function effectively in their everyday lives) are at least 2 ½─3 standard deviations below the mean in two out of three domains.

48. Manual Features Initial Determination of Eligibility Participation Worksheet Review of Eligibility Participation Worksheet Eligibility Criteria Descriptors Description of Sources of Evidence

49. Committee Discussion Guidance for IEP Teams on Determining Participation in theSouth Carolina Alternate Assessment (SC-Alt)

50. Implementationof Common CoreState Standards