South Carolina Alternate Assessment

1. South Carolina Alternate Assessment DTC-Alt Pretest Workshop 2012-13 Web-based Training South Carolina Department of Education American Institutes for Research November 2012

2. Training Topics • New Procedures and Changes for 2013 • Review of 2012 Administration Results • New guidance on determining SC-Alt eligibility • Preparation for 2013 Administration • Testing Plan by Content Area • Test Administrator Training

3. New for 2013 • Guidance on determining SC-Alt eligibility including IEP team worksheet tools • Minor changes to test booklets and materials • Streamlined early identification of teachers sampled to conduct second rater administrations • Improvements for distribution of SCDE training and scoring video for district-level training

4. Review of the 2012 AdministrationResults 4

5. SC-Alt 2012 Participants • 3,431 students were tested in 2012, compared to 3,175 in 2011. • This was a 8.1% increase for 2012 (256 students), compared to increases of 7.9% for 2011 and 6.6% for 2010. • The largest increases were for Autism, Developmental Delay, Mild Intellectual Disability.

6. SC-Alt Participation2007 – 2012

7. SC-Alt 2012 Participation by Primary Disability

8. ELA Performancefor All Students2007 - 2012 8

9. Math Performancefor All Students2007 - 2012 9

10. Science Performancefor All Students2007 - 2012 10

11. Social Studies Performancefor All Students2008 - 2012 11

12. ELA Scores for SC-Alt 2012 by Disability Group 12

13. Math Scores for SC-Alt 2012 by Disability Group 13

14. Score Report User’s Guide • Provides an overview of the SC-Alt assessment and scoring • Describes the types and expected uses of SC-Alt scores • Includes descriptions of achievement levels • Guides the reader through the interpretation of each type of score report 14

15. Score Report User’s Guide(cont.) • Copies have been distributed to each school and DTC-Alt • A downloadable copy is posted on the alternate assessment Web page 15

16. Participation Guidelines for SC-Alt New Guidance on Eligibility Criteria 16

17. Statewide Percent of All Studentsby Disability Group Taking the SC-Alt • Severe Intellectual Disability – 100% • Moderate Intellectual Disability – 96.5% • Autism – 36.9% • Mild Intellectual Disability – 30.8% • Traumatic Brain Injury – 26.7% • Orthopedic Disability – 23.8% • Developmentally Delayed – 17.5% • Other Health Impaired – 2.8% 17

18. Statewide Calculations for Participation in SC-Alt The percent of all tested students (SC-Alt and PASS) who were tested with SC-Alt in 2012 was 0.9% 81.6% of SC-Alt students taking ELA scored Proficient (Achievement Levels 3 and 4) The percent of all students (SC-Alt and PASS) scoring Proficient on SC-Alt ELA was 0.8% (significantly below the NCLB Reg of <= 1.0%) 18

19. Variation in District SC-Alt Identification The percent of all students tested with SC-Alt was less than 1% for most districts The percent of all students (SC-Alt and PASS) scoring Proficient on SC-Alt ELA was less than 0.8% for most districts 27% of districts had high rates of participation related to problematic identification decisions 19

20. Contrasts in District SC-Alt IdentificationNumber Tested and Percent of All Grade 3-8 Students: 20

21. Participation Guidelines Guidance for IEP Teams on Determining Participation in theSouth Carolina Alternate Assessment (SC-Alt) 21

22. Definition ofSignificant Cognitive Disability Characterized by ability scores on both verbal and non-verbal scales that are at least 2 ½ ─ 3 standard deviations below the mean. Deficits in adaptive behavior skills (skills that enable people to function effectively in their everyday lives) are at least 2 ½─3 standard deviations below the mean in two out of three domains. 22

23. Manual Features Initial Determination of Eligibility Participation Worksheet Review of Eligibility Participation Worksheet Eligibility Criteria Descriptors Description of Sources of Evidence 23

24. 24

25. SC-Alt 2013Administration 25

26. Administration Window March 4 - April 26 Testing materials will arrive in the districts by February 21 26

27. Review of Recent Changesin Procedures and Materials 27

28. Response Options Updated in Test Booklet The “NR” symbol was changed to “N”. 28

29. Response Options Updatedin the Answer Folder • The “NR” bubble was changed to “N”, matching the change in the test booklet. • The “AL” bubble was changed to “A”. 29

30. Print Manipulatives Print manipulatives are now printed and packaged as strips. Three picsyms are placed on each strip. Each strip isassociated with a particular item. 30

31. for 2013 Changes in the test booklet: • Green border instead of green arrow to indicate correct response • Updated “Try 2” statement • Change in TA “Say” statement • “Tell (show)” changed to “Show (tell)” 31

32. for 2013 • “Secure Materials” statement has been added to response options and other print manipulatives 32

33. Packaging of Test Materials All test materials for each grade band are bundled into one Test Administrator Kit. Each kit contains a serial number. This number is also on the School Packing List and Teacher Security Checklist. 33

34. Test Administrator Kit • Test Booklets • Print Manipulatives • Physical Manipulatives • Reading Passages 34

35. Items Packed by TeacherNot Included in theTest Administration Kits • Teacher security checklist • Student rosters • Answer Folders (by grade-band form, color coded) • Scoring worksheets • Precoded Bar Code Labels 35

36. Test Administrator Security Affidavits Are included in the student answer folder. Must be signed by the teacher and by the assessment monitor for each content area administration and validated by the principal. 36

37. Materials Shipping Boxes • The size of the materials shipping boxes were reduced for easier handling • Only two test administrator kits are packed in each box 37

38. Web Secure File Transferof District Data Files • Beginning in 2011, a Web SFTP site was implemented for transfer of data files and other secure documents to the DTC-Alt • You will be notified whenever pertinent files or information are posted on the SFTP site for your access. 38

39. Assessing SC-Alt Scoring Fidelity Implementing the Second-Rater Procedure 39

40. Procedures Usedfor Monitoring SC-Alt Scoring Fidelity Special procedures are used to monitor test administration effectiveness and scoring consistency A videotaping procedure was used for this purpose for the 2007-2011 administrations The “second rater procedure” was piloted in 2011 and was fully implemented to replace the videotaping for the 2012 administration 40

41. Second Rater Procedures The second rater procedure involves having a second trained test administrator (a “second rater”) observe an SC-Alt administration and score the student’s responses on a separate second rater answer folder The second rater scores the student responses independently without consulting or comparing scores with the regular test administrator 41

42. Second Rater Procedures(cont.) The scoring by the regular test administrator is used as the official scoring for the student The scoring of the second rater is compared to the scoring of the regular test administrator by the scoring contractor to calculate measures of scoring consistency 42

43. Advantagesof the Second Rater Procedure The second rater procedure eliminated the problems associated with using video cameras and recording media Teachers reported the procedure to be less stressful than videotaping Teachers reported the procedure was less distracting to students than videotaping. 43

44. Second Rater Results Full implementation of the second rater procedure in 2012 was very successful Fewer problems were reported than with videotaping and the identification and training of second raters was not a problem for districts The return rate for complete and scorable second rater materials was significantly greater than for videotaping (94% vs. 79% for 2011) 44

45. 2013 Second Rater Procedures Teachers will be sampled to conduct second rater administrations from the 2012 SC-Alt data file and the names of the teachers will be provided to districts in November prior to regional test administrator training. Second rater administrations for 2013 will be conducted in science only. 45

46. Second RaterSampling Procedures All districts are sampled Sampling is implemented by teacher and student Teachers are sampled according to proportions of students in their district Approximately 1/3 of teachers and 10% of students will be sampled 46

47. Second Rater Qualifications • Must meet the test administrator criteria: • Certified teacher • Administrator (e.g., school administrator, district level special education consultant, or other administrator) • Related services personnel • Must participate in test administration training 47

48. Planning and Implementing the 2013 SC-Alt Administration 48

49. Importance of Precoding • Determines SC-Alt materials distribution, reduces additional material orders • Reduces errors related to: • Student demographic coding • SC-Alt form assignment • Science and Social Studies participation will be pre-assigned for precoded students, preventing the need for teacher/DTC-Alt assignment decisions 49

50. Critical to Precode Success Completion of all coding by the January extraction Use the “Home Room” field to identify the student’s SC-Alt teacher; verify that this is what is being done at each of your schools If you have schools that must put another teacher’s name in the “Home Room” field (not the SC-Alt teacher), then you will need to contact Doug Alexander for assistance with using an optional teacher coding procedure 50