South Carolina Alternate Assessment

1. South Carolina Alternate Assessment DTC-Alt Pretest Workshop 2011-12 Web-based Training South Carolina Department of Education American Institutes for Research November 2011

2. Training Topics • New Procedures and Changes for 2012 • Review of 2011 Administration Results • Preparation for 2012 Administration • Testing Plan by Content Area • Test Administrator Training

3. New for 2012 • The Second Rater Procedure replaces videotaping. • Answer Folder changes • Materials shipment packaging adjustments are being considered. • Improvements for distribution of SCDE training and scoring video for district-level training

4. Review of the 2011 AdministrationResults 4

5. SC-Alt 2011 Participants • 3,175 students were tested in 2011, compared to 2,942 in 2010. • This was a 7.9% increase for 2011 (233 students), compared to increases of 6.6% for 2010 and 4.1% for 2009. • The largest increases were for Mild Mental Disability and Autism.

6. Changes in Rates of Participation • Autism students increased by 14.0%, compared to an 18.3% increase last year. • Mild MD students increased by 17.0%, compared to a 7.4% increaselast year. • The percentages of increase from 2007 to 2011 have been 72% for Autism and 55% for Mild MD, compared to an increase since 2007 of 7% for all other students. • Moderate Mental Disability students actually decreased (-45 students; -4.9%), which follows similar decrease from 2009 to 2010.

7. Alternate Assessment Participation 2006 – 2011* *PACT-Alt/HSAP-Alt 2006; SC-Alt 2007-2011

8. SC-Alt 2011 Participation by Primary Disability

9. ELA Performance for All Students2007 - 2011

10. Math Performance for All Students2007 - 2011

11. Science Performance for All Students2007 - 2011

12. Social Studies Performance for All Students in 2008 - 2011

13. Score Report User’s Guide • Provides an overview of the SC-Alt assessment and scoring • Describes the types and expected uses of SC-Alt scores • Includes descriptions of achievement levels • Guides the reader through the interpretation of each type of score report

14. Score Report User’s Guide • Copies have been distributed to each school and DTC-Alt • A downloadable copy is posted on the alternate assessment Web page

15. Feedback on Procedures Implemented in 2011

16. Print Manipulatives Print manipulatives were printed and packaged as strips. Three picsyms were placed on each strip. Each strip isassociated with a particular item.

17. Packaging of Test Materials All test materials for each grade band were bundled into one Test Administrator Kit. Each kit contained a serial number. This number was also be on the School Packing List and Teacher Security Checklist.

18. Test Administrator Kit • Test Booklets • Print Manipulatives • Physical Manipulatives • Reading Passages

19. Items Packed by Teacher Not Included in the Test Administration Kits • Teacher security checklist • Student rosters • Answer Folders (new configuration - by grade-band form, color coded) • Scoring worksheets • Precoded Bar Code Labels

20. Test Administrator Security Affidavits Are included in the student answer folder. No pre-code or linking labels are needed. Must be signed by the teacher and by the assessment monitor for each content area administration and validated by the principal.

21. Web Secure File Transfer of District Data Files • Implemented for SC-Alt for 2011 data file distribution using a Web SFTP site • We expect to use the site for other document transfers in the future. • Provides a secure alternative for communicating confidential information to and from AIR

22. SC-Alt 2012 Administration

23. Administration Window March 5 - April 27 Testing materials will arrive in the districts by February 23 23

24. What’s New for 2012?

25. New for 2012 • Changes to the Answer Folder • The teacher name field and other fields that were previously on page 2 (back side of cover) have been moved to page 3. • Within the scoring grids, the “AL” (access limited) bubble is now simply “A,” and the “NR” (no response) bubble is now “N”. • A Homeless code box has been added. • The Student’s Learning Characteristics questionnaire has been deleted.

26. Response Options Updated in Test Booklet The “NR” symbol has now been changed to “N”.

27. Response Options Updated in the Answer Folder • The “NR” bubble has now been changed to “N”. • The “AL” bubble has now been change to “A”.

28. New for 2012 • Shipping box sizes may be adjusted following feedback from you today • Use of a Second Rater will replace videotaping

29. Maximum Shipping Box Size • In 2011, schools with four or more teachers received boxes containing four of the TA kits, weighing approximately 52 pounds • Should we eliminate the 4-kit box and pack only a maximum of two kits in a box?

30. Assessing SC-Alt Scoring Fidelity Implementing the Second-Rater Procedure

31. The Second-Rater Procedure Replaces videotaping for capturing administration and scoring fidelity for the 2012 administration. A sample of teachers/students will participate using the same sampling procedure used previously with videotaping (approximately 1/3 of teachers and 10% of students). Math is the only content area included.

32. What is the Second-Rater Procedure? • For the teacher and student sampled, an SC-Alt trained teacher or other staff member (the “second rater”) must observe the mathematics administration. • The second rater scores the student responses at the same time as the TA and records the scores on a separate Second Rater Answer Folder. • The TA scoring is the official scoring record; the second rater scoring is collected to compute scoring consistency.

33. Why Change to the Second-Rater procedure? • The second-rater procedure was piloted by 48 teachers in 25 districts during the spring 2011 administration. • The test administrators, second-rater observers, and DTCs-Alt all provided positive feedback and a preference for using the second-rater procedure over videotaping.

34. Second Rater Advantages Reported from the Pilot • Teachers reported the procedure to be less stressful than videotaping. • The procedure was less distracting to students than videotaping. • Logistical and technical problems associated with videotaping equipment and return of tapes/CD media are eliminated.

35. Second Rater Advantages Reported from the Pilot • In some cases the second rater was able to observe some administration problems related to teacher preparation. • In these instances, the procedure provides information about the need to improve training for test administrators.

36. Second Rater Sampling Procedures All districts are sampled. Sampling is implemented by teacher and student. Teachers are sampled according to the proportion of students in the district. One student per sampled teacher will receive a second-rater administration in the area of mathematics only. Approximately 1/3 of teachers and 10% of students will be sampled.

37. Second Rater Qualifications • Must be a certified teacher or administrator (e.g., teacher, school administrator, district level special education consultant or other administrator). • Must meet the regular SC-Alt test administrator training criteria.

38. Second Rater Qualifications • A teaching assistant may not serve as a second rater. • The second rater may serve as the testing monitor for the second-rater administration. • The DTC-Alt must approve the selection of the second rater and may select second raters based on the district’s particular needs or availability of staff.

39. When Are Teachers Sampled for Second-Rater Administrations? Early Identification Option: • The DTC-Alt may submit the names of all teachers testing students in spring 2012 by November 17. • The district will receive a list of sampled teachers prior to the close of registration for regional test administrator training. • The early identification will not specify the particular student; student names will be provided after the Precode file submission.

40. When Are Teachers Sampled for Second-Rater Administrations? Without Using the Early Identification Option: • The DTC-Alt will receive a listing of the teachers sampled for second-rater administrations during the first week of February, which will be based on the January Precode file submission.

41. Planning and Implementing the 2012 SC-Alt Administration

42. Student Identification • IEP committee decision based on participation guidelines • Criteria designed to identify students with the most significant cognitive disabilities • Not an appropriate assessment alternative for students instructed with the PASS curriculum and capable of PASS assessment

43. Importance of Precoding • Determines SC-Alt materials distribution, reduces additional material orders • Reduces errors related to: • Student demographic coding • SC-Alt form assignment • Science and Social Studies participation will be pre-assigned for precoded students, preventing the need for teacher/DTC-Alt assignment decisions

44. Critical to Precode Success • Completion of all coding by the January extraction • Use the “Home Room” field to identify the student’s SC-Alt teacher; verify that this is what is being done at each of your schools • If you have schools that must put another teacher’s name in the “Home Room” field (not the SC-Alt teacher), then you will need to contact Doug Alexander for assistance with using an optional teacher coding procedure

45. SC-Alt Spring 2012 Operational Assessments • English Language Arts • Mathematics • Science (elementary and middle school) • Biology (high school) • Social Studies 45

46. Assignment to Grade-Band Forms (Elementary, Middle, and High School) Students are assigned to SC-Alt grade-band forms on the basis of their age as ofSeptember 1, 2011, as pre-coded in the PowerSchool data base For students who do not have a pre-assigned form, refer to the Age Reference Sheet in the TAM or on the SCDE website to determine the correct form 46

47. Administration Plan by Form and Grade-Age

48. Science and Social Studies Administration Sampling (Elementary and Middle School Forms) • For students of ages 8, 10, 11, and 13 on September 1 • Random assignment of 50% of students to science and 50% to social studies • Precoded students will be pre-assigned, with science or social studies specified on the student roster

49. Science/Social Studies Assignment Non-Precoded Students • The student’s age and answer folder number will be used by the test administrator to determine science and/or social studies participation. • Teachers will need to carefully calculate student age and follow the assignment rule. • The DTC-Alt is encouraged to follow up to prevent errors.

50. Science/Social Studies Assignment Non-Precoded Students • Students aged 9 and 12 must be administered both science and social studies. • Students aged 8, 10, 11, and 13 will be assigned to science or social studies based on each student’s answer folder security number. • The assignment rule will be covered in regional and district-level training .