Spring Test Administration Training Session

1. Spring TestAdministration Training Session February 2013 Includes information on MCAS, MCAS-Alt, and ACCESS for ELLs

2. Presenters • Dan WienerAdministrator of Inclusive Assessment • Jodie ZalkCoordinator of MCAS Test Administration • Bob LeeMCAS Chief Data Analyst Massachusetts Department of Elementary and Secondary Education

3. Today’s Agenda • Review materials in folders • Updates for 2013 • Test security and test administration protocols • MCAS and ACCESS for ELLs participation requirements • Students with disabilities • ELL students • Reporting • Schedules • Edwin Analytics • Historically anchored growth percentiles • Additional information and orientation for new principals Massachusetts Department of Elementary and Secondary Education

4. Updates for 2013 • MA Secretary of Education — Matthew Malone • Changes in reporting growth • Updates for high school principals • March retest questions will not be released. • Adams requirement will include STE for grade 9 students. • ACCESS for ELLs implementation • Communication with schools and districts • Student Assessment Updates email Massachusetts Department of Elementary and Secondary Education

5. Test Security

6. Why Is MCAS Security Important? • To protect the validity and integrity of test results “When MCAS scores are invalidated, parents, students, educators, and the community are deprived of important information about student achievement...” --Commissioner Mitchell D. Chester, Spring 2012 Massachusetts Department of Elementary and Secondary Education

7. Invalidations of MCAS Results * Of these students, 103 had results invalidated because they used electronic devices, including cell phones. Massachusetts Department of Elementary and Secondary Education

8. Top 10 Types of Security Violations in Spring 2012 • Cell phone • Improper use of the read-aloud accommodation • Improper calculator use • Passing notes • Improper assistance from test administrator • Cheat sheet • MP3 player • Tweeting • Working in wrong session • Posting to Facebook Massachusetts Department of Elementary and Secondary Education

9. Importance of Leadership • Superintendents must review their principals’ test security plans. • Superintendent’s Assurance of Proper Test Administration (optional) — PAM page 116 • Principals must maintain the security of the testing environment and test materials. • PCPA certification — PAM pages 114–115 • Train test administrators and others, and ensure they comply with requirements. • Meet with students. Massachusetts Department of Elementary and Secondary Education

10. A Secure Testing Environment • Do NOT schedule breaks mid-session • Plan for • appropriate testing spaces • students who may need more time • lunch, recess, restrooms • Cover/remove prohibited classroom displays • PAM page 23 • No visitors Massachusetts Department of Elementary and Secondary Education

11. Security Requirements for Test Materials • Store all test materials in a central location each day. • Locked when tests are not being administered • Restricted access • Maintain chain of custody of materials during test administration. • Do not remove test materials from school. • Do not leave test materials unattended. Massachusetts Department of Elementary and Secondary Education

12. Confidentiality of Booklets Do NOT … • review test booklets or answer booklets before, during, or after testing • exceptions for certain accommodations • provide students access to tests before testing • duplicate test materials • retain, recycle, remove, or destroy test materials Massachusetts Department of Elementary and Secondary Education

13. Case Study #1 The test coordinator prepares crates of testing materials for all the test administrators. She counts the number of booklets twice and matches the number to each room roster. The test administrators pick up their crates and take them to their classrooms. Is this sufficient? Massachusetts Department of Elementary and Secondary Education

14. Is this sufficient? • Yes. Materials were counted twice and quantities were matched to room rosters. • No. Test administrators should also have counted materials (in addition to the test coordinator). Also, tracking forms should have been used.   Massachusetts Department of Elementary and Secondary Education

15. Train Test Administrators and Others • All individuals involved in test administration must participate in training. • See topics in PAM — pages 26–28 • Forms requiring signoff by test administrators • Participated in training • Received their TAMs — see PAM page 121 • Signed a nondisclosure agreement if providing certain accommodations — see PAM page 119 Massachusetts Department of Elementary and Secondary Education

16. Test Administrator Responsibilities • Follow directions in TAMs and read scripts verbatim. • Supervise students at all times to prevent cheating. • Prevent use of prohibited materials (e.g., cell phones). • Focus full attention on the testing room. • Do not coach students or alter responses. Massachusetts Department of Elementary and Secondary Education

17. Case Study #2  During Session B of the Composition, some students turn in their completed booklets. While the remaining students continue working, the test administrator reads some of the compositions to see how well the students used the writing techniques they were taught. Is this permitted? Massachusetts Department of Elementary and Secondary Education

18. Is this permitted? • No. Test administrators cannot look at student responses or secure test content. • Yes. One of the purposes of MCAS is to inform instruction. The test administrator acted appropriately.   Massachusetts Department of Elementary and Secondary Education

19. Provide Clear Expectations to Students • Students must not • preview test materials • copy others’ work • accept any coaching or let someone else answer for them • use materials that are prohibited during testing • TAMs instruct students • to work until end of session (and only in that session) • to answer all questions in the session • Also instruct students to • try their best • respond directly to the MCAS ELA Composition prompt • Sample form in PAM (page 122) Massachusetts Department of Elementary and Secondary Education

20. Results will be invalidated for students who use cell phones or other electronic devices at any time during a session, including after a student turns in his or her test materials, during a break, or during the transition to a test completion area. Massachusetts Department of Elementary and Secondary Education

21. Case Study #3 A test administrator notices a student has finished working but hasn’t turned in her test materials. The test administrator notices that the student has her phone out. The test administrator confiscates the phone, collects the student’s test materials, and sends the student to the principal’s office. Upon questioning, the student admits that she took a photo of a drawing she had made in her test booklet, and posted the photo to Facebook. What should the principal do? Massachusetts Department of Elementary and Secondary Education

22. What should the principal do? • Print a copy of the photo. • Have the student delete the photo from Facebook and from the phone. • Determine whether the photo was disseminated in another way (e.g., email, texting). • Call the Department immediately. Massachusetts Department of Elementary and Secondary Education

23. How to Avoid This Situation • Inform students about the cell phone policy. • Inform students of appropriate activities at the end of a session (read a book). • Have test administrators collect test materials when students are no longer working. • Schedule sessions for appropriate lengths of time. Massachusetts Department of Elementary and Secondary Education

24. Tips to Maintain Security (Recommended) • Use seating charts. • Have an empty seat between students. • Have test administrators proctor other teachers’ classes. • Have two test administrators in each room. Massachusetts Department of Elementary and Secondary Education

25. Please contact the Department at 781-338-3625 IMMEDIATELY to report testing irregularities. • See PAM page 7 for more information about reporting testing irregularities. Massachusetts Department of Elementary and Secondary Education

26. Test Administration Protocols

27. Before Materials Arrive • Determine logistics. • Security plan • Testing dates and deadlines • Testing locations and staffing plan (including special populations) • Prepare record of test administrators and students • Check the Department’s website. • Practice tests • Read the PAM. • Student participation guidelines Massachusetts Department of Elementary and Secondary Education

28. Review Data for March SIMS • Students’ names • changes • Demographic information • first-year LEP, IEP, and 504 status • Grade levels Students will be reported as absent if they are enrolled but not tested. Massachusetts Department of Elementary and Secondary Education

29. Order June STE Materials: March 11–15 at www.mcasservicecenter.com Massachusetts Department of Elementary and Secondary Education

30. Receive Test Materials • Complete forms. • Internal tracking forms (sample – PAM pages 117–118) • Materials Summary • Principal’s Certification of Proper Test Administration (PCPA) – www.mcasservicecenter.com • Order additional materials if necessary. • Deadlines listed on PAM pages iii–v • Prepare materials for distribution. • Affix Student ID Labels to answer booklets. Massachusetts Department of Elementary and Secondary Education

31. Be Sure to… • Distribute and track test materials. • Monitor and coordinate test administration. • Securely store materials between sessions and after each day of testing. • Remind test administrators to read scripts verbatim. • Keep a log of test administrators and students. Massachusetts Department of Elementary and Secondary Education

32. Prepare Materials for School Files • Retain your school’s test administration files for three years. • Records may be requested in the event of an investigation. • Lists of materials to retain: • PAM pages 50–51, 68–69 Massachusetts Department of Elementary and Secondary Education

33. Test Administration Updates • Students write the test administrator’s name on the answer booklet cover – not on the test booklet • Exception: Test booklet for ELA Composition Session A • Principals keep reason for change of enrollment status on file – not reported on the answer booklet • Non-grade-10 status reported in SIMS – not on the answer booklet • Updates to scripts in TAMs • Cell phones • Testing time Massachusetts Department of Elementary and Secondary Education

34. Prepare the Scorable Shipment • Assign an answer booklet for • students who participated in ANY part of testing • students who were absent with medical documentation • first-year ELLs • Transcribe responses if necessary • Prepare envelopes • Return, Special Handling, Void • Packing diagrams and instructions in PAM • ELA: PAM pages 46–47 • Math and STE: PAM pages 64–65 Massachusetts Department of Elementary and Secondary Education

35. Complete Forms after Testing • Internal tracking forms • Materials Summary • PCPA • Signature must match the name of the principal in School/District Profiles (http://profiles.doe.mass.edu). Massachusetts Department of Elementary and Secondary Education

36. Options for Returning Materials (March–April and May) Massachusetts Department of Elementary and Secondary Education

37. Schedule Your School’s Pickup • Two methods: • Schedule online at mcasservicecenter.com. • One day prior, and by 3:00 p.m. OR • Give cartons to your regular UPS driver. • Call to inform the MCAS Service Center (800-737-5103). • Grade 10: Call the Service Center to cancel early pickup if using regular return. • Do not take materials to UPS. • Do not leave materials unattended. • Use UPS labels specific for each administration. Massachusetts Department of Elementary and Secondary Education

38. Questions & AnswersShort Break

39. MCAS and ACCESS for ELLs Participation Requirements for Students with Disabilities

40. Eligibility for Test Accommodations for Students with Disabilities • IEP or 504 plan must specify the accommodations to be used for testing. • Accommodations may not be provided if IEP has not yet been signed by the student’s parent/guardian. • Accommodations may be provided if a 504 plan is being developed for a student. Massachusetts Department of Elementary and Secondary Education

41. Prohibitions for Test Accommodations • Providing student with unapproved accommodations or those not listed in the PAM • Allowing student to take a different grade-level test • Violating test security • Providing student with an altered or photocopied test • Coaching or providing clues or other assistance (e.g., defining words, sending student back to write more) • Allowing prohibited materials (e.g., dictionary on ELA Reading Comprehension test) Massachusetts Department of Elementary and Secondary Education

42. Plan Ahead to Provide Accommodations on Test Day • Review IEPs, 504 plans, and PAM Appendix B ahead of time. • Prepare accommodations spreadsheet: Who? Which accommodation(s)? Where? • Ensure sufficient quantities of special test editions (large-print, Braille, Kurzweil, ASL DVD). • Provide only Department-approved ELA graphic organizers and individualized mathematics reference sheets. • Train all test administrators who will provide accommodations, including substitutes. Massachusetts Department of Elementary and Secondary Education

43. Invalidating MCAS Test Results for Inappropriate Accommodation Use • Test results may be invalidated if a student is provided with an accommodation not listed in his or her IEP or 504 plan. • Cautions • Accommodation 16 vs. 26 (read aloud) • Accommodation 20 (approved graphic organizer or mathematics reference sheet) • Accommodation 30 (calculator) • Dictionaries only permitted for the ELA Composition • Refer to last approved IEP or 504 plan. Massachusetts Department of Elementary and Secondary Education

44. If a Student Refuses an Accommodation • Document refusal in student’s file. • Do not ask the student to sign a statement. • Continue to make accommodation available for remainder of test administration. • Amend IEP/504 plan for future testing. • It may be appropriate to list the accommodation in the IEP/504 plan “as requested by student” or remove it. • Do not fill in circles on the student’s answer booklet (back cover) for accommodations that were refused. Massachusetts Department of Elementary and Secondary Education

45. Nonstandard Accommodations (NSAs) • NSAs may be provided only if student • is virtually unable to decode, calculate, write, or spell; AND • receives ongoing intervention in this area. • NSAs may not be provided if student is simply performing “below grade level.” • Score may be invalidated if ineligible student was provided an NSA. • Use of NSAs reported on Parent/Guardian Reports and school and district rosters Massachusetts Department of Elementary and Secondary Education

46. Data Collection for Students with Disabilities • 504 information is collected through SIMS, instead of on the answer booklet. • Answer booklets: • Fill in only those accommodations that were used. • Accommodations that are not applicable for a specific test are not listed on the answer booklet for that test. • e.g., accommodation 30 (calculator) is not listed for ELA tests Massachusetts Department of Elementary and Secondary Education

47. MCAS-Alt General Information • IEP/504 teams annually designate students for MCAS-Alt (decision made in each subject). • Encourage teachers to review the Educator’s Manual and attend Department training. • Consider flexible scheduling and classroom coverage to allow teachers to work on portfolios. Massachusetts Department of Elementary and Secondary Education

48. MCAS-Alt Important Dates Massachusetts Department of Elementary and Secondary Education

49. Schools’ Retention of Returned Portfolios • Portfolios returned to your school become part of the student’s temporary record. • Temporary records must be destroyed within 7 years after student transfers or graduates/exits. (Student Record Regulations) • However, temporary records may be destroyed when they become outdated or irrelevant. • Provided that parent/guardian is notified in writing and given opportunity to receive portfolio prior to its destruction Massachusetts Department of Elementary and Secondary Education

50. Suggested Portfolio Retention Schedule Massachusetts Department of Elementary and Secondary Education