MCAS Test Security and Administration Protocols: For Returning Staff

1. MCAS Test Security and Administration Protocols: For Returning Staff The Office of Student Assessment Services January 29, 2019

2. Presenters • Robert Pelychaty, Accommodations and Appeals Coordinator • David Ragsdale, Test Security Specialist • Jodie Zalk, Test Administration Coordinator • Swampscott MCAS Team: • Lytania Mackey Knowles • Director of Technology & Digital Learning • Catie Porter Roberts • Digital Learning Specialist & Professional Development Coordinator

3. Logistics for This Session • Questions may be asked at any time using the Questions feature on the GoToWebinar screen. • Questions about a specific student should be sent by email to mcas@doe.mass.edu. • We will attempt to answer all questions asked during the session; after the session, all questions will be answered and emailed out to participants. • This session is being recorded and will be available online in about one week at: • MCAS Resource Center ― mcas.pearsonsupport.com/training • In the section entitled “Training Sessions Previously Offered This School Year”

4. 01 06 Test Security Requirements Protocols for PBT and High Schools 02 Test Administration Protocols 03 Accessibility and Accommodations CONTENTS 04 A District’s Perspective 05 Resources, Support, and Next Steps

5. Test Security Requirements 01

6. Test Security Problems Worse than Yours www.cbsnews.com/news/indian-parents-scale-school-wall-to-help-students-cheat-on-exams/

7. Test Security ‘reductio ad absurdum’ www.telegraph.co.uk/news/newsvideo/weirdnewsvideo/10246867/University-students-made-to-wear-anti-cheating-helmets.html

8. Most Common Security Issues from 2018 • Cell phone use • Improper read-aloud or text-to-speech • Extra reference sheets/notes • Going forward or back to another session (PBT) • Student communication (talking or passing notes) • Copying another student’s work/answers • Accessing the Internet • Calculators on noncalculator tests/sessions

9. A Framework for the Test Security Plan • Prevent • The foundation of a test security plan is always prevention. • Detect • How do you detect incidents? • Respond • What do you do when an incident occurs?

10. The Importance of Leadership • People take their cues from above – leaders set the tone. • Superintendents • must review their principals’ test security plans and be comfortable with the procedures in their buildings (PAM, page 3) • are encouraged visit their schools and observe testing • Principals • must ensure that test administrators are properly trained • must ensure that MCAS protocols are being followed • must ensure that accommodations are correctly given • must sign the PCPA and attest to a proper test administration

11. Secure Content and Materials for CBT • Content • MCAS questions not publicly released by the Department • Student responses to test questions • Passages, questions, writing prompts, and any other on-screen content • Materials • Student testing tickets and proctor testing tickets • Used scratch paper after testing (until it is securely destroyed)

12. Confidentiality of Secure Test Content • Principals, test administrators, and others are prohibited from: • Viewing on-screen test content • exceptions for certain accommodations (nondisclosure form required) • exception for reading a word aloud for Mathematics and STE • exception for assisting a student with the computer • Duplicating or reproducing test content • technology staff cannot take photographs of computer screens. • Discussing test content with anyone before, during, or after testing

13. Handling and Storage of Secure Materials • Store all secure materials in a secure central location. • locked when tests are not being administered • restricted access • Maintain chain of custody of materials during test administration. • Use internal tracking forms (to be posted on DESE site soon). • Independent counts of secure materials • Do not leave materials unattended.

14. A Secure Testing Environment – Out of the Room • Students must be supervised when they are out of the testing room. • Plan for • hallway monitoring • test completion rooms, lunch, students using the restroom or going to the nurse • Optional/recommended scripts in TAMs to read prior to • the transition to a test completion room • a supervised lunch break • students visiting the restroom

15. A Secure Testing Environment – In the Room • Choose appropriate testing spaces. • Cover or remove prohibited classroom displays. • Remember this if you add new rooms later. • No unauthorized visitors in testing rooms • parents, media, non-testing students • teachers • Testing rooms may be entered by • Technology staff for troubleshooting • School administrators • District personnel • Department observers • When possible, use two test administrators in the room.

16. Secure Room Set-up for Computer-Based Testing • Students must not be able to view any screen but their own. • Set up rooms in advance to test out different seating arrangements. • Sit in the seats and look around. What can you see? • Have an administrator walk around at the start of testing and check all the rooms.

17. Examples of Seating Arrangements

18. Using Partitions between Computers

19. Prohibited Materials • Make sure students understand that they are not allowed to have • cell phones and other electronic devices (e.g., music players, e-readers, smart watches) • calculators on noncalculator tests and sessions • notes or reference material beyond what they’re given, such as additional reference sheets, graphic organizers, multiplication grids (exceptions for certain accommodations) • dictionaries • A poster showing examples of prohibited materials to be posted on DESE site soon

20. An Example of Collecting Cell Phones

21. Training Test Administrators and Other Personnel • All individuals involved in MCAS administration must participate in a school-provided training session. • MCAS test security requirements (Part I of the PAM) • MCAS test administration protocols (Part III of the PAM) • School-specific procedures and logistics • Training section of the PAM (Part III, section F) now focuses on areas where schools have discretion in implementing security protocols • Sample slides will be posted on DESE website soon.

22. Training Test Administrators and Other Personnel – Documentation • Documentation • Test administrators must sign forms to acknowledge the following: • participation in training • receipt of TAMs • combined sample form (to be posted on DESE site soon) • Test administrators who will administer certain accommodations to students with disabilities or EL students (listed in Appendix C of the PAM) must sign an MCAS Nondisclosure Acknowledgment form.

23. Test Administrator Responsibilities • Maintain the chain of custody of secure materials • Make sure students have the correct materials for each session • Focus their full attention on the testing room • Follow instructions in the TAMs and read scripts verbatim • Circulate throughout the room to prevent cheating and the use of prohibited materials – active proctoring • Refrain from coaching students or influencing their responses in any way

24. What is Coaching? • Providing hints or clues • “What did we study in class last week?” “What’s the acronym for order of operations?” “What’s the main character’s perspective?” “Don’t forget to use specific examples in your essay.” • Indicating in any way that a student has answered a question incorrectly • “Check your work” • Defining or spelling words • Explaining, simplifying, or paraphrasing any part of the test • Suggesting that a student write more • Influencing a student’s response through gestures, facial expressions, nods, or changes in voice inflection

25. What is Permitted? • Reading a word or short phrase aloud on the Mathematics and STE tests (UF 10) • Assisting students with technology-related problems • helping a student sign in to TestNav • pointing to a tool button that a student cannot find • helping a student understand how to enter a result in the equation editor • helping a student submit his or her test • Re-reading part of the TAM script to students • Telling students to do their best if they have questions about test content

26. Test Security Requirements for Students • Students’ results may be invalidated if they • use notes, books, or other reference materials • access a cell phone or other electronic device (other than their testing device) • look at another student’s screen or allow another student to see their screen • pass notes or talk to other students • access the Internet or use any application on their testing device other than TestNav • See sample form for students and a sample letter to send to parents (recommended for grades 6–8 and 10) • (to be posted on DESE site soon)

27. Steps to Take After a Security Incident or Irregularity • Call the Department at 781-338-3625. • Gather information about the incident if necessary. • Talk to the test administrator(s) involved. • Talk to any students involved. • Preserve documents (e.g., notes confiscated). • If reporting a read-aloud violation, determine how much of the test was read. • If reporting a calculator violation, determine which items were solved with the calculator. • Submit a report if instructed to. • Take any local action that is appropriate.

28. Test Administration Protocols 02

29. Transition to Next-Generation Tests for Grades 3–8 and 10 ELA and Math • * Paper test accommodation for students with disabilities or new ELs

30. Streamlined Manuals for 2019 • Manuals are expected to be available by the end of this week/beginning of next week. • Based on feedback on the 2018 school survey, we reorganized manuals for 2019. • TAMs combine scripts as much as possible for Math and STE.

31. 2019 Updates: Test Administration Manuals • PAM: Contains instructions for tests and HS field tests • CBT TAM: • All grades 3–8 tests • Grade 10 ELA and Mathematics • High school Biology and Introductory Physics field tests • PBT TAM: • All grades 3–8 tests • Grade 10 ELA and Mathematics • Operational high school STE tests

32. 2019 Updates: The Principal’s Administration Manual • New infographics and new icons – e.g., Part IV, Section A: new checklist and timelines • Appendix B: PBT instructions • Appendix C: accessibility and accommodations • Sample administration materials/forms moved online

33. 2019 Updates: Scheduling Guidance for Grades 3–8 • For make-up testing, schools may group students taking different tests together if the TAM script and the materials are the same. • Do not group together students doing CBT and PBT. • www.doe.mass.edu/mcas/cal.html

34. 2019 Updates: Grade 10 Test Schedule • Grade 10 ELA and Mathematics • Administer tests to the maximum number of students who can be tested concurrently on the first set of days. • Schools with technology constraints may use the second set of dates to test the remainder of their students. • Schools with special circumstances that cannot test all their students on these dates must email mcas@doe.mass.edu to discuss options.

35. Additional Scheduling Guidance • Short break mid-session for all tests with the exception of the legacy operational high school STE tests • TestNav will be available between 7:00 a.m. and 4:00 p.m., Monday–Friday, except the week of April 15. • Principals will need to request special access to test outside these hours (e.g., evening high schools). See PAM Part III, section C.

36. 2019 Updates: High School Biology and Intro Physics Field Tests Requirements Participation Notes • Administration window • May 28–June 14, except June 3–5 (Recommended date: June 6) • Participation guidelines • All grade 9 students taking the operational high school Biology or Introductory Physics test are expected to take the field test in the same subject (exceptions below). • Grade 10 participation is optional/at principals’ discretion. • Students not expected to participate: • first-year EL students • students who take the MCAS-Alt • repeating grade 9 students • Unavailable accommodations • A1 and EL1 – Paper-based edition • A2 – Large-print edition (Onscreen text can be magnified.) • A3.1 and A3.2 – Screen reader and Braille edition • Unique Accommodation – Use of a compatible assistive technology device

37. 2019 Updates: Principal’s Certification of Proper Test Administration • The PCPA is streamlined to reflect the shift to computer-based testing. • The “before testing” section is removed, and schools will complete the PCPA only after testing as shown in the table:

38. 2019 Updates: Administration Procedures • Scratch paper for all tests • New online form for requesting an alternate setting • PAM Part II, section E • Student questionnaire now administered at grade 4 (as well as grades 5, 8, and 10 as before) • Data will be based on data from the School Interoperability Framework (SIF) for the SR/PNP for districts that use SIF. For non-SIF districts, data will be based on the October SIMS submission.

39. Preparing Students for CBT • Technology skills and tools • DLCS Curriculum Framework • Student Tutorial • Demonstration of the navigation, tools, and features for CBT • Students complete it independently (no audio) • Practice tests • Simulation of the tools and features for CBT • Include accessibility features and accommodated test forms (e.g., TTS) • Sample student work from the spring 2018 MCAS tests • Answer keys and scoring rubrics • Math and STE: Equation editor guides and reference sheets • Resources for parents/guardians

40. PearsonAccess Next User Roles • PAN user accounts can be created with an “Active End Date.” Once that date has passed, a user cannot access his or her account. • District or school test coordinators should review accounts for all staff from 2018 to update the Active End Date if necessary. • Remove or change the Active End Date by file upload or through the user interface. • Recommended: Users should update their passwords after the Active End Date has been updated.

41. Training the Test Administration Team in CBT Procedures • Technology staff need to assist with set-up and troubleshooting. • See list of tasks in PAM Part III, section E • Training for technology coordinators on February 12 (Register at www.doe.mass.edu/conference/?ConferenceID=9954.) • Precache test content in mid-March. • Schedule district support techs for testing days whenever possible. • Familiarize test administrators with tasks in PearsonAccess Next. • List of modules and tasks to complete in PearsonAccess Next in the PAM in Part III, section F • Expanded troubleshooting steps in PAM Appendix A. • If there is a technology situation in which a student is waiting more than 15 minutes, then schedule the student to complete the session at another time.

42. Hardware considerations • Power strips • Battery adapters for mobile devices • Extension cords • Extra devices for each testing area • Headphones required for text-to-speech accommodation • External keyboards strongly recommended for tablets

43. Infrastructure Trials Infrastructure Trials Confirm Main Steps • TestNav is configured correctly. • ProctorCache software is set up correctly. • Devices can successfully run TestNav and that online test content can pass through the content filters at the school. • The network can bear the full load. • Participating staff understand their roles for MCAS CBT. • Students are familiar with the CBT tools/format. • Set up and configure the test management and test delivery environment. • Generate sample students and create PAN Sessions. • Administer practice tests. • Conduct follow-up activities and review lessons learned with the test administration team.

44. Questions & Answers (Use the “Questions” feature to ask questions.)

45. Accessibility and Accommodations 03

46. MCAS Accessibility and Accommodations • See the Accessibility and Accommodations Manual at www.doe.mass.edu/mcas/accessibility/ • View recordings of previous trainings at mcas.pearsonsupport.com/training/: • January 15: Accessibility and accommodations training • January 24: SR/PNP training

47. MCAS Accessibility and Accommodations • Accessibility features available to all students: • Universal Accessibility Features (may differ, depending on CBT or PBT) • Designated Accessibility Features (at discretion of principal) • Accommodations available only for students with disabilities and ELs • Standard and Special Access accommodations for students with disabilities • Accommodations for ELs, with guidance based on English proficiency level • Must pre-order the accommodations in SR/PNP if listed with an “SR/PNP” designation in Accessibility and Accommodations Manual (e.g., TTS, large print, paper, Braille)

48. Test Preparation STUDENTS should: • Become familiar with features and basic functionality of TestNav (the computer-based test platform) AND • Take online practice tests prior to test administration AND • Review student tutorial at mcas.pearsonsupport.com. SCHOOLS should: • Plan ahead for testing students with accommodations. • Schedule testing locations, test administrators, and testing time. • Set up specialized computers/AT after reviewing IEPs, 504 plans, and local EL accommodations forms.

49. Important Reminders for CBT • Test administrators for Human Read-Aloud and Human Signer accommodations will use their own computer, print their own proctor testing ticket, and access a “test administrator” test edition. • A separate PAN Session must be created. • Responses are not saved when using a proctor testing ticket.

50. Prohibitions for Accommodations Test administrators may not: Students’ results may be invalidated for these situations: • Provide accommodations not listed in • a student’s IEP or 504 plan • Appendix C of the PAM, without Department approval • Provide a student with a modified version of a test or a test for a different grade. • Coach or assist a student with their responses • E.g., “Write more” or “Go back and review” • Provide clues/assistance/definitions • Using an English-language dictionary for ELA • Reading aloud the ELA test to a student who does not have this Special Access accommodation in his/her IEP • Giving a calculator on noncalculator session of the Mathematics test to a student who does not have this Special Access accommodation in his/her IEP • Test administrator coaching or assisting a student