MCAS Test Security and Administration Protocols: For New Staff

1. MCAS Test Security and Administration Protocols: For New Staff The Office of Student Assessment Services February 5, 2019

2. Presenters • David Ragsdale, Test Security Specialist • Dan Wiener, Administrator of Inclusive Assessment • Jodie Zalk, Test Administration Coordinator • Swampscott MCAS Team: • Lytania Mackey Knowles • Director of Technology & Digital Learning • Catie Porter Roberts • Digital Learning Specialist & Professional Development Coordinator

3. Logistics for This Session • Questions may be asked at any time using the Questions feature on the GoToWebinar screen. • Questions about a specific student should be sent by email to mcas@doe.mass.edu. • We will attempt to answer all questions asked during the session; after the session, all questions will be answered and emailed out to participants. • This session is being recorded and will be available online in about one week at: • MCAS Resource Center ― mcas.pearsonsupport.com/training • In the section entitled “Training Sessions Previously Offered This School Year”

4. 01 06 Test Security Requirements Protocols for PBT and High Schools 02 Test Administration Protocols 03 Accessibility and Accommodations CONTENTS 04 A District’s Perspective 05 Resources, Support, and Next Steps

5. Test Security Requirements 01

6. Test Security Problems Worse than Yours www.cbsnews.com/news/indian-parents-scale-school-wall-to-help-students-cheat-on-exams/

7. Test Security ‘reductio ad absurdum’ www.telegraph.co.uk/news/newsvideo/weirdnewsvideo/10246867/University-students-made-to-wear-anti-cheating-helmets.html

8. Most Common Security Issues from 2018 • Cell phone use • Improper read-aloud or text-to-speech • Extra reference sheets/notes • Going forward or back to another session (paper-based testing) • Student communication (talking or passing notes) • Copying another student’s work/answers • Accessing the Internet • Calculators on noncalculator tests/sessions

9. A Framework for the Test Security Plan • Prevent • The foundation of a test security plan is always prevention. • Detect • How do you detect incidents? • Respond • What do you do when an incident occurs?

10. The Importance of Leadership • People take their cues from above – leaders set the tone. • Superintendents • must review their principals’ test security plans and be comfortable with the procedures in their buildings (Principal’s Administration Manual - PAM, page 3) • are encouraged visit their schools and observe testing • Principals • must ensure that test administrators are properly trained • must ensure that MCAS protocols are being followed • must ensure that accommodations are correctly given • must sign the Principal’s Certification of Proper Test Administration (PCPA) and attest to a proper test administration

11. Secure Content and Materials for CBT • Content • MCAS questions not publicly released by the Department • Student responses to test questions • Passages, questions, writing prompts, and any other on-screen content • Materials • Student testing tickets and proctor testing tickets • Used scratch paper after testing (until it is securely destroyed)

12. Student Testing Ticket

13. Confidentiality of Secure Test Content • Principals, test administrators, and others are prohibited from: • Viewing on-screen test content • exceptions for certain accommodations (nondisclosure form required) • exception for reading a word aloud for Mathematics and STE tests (UF10) • exception for assisting a student with the computer • Duplicating or reproducing test content • Technology staff may not take photographs of computer screens. • Discussing test content with anyone before, during, or after testing

14. Handling and Storage of Secure Materials • Store all secure materials in a secure central location. • locked when tests are not being administered • restricted access • Maintain chain of custody of materials during test administration. • Use internal tracking forms (to be posted on DESE site soon). • Independent counts of secure materials • Do not leave materials unattended.

15. A Secure Testing Environment – Out of the Room • Students must be supervised when they are out of the testing room. • Plan for • hallway monitoring • test completion rooms, lunch, students using the restroom or going to the nurse • Optional/recommended scripts in Test Administrator’s Manuals (TAMs) to read prior to • the transition to a test completion room • a supervised lunch break • students visiting the restroom

16. A Secure Testing Environment – In the Room • Choose appropriate testing spaces. • Cover or remove prohibited classroom displays. • Remember this if you add new rooms later. • No unauthorized visitors in testing rooms • parents, media, non-testing students • teachers • Testing rooms may be entered by • Technology staff for troubleshooting • School administrators • District personnel • Department observers • When possible, use two test administrators in the room.

17. Secure Room Set-up for Computer-Based Testing • Students must not be able to view any screen but their own. • Set up rooms in advance to test out different seating arrangements. • Sit in the seats and look around. What can you see? • Have an administrator walk around at the start of testing and check all the rooms.

18. Examples of Seating Arrangements

19. Using Partitions between Computers

20. Prohibited Materials • Make sure students understand that they are not allowed to have • cell phones and other electronic devices (e.g., music players, e-readers, smart watches) • calculators on noncalculator tests and sessions • notes or reference material beyond what they’re given, such as additional reference sheets, graphic organizers, multiplication grids (exceptions for certain accommodations) • dictionaries • A poster showing examples of prohibited materials to be posted on DESE site soon

21. An Example of Collecting Cell Phones

22. Training Test Administrators and Other Personnel • All individuals involved in MCAS administration must participate in a school-provided training session. • MCAS test security requirements (Part I of the PAM) • MCAS test administration protocols (Part III of the PAM) • School-specific procedures and logistics • Training section of the PAM (Part III, section F) now focuses on areas where schools have discretion in implementing security protocols • Sample slides will be posted on DESE website soon.

23. Training Test Administrators and Other Personnel – Documentation • Test administrators must sign forms to acknowledge the following: • participation in training • receipt of TAMs • combined sample form (to be posted on DESE site soon) • Test administrators who will administer certain accommodations listed in Appendix C of the PAM to students with disabilities or English learner (EL) students must sign an MCAS Nondisclosure Acknowledgment form.

24. Test Administrator Responsibilities • Maintain the chain of custody of secure materials • Make sure students have the correct materials for each session • Focus their full attention on the testing room • Follow instructions in the TAMs and read scripts verbatim • Circulate throughout the room to prevent cheating and the use of prohibited materials – active proctoring • Refrain from coaching students or influencing their responses in any way

25. What is Coaching? • Providing hints or clues • “Think about what we studied in class last week.” “What’s the acronym for order of operations?” “Consider it from the main character’s perspective.” “Don’t forget to use specific examples in your essay.” • Indicating in any way that a student has answered a question incorrectly • “Check your work” • Defining or spelling words • Explaining, simplifying, or paraphrasing any part of the test • Suggesting that a student write more • Influencing a student’s response through gestures, facial expressions, nods, or changes in voice inflection

26. What is Permitted? • Reading a word or short phrase aloud on the Mathematics and STE tests – Universal Accessibility Feature (UF10) • Assisting students with technology-related problems • helping a student sign in to TestNav • pointing to a tool button that a student cannot find • helping a student understand how to enter a result in the equation editor • helping a student submit his or her test • Re-reading part of the TAM script to students • Encouraging students if they have questions about test content • “Just do your best.” “Just give your best effort.” “Do the best you can.”

27. Test Security Requirements for Students • Students’ results may be invalidated if they • use notes, books, or other reference materials • access a cell phone or other electronic device (other than their testing device) • look at another student’s screen or allow another student to see their screen • pass notes or talk to other students • access the Internet or use any application on their testing device other than TestNav • See sample form for students and a sample letter to send to parents (recommended for grades 6–8 and 10) • (to be posted on DESE site soon)

28. Steps to Take After a Security Incident or Irregularity • Call the Department at 781-338-3625. • Gather information about the incident if necessary. • Talk to the test administrator(s) involved. • Talk to any students involved. • Preserve documents (e.g., notes confiscated). • If reporting a read-aloud violation, determine how much of the test was read. • If reporting a calculator violation, determine which items were solved with the calculator. • Submit a report if instructed to. • Take any local action that is appropriate.

29. Test Administration Protocols 02

30. Don’t go it alone! Rely on others in the school and district with test administration experience for support.

31. Transition to Next-Generation Tests for Grades 3–8 and 10 ELA and Math • * Paper test accommodation for students with disabilities or new ELs

32. Manuals and Trainings • Introduction to Computer-Based Testing fall 2018 session • Slides and recording posted at mcas.pearsonsupport.com/training/ • Manuals are expected to be available this week • www.doe.mass.edu/mcas

33. Orientation to the 2019 Test Administration Manuals • PAM: Contains instructions for all tests, including the high school field tests • CBT TAM: • All grades 3–8 tests • Grade 10 ELA and Mathematics • High school Biology and Introductory Physics field tests • PBT TAM: • All grades 3–8 tests • Grade 10 ELA and Mathematics • Operational high school STE tests

34. Orientation to the 2019 Principal’s Administration Manual • New infographics and new icons – e.g., • Part IV, Section A: new checklist/timelines • Appendix B: PBT instructions • Appendix C: accessibility and accommodations, including those for EL students • Sample administration materials/forms moved online

35. Tasks to Complete This Winter • Read the biweekly Student Assessment Update (www.doe.mass.edu/mcas/updates.html) • Update student information in both of the following: • SIMS (DESE Student Information Management System) • Data will be based on data from the School Interoperability Framework (SIF) for the SR/PNP for districts that use SIF. • For non-SIF districts, data will be based on the October SIMS submission. • PearsonAccess Next (online test management system) via the Student Registration/Personal Needs Profile (SR/PNP) • Meet with the technology coordinator (ProctorCache, technology specifications, configure devices, Infrastructure Trials) • Gather materials provided by the school • E.g., pencils, calculators, bilingual word-to-word dictionaries, scratch paper

36. “Ready for Another Banner Year” • 20,400 baseballs • 1,100 bats • 200 batting gloves • 200 batting helmets • 320 BP tops • 160 jerseys • 300 pairs of pants • 400 t-shirts • 400 pairs of socks • 20 cases of bubble gum • 60 cases of sunflower seeds

37. Identify “Who,” “Where,” and “When” (Late Winter/Early Spring) • Identify students who will be participating in each grade’s tests, and assign accessibility features and accommodations. • Submit alternate setting requests (PAM Part II, section E) for • Students receiving instruction at home or hospital • Students who are suspended/expelled, but still enrolled • Establish testing locations, and develop staffing plans. • Prepare list of test administrators and students for school records. • Begin training plans for test administrators and staff. • Plan for secure testing environments. • Plan for accessibility features and accommodations.

38. Plan the School’s Schedule and Begin Preparations • Review prescribed windows/dates and administration deadlines • www.doe.mass.edu/mcas/cal.html • Plan the daily schedule during test administration • New testing time recommendations and number of ELA sessions • One break per session, at test administrator’s discretion • Student questionnaire • ESE recommends dismissal waves for students who finish early • Plan for lunch for students who need extra time

39. Scheduling Guidance for Spring 2019 Grades 3–8 Tests • For make-up testing, schools may group students taking different tests together if the TAM script and the materials are the same. • Do not group together students doing CBT and PBT. • www.doe.mass.edu/mcas/cal.html

40. Spring 2019 Grade 10 Test Schedule • Grade 10 ELA and Mathematics • Administer tests to the maximum number of students who can be tested concurrently on the first set of days. • Schools with technology constraints may use the second set of dates to test the remainder of their students. • Schools with special circumstances that cannot test all their students on these dates must email mcas@doe.mass.edu to discuss options.

41. Additional Scheduling Guidance • Short break mid-session for all tests with the exception of the legacy operational high school STE tests • Student questionnaire now administered at grade 4 (as well as grades 5, 8, and 10 as before) • Questionnaire appears after grades 4 and 10 Math Session 2, and after grades 5 and 8 STE. • TestNav will be available between 7:00 a.m. and 4:00 p.m., Monday–Friday, except the week of April 15. • Principals will need to request special access to test outside these hours (e.g., evening high schools). See PAM Part III, section C.

42. Spring 2019 High School Biology and Intro Physics Field Tests Requirements Participation Notes • Administration window • May 28–June 14, except June 3–5 (Recommended date: June 6) • Participation guidelines • All grade 9 students taking the operational high school Biology or Introductory Physics test are expected to take the field test in the same subject (exceptions below). • Grade 10 participation is optional/at principals’ discretion. • Students not expected to participate: • first-year EL students • students who take the MCAS-Alt • repeating grade 9 students • Unavailable accommodations • A1 and EL1 – Paper-based edition • A2 – Large-print edition (Onscreen text can be magnified.) • A3.1 and A3.2 – Screen reader and Braille • Unique Accommodation – Use of a compatible assistive technology device

43. 2019 Principal’s Certification of Proper Test Administration • The PCPA is streamlined to reflect the shift to computer-based testing. • The “before testing” section is removed, and schools will complete the PCPA only after testing as shown in the table:

44. Preparing Students for CBT • Resources from the Office of Digital Learning • Resources on acquiring technology, digital citizenship, and online courses may be particularly helpful for schools new to CBT • Student Tutorial • Demonstration of the navigation, tools, and features for CBT • Students complete it independently (no audio) • Practice tests • Simulation of the tools and features for CBT • Include accessibility features and accommodated test forms (e.g., TTS) • Sample student work from the spring 2018 MCAS tests • Answer keys and scoring rubrics • Math and STE: Equation editor guides and reference sheets • Resources for parents/guardians • See topics in the PAM for the student meeting and for parents.

45. PearsonAccess Next User Roles • PAN user accounts can be created with an “Active End Date.” Once that date has passed, a user cannot access his or her account. • District or school test coordinators should review accounts for all staff from 2018 to update the Active End Date if necessary. • Remove or change the Active End Date by file upload or through the user interface. • Recommended: Users should update their passwords after the Active End Date has been updated.

46. Training the Test Administration Team in CBT Procedures • Technology staff need to assist with set-up and troubleshooting. • See list of tasks in PAM Part III, section E • Training for technology coordinators on February 12 (Register at www.doe.mass.edu/conference/?ConferenceID=9954.) • Precache test content in mid-March. • Schedule district support techs for testing days whenever possible. • Familiarize test administrators with tasks in PearsonAccess Next. • List of modules and tasks to complete in PearsonAccess Next in the PAM in Part III, section F • Expanded troubleshooting steps in PAM Appendix A. • If there is a technology situation in which a student is waiting more than 15 minutes, then schedule the student to complete the session at another time.

47. Hardware considerations • Power strips • Battery adapters for mobile devices • Extension cords • Extra devices for each testing area • Headphones required for text-to-speech accommodation • External keyboards strongly recommended for tablets

48. Infrastructure Trials Infrastructure Trials Confirm Main Steps • TestNav is configured correctly. • ProctorCache software is set up correctly. • Devices can successfully run TestNav and that online test content can pass through the content filters at the school. • The network can bear the full load. • Participating staff understand their roles for MCAS CBT. • Students are familiar with the CBT tools/format. • Set up and configure the test management and test delivery environment. • Generate sample students and create PAN Sessions. • Administer practice tests. • Conduct follow-up activities and review lessons learned with the test administration team. Trainings on February 13 and 15: www.doe.mass.edu/mcas/training.html

49. Questions & Answers (Use the “Questions” feature to ask questions.)

50. Accessibility and Accommodations 03