CAA Field Test Overview & Preparation Webcast

1. California Assessment of Student Performance and Progress (CAASPP) Spring 2015 California Alternate Assessments (CAA) Field Test for English–Language Arts (ELA) and Mathematics March 11, 2015 Sheila Self, M.Ed., California Department of Education Kellie Taylor-White, MA, Lead Test Developer, ETS Dan Stanley, Program Manager, ETS

2. Purpose and Target Audience • This Webcast will help LEA CAASPP Coordinators to: • Learn all you need to know to prepare for administering the CAA Field Test in your LEA • Learn how to prepare your Site Coordinators for CAA Field Testing • Site Coordinators may also wish to view this Webcast • This Webcast also serves as an introduction and explanation of the CAA Field Test for Site Coordinators and CAA Examiners • CAA Test Examiners are also required to view a Test Administrator Tutorial that will be available starting March 20

3. Agenda • Overview • Student Eligibility Requirements • Purpose of the CAA • Features of the CAA • Roles and Responsibilities • Test Security • Registering Students for CAA • Universal Tools, Designated Supports, and Accommodations • Preparing Technology • Administering the Tests • After Testing

4. Overview—Test Name • A new alternate assessment for California as part of the CAASPP program • The California Alternate Assessments (CAA) • English–language arts (ELA) • Mathematics • Replaces the California Alternate Performance Assessment (CAPA) for ELA and mathematics • Commonalities with other “next generation” alternate assessments

5. Overview—Testing Window • Testing window for all LEAs is April 15 through June 10, 2015. • All scheduling within this window is according to local preference • Do not need to inform ETS or CDE of testing schedule • All testing must take place within window, including make-ups

6. Overview—Test Takers • All eligible students in grades 3–8 and grade 11 must participate • Students in ungraded programs will need to be assigned a grade level for testing • Students in grades 5, 8, and 10 still take the CAPA for Science (in addition to CAA at grades 5 and 8)

7. Overview—Test Delivery • Testing is online only, using a compatible Web-browser with no software installation necessary. • One-on-one test sessions with a Test Examiner, who is familiar with the student. • All items are grade level, aligned with the Common Core State Standards (CCSS) and based on the Core Content Connectors developed by NCSC. • Three different levels of complexity, and all three levels will be administered to all students. • Items are being tested, not students.

8. Overview—Resources Key resources for CAA Field Test: • CAA Field Test Administration Manual (TAM) • Sample Directions for Administration (DFA) • Will be paired for use with a Training Test when available • Grade-level DFAs • Individual Student Assessment Accessibility Profile (ISAAP) Tool • Test Administration Tutorial http://caaspp.org/about/caa/

9. Student Eligibility Requirements • The student has a significant cognitive disability. • The student is learning content derived from the Common Core State Standards (CCSS). • The student requires extensive direct individualized instruction and substantial supports to achieve measurable gains in the curriculum. Additional detail on these requirements can be found in the TAM on page 5.

10. Purpose of the CAA • Give students with significant cognitive disabilities the opportunity to demonstrate their achievement of the Common Core State Standards (CCSS) by taking a test commensurate with their abilities. • Assessed through Core Content Connectors (CCCs) that are rewritten main achievement standards from the CCSS. • Provide more access to students with significant cognitive disabilities.

11. Features of the CAA • CAA is a grade-level assessment only • Student grade levels are taken from CALPADS. • For students in ungraded programs, testing grades are calculated by the Test Examiner using the “subtract 5” rule • For the Field Test, students in the same grade will see items reflecting all levels of difficulty, from least complex to more complex. • Approximately 15 items per content area that take 30–60 minutes to administer.

12. Features of the CAA (cont.) • Tests are administered with grade-specific and version-specific DFAs. • Versions are assigned at the school level. • Version assignments for each school in California will be posted to caaspp.org in late March, shortly before DFAs are available for download in TOMS. http://caaspp.org/about/caa/

13. Roles and Responsibilities—LEA CAASPP Coordinator For the CAA Field Test, LEA CAASPP Coordinators are responsible for all of the following activities: • Registering students in TOMS • Training CAASPP Test Site Coordinators (SCs) • Configuring student test settings in TOMS (with SCs) • Confirming with SCs that each CAA Test Examiner has viewed the Test Administration Tutorial and verifying training completion • Ensuring that all Test Examiners have read and signed a test security affidavit • Reporting all test security irregularities and/or breaches

14. Roles and Responsibilities—CAASPP Test Site Coordinator For the CAA Field Test, CAASPP Test Site Coordinators are responsible for the following activities: • Identifying certificated CAA Test Examiners, ensuring that they view the CAA Test Administration Tutorial, and collecting the resulting certificate of completion. • CAA does not use the train-the-trainer model of providing instruction to Test Examiners. Instead, Test Examiners must participate in the Test Administration Tutorial. • The Test Administration Tutorial is required for all Test Examiners.

15. Roles and Responsibilities—CAASPP Test Site Coordinator (cont.) • Collecting signed CAASPP Test Security Affidavits from CAA Text Examiners • Adding each new CAA Test Examiner into TOMS and tracking completion of the Test Administration Tutorial • Entering and/or verifying test settings for students in TOMS (universal tools, designated supports, and accommodations [UDAs] that were uploaded into TOMS with the ISAAP tool) • Creating or approving testing schedules and procedures for the school

16. Roles and Responsibilities—CAASPP Test Site Coordinator (cont.) • Working with technology staff to ensure that testing devices are functioning properly • Coordinating with CAA Test Examiners so they administer all assessments • Mitigating and reporting all test security incidents in a manner consistent with state and LEA policies • Collecting secure DFAsand any print-on-demand items from Test Examiners following testing and securely destroying them

17. Roles and Responsibilities—CAA Test Examiner For the CAA Field Test, CAA Test Examiners are responsible for the following activities: • Reading and submitting a signed CAASPP Test Security Affidavit to your SC • Viewing the Test Administration Tutorial, printing the resulting certificate, and submit to your SC • Reviewing designated supports and accommodations in TOMS for each student you are testing • Downloading from TOMS and reading the DFAs before testing

18. Roles and Responsibilities—CAA Test Examiner (cont.) • Conducting a student response check for each student (required) • Completing the learner characteristics inventory for each student (optional) • Administering the tests in accordance with the CAA Field Test Administration Manual • After testing, give any paper copies of the DFAs to your SC for secure destruction and work with your SC to permanently delete any PDF copies

19. Test Security • Follow the same security guidelines as for other CAASPP assessments. • Information about Field Test items, including DFA content, should not be shared with anyone not directly responsible for test administration.

20. Registering Students for CAA • Students are registered for CAA in TOMS • Any students previously registered for “alternate” using the CAPA flag in the CAASPP registration upload template (available since November) should automatically be registered for CAA by TOMS. • Registrations were accepted for all CAA/CAPA grades (3–8, 10, 11). • Because tests are grade-level only, students must be assigned a grade for testing. • For students who are in ungraded programs and whose IEP teams designate that they take the CAA Field Test, their grade for testing is calculatedby subtracting five from their chronological age on September 1, 2014. • A student who was 10 years old on September 1 would be given the grade five test (10 – 5 = 5). • Confirm the grade assignment is correct for each student.

21. Registering Students for CAA (cont.) • Starting March 9, one-by-one student registration available in TOMS • Registration for CAA/CAPA automatically un-registers the student for any other CAASPP assessments • When CAPA or CAA is selected, student will be registered for both. • To unregister a student from CAPA/CAA, a CST or CMA must be selected and CAA must have a selection of “No”. • Starting on March 20, CAA registrations will be viewable in the TOMS student roster report • Bulk registration is still available, for CAA only, using the same paper-pencil registration template • In TOMS, select “Paper test assignments” from the “Students” menu • Registering for CAA/CAPA will not automatically change any CAPA materials orders. If needed, contact CalTAC to order additional CAPA materials through the supplemental order process.

22. Registering Students for CAA (cont.) • There is no specific deadline for registering students for CAA, however: • Registration should be completed as soon as possible so that student rosters will accurately reflect the correct test assignments. • Students should be registered for the correct assessments before test settings (designated supports and accommodations) are configured for those students in TOMS.

23. Universal Tools, Designated Supports, and Accommodations The CAA field test makes many of the same universal tools, designated supports, and accommodations (UDAs) available as for other CAASPP assessments. • Universal tools are available to all students by default, although they can be disabled if they are distracting for a student. • Designated supports can be provided at the discretion of a teacher familiar with the student’s needs. • Accommodationsmust be specified in a student’s IEP. • Note that, for the field test, students may use any accommodation routinely used in the classroom, even if not currently in IEP. • During the next cycle of IEP review, you may update with state testing accommodations.

24. Universal Tools, Designated Supports, and Accommodations (cont.) Many of the same UDAs available for other CAASPP assessments are available in the same way for the CAA field test, with the following exceptions listed below: • Text-to-speech, which is not available as an embedded support but may be provided as a non-embedded support and/or through an assistive device. • Braille, which is not available for the CAA field test. • American Sign Language, which is not available as an embedded support. • Translations/Glossaries, which are not available as embedded supports. • Closed Captioning, which is not available for the CAA field test.  

25. Universal Tools, Designated Supports, and Accommodations (cont.) The Individual Student Assessment Accessibility Profile (ISAAP) tool should be used to identify the right UDAs for each student taking the CAA. • Same tool for Smarter Balanced is now enhanced to work for CAA. Select the “CAA” checkbox to configure the tool according to UDA availability for the CAA Field Test. • Enter the specific needs of students and tool will automatically generate a file that can be uploaded into TOMS. • ISAAP tool is available from caaspp.org, from http://caaspp.org/about/caa/.

26. Preparing Technology • The CAA are designed for one-on-one administration between a student and a certificated Test Examiner. • Online testing using a supported Web browser (secure browser not used) • A list of supported systems, devices and browsers can be found at http://ca.browsers.airast.org/supported-browsers/ • Refer to “TA Sites” column in the table • A single device is needed to launch a test session and administer the test. (TA Interface not used) • Other technology and device specifications, including headphones and keyboard requirements, are generally the same as for Smarter Balanced. • Assistive devices may be used, according to the communication and response method of each student.

27. Other Technology Resources Refer to the following CAASPP resources at http://caaspp.org/administration/instructions/for other instructions and guidelines. (Note that parts of these manuals will not apply to CAA, such as use of the secure browser, TA Interface, and the test settings not available for the CAA field test): • Technical Specifications Manual—for instructions on preparing technology for testing • Online Test Administration Manual—for general CAASPP system instructions such as security • 2015 Online Student Test Settings User Guide—for information about assigning accessibility tools and supports

28. Administering the Tests • A separate, grade- and version-specific Directions for Administration (DFA) document guides Test Examiners through the CAA with item-specific administration instructions, including scoring rubrics for writing items. • Version assignments for each school in California will be posted at http://caaspp.org/about/caa/ and DFAs will be available for download from TOMS before the testing window opens. • DFAs are in PDF format and may be printed or used electronically according to local preference. • Because these DFAs contain live testing material, they must be kept secure at all times and must be securely destroyed after testing. • LEA or site coordinators must ensure that electronic copies are securely destroyed on each device they were loaded to.

29. Administering the Tests (cont.) Administration steps on test day: • On the device that will be used for testing, Test Examiner opens a supported Web browser and navigates to the CAA Online Testing Interface, with DFA in-hand or on a separate device. • Test Examiner logs on using TOMS username and password. • Test Examiner logs onto student test session using student first name and SSID. • Test Examiner is presented with three testing options: • Learner Characteristics Inventory (LCI) • ELA • Mathematics • Test Examiner completes the LCI (optional)

30. Administering the Tests (cont.) Administration steps on test day: • Test Examiner launches ELA or mathematics test and engages student for the start of testing • Test Examiner is prompted to administer a Student Response Check (SRC) to observe and record the student response mode and how the student will indicate their responses during testing. Answering these SRC questions is required for all students. Examples of student responses include: • Mouse/keyboard • Verbal response • Touch screen, gestures, or point • Augmentative and/or alternate communication device • Eye gaze • ELA and mathematics tests are untimed, and the Test Examiner may administer as many sessions over as many days as necessary according to individual student needs.

31. Administering the Tests (cont.) Learner Characteristics Inventory • An optional Learner Characteristics Inventory (LCI) is available after logging in to a student’s tests. If completed, the LCI questions should be answered by the Test Examiner before the start of ELA and mathematics testing for each student, based on his/her familiarity with that student. • Please note that LCI responses will be aggregated; neither Test Examiners nor student(s) will be identified in or by the results.

32. Administering the Tests (cont.) Student Response Check • CAA Test Examiners must administer the Student Response Check (SRC) to each student at the beginning of testing. The purpose of the SRC is to document that the student has a consistent and observable way to communicate his/her response to the test items, using the intended response mode(s).  • Hand-over-hand or other physical prompting by the test examiner is not permitted in the CAA field test. • The SRC does not need to be administered twice. (Can be skipped at the start of the second content area).

33. Administering the Tests (cont.) Student Response Check (Cont.) There are three possible outcomes from the SRC: • The student demonstrates an observable, consistent response • The student demonstrates an observable, but inconsistent, response • The student does not demonstrate any observable responses

34. Administering the Tests (cont.) Student Response Check (Cont.) • If the outcome is “1,” Test Examiners administer the entire assessment.  • If the outcome is “2,” Test Examiners administer the first four ELA or mathematics items. If a consistent, observable response is elicited for any of these four items, administer the entire assessment.  • The response does not need to be correct but rather the student orients to the test and attempts to respond. • If no observable response for the first four items, the Test Examiner should stop testing and report this to the Site Coordinator. • If the outcome is “3,” Test Examiners do not administer the assessment.

35. Administering the Tests (cont.) Pause Rules The following pause rules apply regardless of whether the student or the Test Examiner pauses the test or there was a technical issue that resulted in the student being logged off. The CAA may be paused and resumed as many times as necessary to elicit the student’s best testing performance. • When a test is paused, you must log back on to resume testing. Upon resumption, students will automatically be directed to the first page that has an unanswered item. • In the event of a technical issue (e.g., power outage or network failure), student’s test will be paused and the students will be logged off. • Students will return to their last unanswered test question when logged on again.

36. Administering the Tests (Cont’d) Test Timeout (Due to Inactivity) • As a security measure, test sessions are automatically logged off after 30 minutes of test inactivity. This timeout also results in the test being paused automatically. • Activity is defined as selecting an answer or navigation option in the test (e.g., selecting [Next] or [Back] or using the Questions drop-down list to navigate to another item). Selecting an empty space on the screen is not considered activity. • Before the system logs off of the test, a warning message will be displayed on the screen. If you do not select [OK] within 30 seconds after this message appears, he or she will be logged off.

37. CAA Training Test • For practice and training purposes, Test Examiners will have access to an online training test starting in early April. • Used in conjunction with the sample DFA, this training test can help both Test Examiners and students become familiar with the new testing format and functionalities of the CAA assessments. • More information about the Training Test will be provided when available.

38. After Testing • Site Coordinators ensure all DFAs are collected and securely destroyed • Shred or otherwise destroy printed copies • Permanently delete electronic PDF copies from all devices • As necessary, enter any special testing condition codes in TOMS. • After testing, you may be asked to complete a survey about your field-testing experience. This information will help to improve the test.

39. Comparing CAPA to CAA

40. Support for the CAA Field Test The CAASPP portal athttp://caaspp.org/ is the home for all CAASPP assessment administration information, including the CAA field test. All CAA materials and resources will be available on the CAASPP CAA Field Test Web page at http://caaspp.org/about/caa/. The California Technical Assistance Center (CalTAC) is here to support all LEA CAASPP coordinators! Monday–Friday from 7 a.m.–5 p.m. PT E-mail:caltac@ets.org Phone: 800-955-2954 Web site:http://caaspp.org

41. Questions?