California Alternate Assessments (CAAs) Post-Test Reporting Webcast

1. California Alternate Assessments (CAAs) Post-Test Reporting Webcast Tom Bjorkman, CDESheila Self, CDE Dan Stanley, ETSJoyce Wang, ETSGrace Yoo, ETS November 17, 2016

2. Purpose • Provide information about student scores and reporting for the California Alternate Assessments (CAAs) for English language arts/literacy and mathematics. • By the end of this Webcast, viewers will be able to: • understand the Online Reporting System (ORS) to view final CAA results; and • use the components of the CAA student score report (SSR); • use the Test Operations Management System (TOMS) to download final CAA results CAA Reporting Webcast

3. Overview of 2016 Reporting Timeline • Understanding Available Reporting for the CAAs • Viewing Test Results in ORS • Reading CAA Student Score Reports • Accessing Student Score Reports in the Test Operations Management System (TOMS) • Interpreting CAA Results • Additional Reporting Resources • Questions and Answers CAA Reporting Webcast

4. Overview of 2016 Reporting Timeline CAA Reporting Webcast

5. Overview of 2016 Reporting Timeline • Nov 16—Final CAA Results Available on the Online Reporting System. • Nov 28—LEA-specific student-level CAA test results will be available for download in the Test Operations Management System. • Dec 6 to 16—Paper CAA student score reports (ISRs) are shipped to LEAs. • Dec—Statewide aggregate CAA results will be available on the California Department of Education (CDE) Public Web Reporting Site. CAA Reporting Webcast

6. Understanding Available Reporting for the CAAs CAA Reporting Webcast

7. Simplified Data Flow Student Demographic Data TOMS (Test Assignments and Settings) Test Settings & Basic Student Data CALPADS Test Delivery System Student Responses ORS Scoring System Test Results TOMS (Reporting) CAA Reporting Webcast

8. Available Reports † Results in ORS will be updated to include CAA results on November 16. † † PDFs of the Student Score Reports will be available in TOMS for both the local educational agency (LEA) and the school. PDFs of the SSRs will be downloadable in batches by school in TOMS. ††† LEAs must forward or mail the copy of the paper CAA Student Score Report to each student’s parent/guardian within 20 working days of its delivery to the LEA. ††††The existing CAASPP downloadable student data file will be updated later this year to include CAA results. LEA CAASPP Coordinators may download the results again. CAA Reporting Webcast

9. Viewing Test Results in ORS CAA Reporting Webcast

10. ORS Summary The Online Reporting System (ORS) is a Web-based system that displays score results for students taking CAASPP assessments, now including CAAs for ELA and mathematics. The ORS will be updated on November 16 with CAA results. CAA Reporting Webcast

11. Users with Access to Score Reports in ORS CAA Reporting Webcast

12. Features and Reports in ORS for CAA Accessing the ORS Quick Start Guide is available at http://www.caaspp.org/administration/reporting/ Home Page Dashboard Subject Detail Report Student Listing Report Manage Rosters CAA Reporting Webcast

13. Home Page Dashboard 1 2

14. Home Page Dashboard: Select Test, Administration, and Enrollment Status Select Test 1 Select Administration Enrollment Status View CAA Reporting Webcast

15. Home Page Dashboard: Select Grade and Subject 2 Select the number of students tested to view subject detail report. CAA Reporting Webcast

16. Subject Detail Report Report Name Legend Show/Hide Comparison Data Show/Hide Columns Exploration Menu Time Stamp CAA Reporting Webcast

17. Exploration Menu **Assessment Target Reports will NOT be available for CAAs. CAA Reporting Webcast

18. Student Listing Report CAA Reporting Webcast

19. Student Detail Report Report Name Summary Additional details of the student’s performance Average LEA and school scaled scores for comparison CAA Reporting Webcast

20. ORS Resources • For more information on how to use all other functionalities of the ORS, please refer to the 2015–16 Post-Test Webcast. http://www.caaspp.org/rsc/videos/archived-webcast_051916.html • For further in-depth instructions on how to use the ORS, please refer to the Online Reporting System User Guide for California: http://www.caaspp.org/rsc/pdfs/CAASPP.ORS-guide.2016.pdf CAA Reporting Webcast

21. Reading CAA Student Score Reports CAA Reporting Webcast

22. CAA Student Score Reports Mockups of the Student Score Report 6 1 5 2 7 3 8 4 CAA Reporting Webcast

23. Information about CAA Elements of the Student Score Report—Front Page 1 CAA Reporting Webcast

24. Overall Score Elements of the Student Score Report—Front Page 2 Overall score and level for each content area CAA Reporting Webcast

25. Test Results from Multiple Years Elements of the Student Score Report—Front Page The 2015–16 CAA SSR will reflect only one year of operational testing. 3 CAA Reporting Webcast

26. Descriptions of Student Performance Elements of the Student Score Report—Front Page 4 CAA Reporting Webcast

27. Link to State Scores Elements of the Student Score Report—Front Page http://caaspp.cde.ca.gov/ Access aggregate results for the 2015–16 administration of the CAAs here in fall 2016. 4

28. Student Address Elements of the Student Score Report—Back Page 5 CAA Reporting Webcast

29. Letter from the State Superintendent of Public Instruction Tom Torlakson Elements of the Student Score Report—Back Page 6 CAA Reporting Webcast

30. Information about CAAs Elements of the Student Score Report—Back Page 7 CAA Reporting Webcast

31. CAPA for Science Results Elements of the Student Score Report—Back Page 8 Grades five and grade eight only Grade ten Student Score Report, CAPA for Science only CAA Reporting Webcast

32. Accessing Student Score Reports in the Test Operations Management System (TOMS) CAA Reporting Webcast

33. How to Download Student Score Reports from TOMS TOMS Student Score Report Downloadable Files Quick Start Guide available at http://www.caaspp.org/administration/reporting/ CAA Reporting Webcast

34. Download Student Score Report PDFs Select year • Select the school and grade of the reports you want to compile and name the report download file. • To select contiguous grades (e.g., grades 3–6), hold down the [Shift] key as you select each grade. • To select specific non-contiguous grades (e.g., grade 3, grade 4, grade 6), hold down the [CRTL] key as you select each grade. CAA Reporting Webcast

35. Student Score Report PDF Download File LEA CDS Code School CDS Code Grade SSID Date stamp • A compressed file (.zip) will be generated. • Large files may take time to process. • Each PDF contains the Student Score Report for one student.

36. Student Score Report PDF Download File Hyperlinks to the individual Student Score Reports

37. Interpreting CAA Results CAA Reporting Webcast

38. Scores to be Reported Scaled scores for English language arts/literacy (ELA) and mathematics Performance levels for ELA and mathematics No Claim Scores are reported CAA Reporting Webcast

39. Scaled Scores “Scaling” is a statistical process that transforms scores from one set of numbers to another that allows scores to be comparable in some way. A scaled score DOES NOT mean “percent correct.” A scaled score DOES NOT mean “raw score.” CAA Reporting Webcast

40. CAA for ELA and MathematicsScaled Score Ranges by Grade Scaled score ranges are the same for both the English language arts/literacy and the mathematics CAA assessments. CAA Reporting Webcast

41. Understanding How Scaled Scores Are Assigned Students who did not respond or orient to the test will receive a Lowest Obtainable Scaled Score (LOSS) (e.g., 300, 400, 500, and so forth). Students who responded to fewer than four questions will receive a “LOSS + 1” (e.g., 301, 401, … ,901). CAA Reporting Webcast

42. How to Use CAA Results • Scaled scores for the California Alternate Assessments should not be compared across different content areas and grade levels. • Scaled scores for the California Alternate Assessment should not be compared to different assessments such as: • Smarter Balanced Assessments • California Alternate Performance Assessment (CAPA) CAA Reporting Webcast

43. How To Use CAA Results • In this first year of reported results for the California Alternate Assessments, scaled scores in the same subject can be compared within grade levels, and from one year to another. For example: • Mean scaled scores for Grade 4 math in LEA 1 may be compared with Grade 4 math for the whole state. • Mean scaled scores for Grade 5 ELA in School 1 may be be compared with Grade 5 ELA for School 2. CAA Reporting Webcast

44. Performance Levels • Three performance levels are assigned on the basis of a scaled score • Level 1 – Limited Understanding • Level 2 – Foundational Understanding • Level 3 – Understanding CAA Reporting Webcast

45. Performance Level Descriptors CAA Reporting Webcast

46. California Alternate Assessments: Recap These assessments are built on the assumption that students are receiving the appropriate grade-level curriculum, instruction, accommodations and access to technology they require. Each year presents new opportunities to improve their instruction of students with significant cognitive disabilities, but also to improve the delivery of the alternate assessments. CAA Reporting Webcast

47. California Alternate Assessments: Recap • The CDE (Assessment and Special Education Divisions) along with ETS will continue to work collaboratively to support the field. • As part of that collaboration we will continue to support and listen to the field on: • how we can improve both the testing experience and, • assist educators in identifying the most appropriate accessibility supports. CAA Reporting Webcast

48. CAA Reporting Webcast

49. CAA Reporting Webcast

50. CAA Reporting Webcast