ASSESSMENTS & ACCOUNTABILITY

1. ASSESSMENTS & ACCOUNTABILITY Suspense Resolution and Reports Assessments and Accountability v2.0

2. Mute phones while listening (*6 to mute and un-mute) Do not place your phone on HOLD Silence cell phones Hold questions until Q&A periods Submit all questions via the Chat function GROUND RULES Assessments and Accountability v2.0

3. ASSESSMENTS & ACCOUNTABILITY (A/A) EXPECTATIONS Assessments and Accountability v2.0

4. A/A Expectations • Assessment and Accountability functionality in CALPADS is being implemented to support: • Building a complete individual student assessment history for STAR, CAHSEE, and CELDT • Improving the accuracy and consistency of data used to evaluate performance of individual students and educational entities Assessments and Accountability v2.0

5. A/A Expectations • Achieving these objectives will require several iterations of CALPADS functionality • Current phase of functionality is focused on identifying issues for both LEAs and CDE related to the loading of data • Future functionality will address these issues as well as enhance the accessibility and usability of the data for LEAs and CDE Assessments and Accountability v2.0

6. A/A Expectations • The initial CALPADS A/A functionality requires no changes in the way LEAs currently work with assessment vendors or with CDE • Student Demographic changes will continue to be made through the testing vendor (within vendor’s test correction window) • Testing Vendors will provide test results & reports as usual • Pre-ID, where applicable, will continue as usual • Testing processes will continue as directed by assessment program offices • AYP/API results will continue to use the student data you supply to test vendors Assessments and Accountability v2.0

7. A/A Expectations • LEAs will: • Match vendor results records for your LEA to the correct SSID when CALPADS cannot make the match automatically • Verify the reports • Provide feedback to improve functionality Assessments and Accountability v2.0

8. A/A Expectations • CDE will: • Match vendor results records to the correct SSID in CALPADS, when a LEA flags a record as “Not Mine” • Negotiate contracts with assessment vendors that ensure the data needed for a student’s complete assessment history in CALPADS is available and timely • Coordinate with each assessment program area to enable CALPADS to… • Provide a consistent approach to managing and reporting assessment data • Report assessment results that reflect the testing procedures communicated to LEAs Assessments and Accountability v2.0

9. A/A Expectations • CDE will (cont): • Evaluate the outcome to determine the impact on LEAs and Accountability reports of using CALPADS data instead of vendor data • Work with LEAs to provide functionality focused on data accessibility and usability • Communicate When and How: • Results data left in suspense will impact Accountability reporting • Current vendor relationships will be impacted Assessments and Accountability v2.0

10. A/A Expectations Logistics • A schedule for data loads is included in the report sections for each Program • Assessment and Accountability Coordinators receive an email when vendor data is loaded • Report content is frozen when CDE performs the last snapshot for a Program Administration after all data received from the assessment vendor has been loaded Assessments and Accountability v2.0

11. SUSPENSE MANAGEMENT Assessments and Accountability v2.0

12. Suspense Management CALPADS Match Results: When vendor results data are loaded to CALPADS, the result of the matching process can be either … • Suspense - (i.e. unsuccessful match) • Matched – (i.e. successful match) Assessments and Accountability v2.0

13. Suspense Management Reason for Suspense Record: • Mismatch in SSID and/or student demographic data (gender, date of birth year) between the assessment results file and CALPADS • Missing CALPADS enrollment at time of testing • Missing SSID Administration Date As soon as one of these conditions is not met, CALPADS places the record in suspense. SSID Administration Date is the datein an SSID’s assessment results record that is used in SSID Enrollment Validation. STAR = Testing Start Date; CAHSEE = ELA Test Date and Math Test Date; CELDT = Date Testing Completed. (See CALPADS User Manual Glossary for additional information) Assessments and Accountability v2.0

14. Suspense Management Suspense Errors: • Date of Birth Mismatch - If Student Date of Birth Year for SSID in results record <> Student Date of Birth Year in ODSR SINF data that was valid at the time of testing • Gender Mismatch - If Student Gender Code for SSID in results record <> Student Gender Code in ODSR SINF data that was valid at the time of testing • SSID Mismatch - If SSID in results record does not exist in ODSR • Admin Date Empty - If the Administration Date in the results record is blank • LEA Mismatch - If enrollment in LEA at the time of testing could not be validated Assessments and Accountability v2.0

15. Suspense Management Suspense Resolution Actions: • The LEA correcting CALPADS data • The vendor correcting results data • The LEA and/or CDE assigning the suspense record to the correct SSID Suspense Resolution Methods: • Manual match which provides reporting data immediately • Automated match (nightly suspense, load suspense, CDE snapshot suspense) which provides reporting data at completion of process Assessments and Accountability v2.0

16. Suspense Management Suspense Resolution Workflow: LEA Workflow: • Review Suspense Records • Determine best resolution and apply • Wait for data refreshand verify suspense resolved Assessments and Accountability v2.0

17. Suspense Management A record in suspense will not be matched to a CALPADS SSID, which leaves the CALPADS assessment longitudinal record incomplete for this SSID As students enroll at other districts their assessment data will be missing if not matched Records in suspense do not appear in reports; therefore, your CALPADS assessment and AYP/API reports will not be complete Importance of Suspense Resolution: Assessments and Accountability v2.0

18. Suspense Management Suspense data can be resolved both Online & in Batch Both methods are accessed from the Assessment menu CALPADS Functionality Note: Batch resolution is normally used only by the largest LEAs. High level instructions are included in the Wrap-Up Documentation. Assessments and Accountability v2.0

19. MANAGING SUSPENSE DATA - ONLINE Assessments and Accountability v2.0

20. Managing Suspense Data - Online SUSPENSE STEP 1: Review Student List • Assessments > Online Maintenance > Manage Suspense Data • Select filter for Program and Administration (subject to user’s access rights), and Academic Year • Select Match Result = “LEA Suspense” to locate all students or an individual student with a record in suspense LEA Matched LEA Suspense CDE Suspense ADMIN Date & Suspense Reason Assessments and Accountability v2.0

21. Manage Suspense Data - Online SUSPENSE STEP 2: Determine Resolution 1 Manual match can be first, but incorrect data is provided to reports until CALPADS is corrected. 2 Data load confirms vendor’s change when suspense resolved. Assessments and Accountability v2.0

22. Managing Suspense Data - Online SUSPENSE STEP 3: Apply Resolution • If the student record in suspense does not belong to your LEA, select “Not Mine” • The record is placed in CDE’s queue for resolution. CDE can match the record to a student in another LEA. 1 1 2 2 Assessments and Accountability v2.0

23. Managing Suspense Data - Online SUSPENSE STEP 3: Apply Resolution (cont) • If the resolution for the Suspense Reason requires … • Changes in vendor data, the suspense process that runs during a load will eliminate the suspense when CDE receives and loads the corrected data from the vendor. • Data addition/changes in CALPADS, make the additions/changes and then either 1) match immediately to the corrected record using STEP 4, since the changes to ODS are available real-time, or 2) wait for the nightly suspense process to recognize that the CALPADS demo has changed and now matches the results record demo, eliminating the suspense automatically • Matching the results record with incorrect demographic data to a CALPADS record with the correct demographic data, use STEP 4. Assessments and Accountability v2.0

24. Annual Managing Suspense Data - Online SUSPENSE STEP 4: Initiate Match • Select Match initiates the user-defined search for a CALPADS SSID to which the result record belongs Annual Select Match John Doe Assessments and Accountability v2.0

25. Managing Suspense Data - Online SUSPENSE STEP 5: Enter Search Criteria • Enter SSID and/or demographics, using * wildcard if needed Enter Search Criteria Assessments and Accountability v2.0

26. Managing Suspense Data - Online SUSPENSE STEP 6: Review Returns • Matching results are displayed by SSID and/or Demographics, depending on the criteria entered Note the incorrect Birth Year Suggested Correction Record Assessments and Accountability v2.0

27. Managing Suspense Data - Online SUSPENSE STEP 7: Choose Record • Make selection and answer “OK” to initiate the match Select the record to which suspense record will be matched Assessments and Accountability v2.0

28. Managing Suspense Data - Online SUSPENSE STEP 8: Match Records • Both the suspense record and the selected match record are displayed Select “Match Records” Assessments and Accountability v2.0

29. Managing Suspense Data - Online SUSPENSE STEP 9: Resolve Problems • Return to STEP 2 and determine resolution • Repeat STEPS 3 – 10 (avoiding STEP 9) When the resolution requires changing CALPADS data, the new correction record will be available to your next SEARCH. Valid enrollment required!! Assessments and Accountability v2.0

30. Managing Suspense Data - Online SUSPENSE STEP 10: Return to Student List • When match is successful, return to Student List to resolve another record The newly matched record is available for reporting immediately Assessments and Accountability v2.0

31. Managing Suspense Data - Online MATCHED STEP 1: Confirm Matched Records • The matched Student List displays the Assessment Results record and the CALPADS record to which it has been matched • The source of the match is identified in the field “Match Source”. The source could be Batch, LEA, or State. LEA Matched LEA Suspense CDE Suspense Assessments and Accountability v2.0

32. LEA Matched LEA Suspense CDE Suspense Managing Suspense Data - Online MATCHED STEP 2: Resolve Incorrect Match • Matched records can be …. • unmatched if incorrectly matched (UNMATCH) and re-matching will occur later • unmatched if incorrectly matched and re-matching will occur now (SELECT NEW MATCH) • unmatched and sent to CDE for matching to another LEA (NOT MINE) Assessments and Accountability v2.0

33. VERIFY RESOLUTION Assessments and Accountability v2.0

34. Verify Resolution Data refresh timing depends on the choice you made for resolution Manual match with immediate resolution and data refresh Automated match (nightly suspense, load suspense, CDE snapshot suspense) with resolution and data refresh at completion of process WAIT FOR REFRESH Reminder: When you make changes to data in the ODS, they are replicated to the ODSR in real time. However, the change will not resolve a suspense until one of the following: • You match the suspense data to the corrected CALPADS record • Suspense processing clears the suspense (in nightly process, load, or CDE snapshot) Assessments and Accountability v2.0

35. Verify Resolution Review Student List – Online Assessment Reports Status Reports VERIFICATION METHODS Assessments and Accountability v2.0

36. Verify Resolution STATUS REPORTS CDE Only • 50.1 Assessment Upload – Detail • 52.1 Assessment Snapshots - Detail Assessments and Accountability v2.0

37. Verify Resolution 51.1 Assessment Suspense - Summary Description: Displays the LEA and School summary results of the Assessment Suspense process by Administration. Data as of: Last Update field in header, which specifies the date for the last Suspense process included in this report Data Included: For a County, the number of LEAs and Schools with data in suspense is reported. For an LEA/School displayed, the number of records in suspense is reported by Suspense Type (i.e. LEA or CDE) and then broken down within the Suspense Type by the specific conditions that generated the suspense. Report Operation: The data can be displayed by County, LEA, and/or School for all or selected Administrations for a Program. Data can be filtered also by Suspense Type (i.e. CDE, LEA) and its appropriate Suspense Code. Links: None Calculations: Totals Glossary References: None Assessments and Accountability v2.0

38. 3224589451, Grey, Jordan Abdul, Jordan 2274589451, 5288589451, Ferrell, J Verify Resolution 51.2 Assessment Suspense – Student Detail Description: Displays the LEA and School results of the Assessment Suspense process for individual students for a Programs and Administration. Data as of: Last Update field in header, which specifies the date for the last Suspense process included in this report Data Included: Individual students with results record in suspense. Report Operation: The data is displayed for a Program and Administration by County, LEA, and/or School. Data can be filtered also by Suspense Type and its appropriate Suspense Code. Note that all student data is sourced from vendor results records, not from CALPADS ODSR data. Links: None Calculations: None Glossary References: SSID Administration Date Assessments and Accountability v2.0

39. Assessments & Accountability Reports Assessments and Accountability v2.0

40. A/A Reports REPORTS AVAILABLE • The following Assessment reports are available (initially for 2010-2011 data) • Summary LEA results for STAR, CAHSEE and CELDT, including state comparisons • Individual student result detail supporting the summary results for STAR, CAHSEE, and CELDT • Currently enrolled students missing CAHSEE • Currently enrolled students needing CELDT • The following AYP/API reports will be available • LEA and School summary of students included in the AYP/API results by subgroup • Individual students included in the AYP/API results • The following individual student report is available • Individual student assessment history (initially 2010-2011 data) Assessments and Accountability v2.0

41. A/A Reports • Assessment reports use vendor score/result data for an SSID that is matched to CALPADS Student Profile data for that SSID • The Student Profile data display as follows: • For reports that present results data by ADMIN, data valid in CALPADS at the time of testing is used for filters and sorting/displaying report content, while vendor score/result data is used for report content • For reports that present results data for Current Students, data valid in CALPADS currently is used for filters and sorting/displaying report content, while vendor score/result data for the reported test ADMIN is used for report content Assessments and Accountability v2.0

42. ASSESSMENT REPORTS • Report content for a Program Administration can continue to change as long as suspense processing is occurring for that Program Administration • Students can be added when a new match is made through the Suspense Resolution Process, either online or batch, which updates the AADS immediately, as well as when new vendor data are loaded that resolves a record in suspense • Student demographic data on the report can change when the Suspense Process, either nightly or in a data load, refreshes student data in AADS for students whose data have changed in the ODS (Note: Non-Participant only refreshed in Load) • Students can be removed when the Suspense Process, either nightly or in a data reload, is not able to re-validate the student’s enrollment due to changes made in ODS enrollment data that were valid at the time of testing Summary: Nightly suspense process and the suspense process in a reload impact existing matches only if data have changed. Students matched by LEA or CDE will not be unmatched unless enrollment can no longer be verified. However, CALPADS demo data changes will be incorporated. Assessments and Accountability v2.0

43. ASSESSMENT REPORTS ASSESSMENT MENU Two entry points from Assessment Menu Assessments and Accountability v2.0

44. ASSESSMENT REPORTS REPORTS • xxxxxxxxx Assessments and Accountability v2.0

45. ASSESSMENT REPORTS REPORTS (Cont) • xxxxxxxxx Coming CDE Only 50.1 Assessment Upload – Detail 52.1 Assessment Snapshots - Detail Assessments and Accountability v2.0

46. UNDERSTANDING STAR REPORTS Assessments and Accountability v2.0

47. Understanding STAR Reports INTRODUCTION • The STAR Program has one Administration, called ANNUAL • The ANNUAL Administration includes multiple Assessments: • California Standards Tests (CST) • California Modified Assessment (CMA) • California Alternate Performance Assessment (CAPA) • Standards-based Tests in Spanish (STS) Assessments and Accountability v2.0

48. Understanding STAR Reports INTRODUCTION (Cont) • STAR vendor results for the ANNUAL Administration are loaded multiple times • The loads are named P1, P2, P3, and Post P-3 • Loads P2 and P3 overwrite the previous load • The Post-P3 load does not overwrite P3 Assessments and Accountability v2.0

49. Understanding STAR Reports GENERAL CALENDAR - STAR Assumption: Results data is loaded to CALPADS within x days of its receipt by CDE Assessments and Accountability v2.0

50. Understanding STAR Reports GENERAL CALENDAR – STAR (Cont) Assumption: Results data is loaded to CALPADS within x days of its receipt by CDE Assessments and Accountability v2.0