Assessment and Accountability Update Chun-Wu Li, Ph.D .

1. Assessment and Accountability Update Chun-Wu Li, Ph.D. Administrator II, Assessment and Accountability Assessment Accountability Continuous Improvement Unit Division of Educational Services December 8, 2017

2. Assessment Update

3. English Language Proficiency Assessments for California (ELPAC)

4. ELPAC Practice Test Enhancements Practice tests will be posted on www.ELPAC.org by October 30. • Provide examples of task types that are administered in each grade/grade span. • Provide a sense of how the 2012 ELD Standards are assessed. • Available to students, parents, teachers and other educators, and interested members of the public.

5. K–12 Reporting Hierarchy for the Summative Assessment

6. K–12 Reporting Hierarchy for the Initial Assessment

7. ELPAC SA Administration and Scoring Trainings • California Department of Education-sponsored statewide trainings: October 24 through December 13 • Optional half-day training to provide regional training hosts with additional support: October 25 • To register, access the registration link on www.ELPAC.org at: https://ord.scoe.net/elpac/pg00.php.

8. Preparing for the ELPAC SA • Test Operations Management System (TOMS) opens mid-October. • SA test materials ordering window: • Training Webcast mid-October October 17, 2017: ELPAC Test Operations Management System (TOMS) Overview Webcast • Round 1, mid-October–November 17, 2017 • Round 2, November 18–December 21, 2017 • Supplemental, January 23–May 15, 2018

9. Preparing for the ELPAC SA (cont.) • Pre-identification (Pre-ID) window: • Training Webcast mid-October October 17, 2017: ELPAC Test Operations Management System (TOMS) Overview Webcast • Round 1 Pre-ID Label order period (38¢ each) January 2–April 9, 2018 • Round 2 Pre-ID label order period (52¢ each) April 10–May 14, 2018 • Pre-ID highlights • No set up fees are charged to LEAs. • Pre-ID labels are ordered through TOMS. • Purchase order required.

10. New ELPAC Resources • “Do We Administer the California English Language Development Test (CELDT) or the ELPAC in 2017–18?” • “A Parent Guide to Understanding the ELPAC” • Translated into seven languages • ELPAC Assessments Fact Sheet • Domain Information Sheets • Matrix 4: Universal Tools, Designated Supports, and Accommodations for the ELPAC These resources are located at http://www.cde.ca.gov/ta/tg/ep/

11. New Resources http://www.cde.ca.gov/ta/tg/ep/documents/howtotestel.pdf http://www.cde.ca.gov/ta/tg/ca/documents/elpacassessmenrfs.pdf

12. New Resources (cont.)

13. New Resources (cont.) http://www.cde.ca.gov/ta/tg/ep/elpacparentguide.asp

14. California Assessment of Student Performance and Progress (CAASPP)

15. CAASPP Summative Assessment

16. Back-to-School Toolkit http://www.cde.ca.gov/ta/tg/ca/backtoschoolkit.asp California Department of Education

17. Summative Assessment Fact Sheet Assessment Fact Sheet Web page available at http://www.cde.ca.gov/ta/tg/ca/assessmentfactsheets.asp California Department of Education

18. A Parent Guide to Understanding A Parent Guide to Understanding for each assessment is available at http://www.cde.ca.gov/ta/tg/ca/parentguidetounderstand.asp California Department of Education

19. CAASPP Interim Assessment

20. Interim Assessment Fact Sheet http://www.cde.ca.gov/ta/tg/ca/documents/sbinterimfactsheet.pdf California Department of Education

21. 2017–18 Interim Assessments http://www.cde.ca.gov/ta/tg/sa/documents/iaboverview2017.pdf California Department of Education

22. Interim Assessment Hand Scoring Tables http://www.cde.ca.gov/ta/tg/sa/documents/iasbygrade2017.pdf California Department of Education

23. Interim Assessment Viewing System http://www.caaspp.org/ta-resources/interim.html California Department of Education

24. Manner of Administration—Set for Entire Test Session http://www.caaspp.org/rsc/pdfs/CAASPP.Manner-of-Administration-Tip-Sheet.pdf California Department of Education

25. The New Interim Assessment Reporting System! https://reporting.smarterbalanced.org/ California Department of Education

26. Interim Assessment Reporting System—Key User Roles Assigned through the Administration and Registration Tools (ART) interface: • “PII” user role—Assign only to staff who need access to all students within the school/LEA. • “PII_GROUP” user role (NEW!)—Assign to staff who need access to results for a subset of students. ____________________________________________________________________________________________________________________ • “GROUP_ADMIN” user role (NEW!)—Assign to school and district administrators who will be responsible for creating and managing student groups. For further details, see the e-mail dated September 11 from CalTAC to LEA CAASPP coordinators, or contact CalTAC.

27. Key Features • Create and manage student groups • View aggregate student results • Compare test results by student group • Use advanced filtering options • View student responses to test items • View detailed item information • View test-item results within a group • Access links to related Digital Library resources

28. View Student Responses to Items and Item Information

29. Accountability Update

30. SBE Action on Academic Indicator • Approved: • New Change cut scores for English language arts/literacy (ELA) and mathematics • New High and Medium Status cut scores for mathematics • New color designation for five-by-five grid California Department of Education

31. Background on Academic Indicator • Based on Smarter Balanced Summative Assessments results for ELA and mathematics • Fall 2017 Dashboard uses data from spring 2017 administration for grades 3–8 • Status and Change calculated using “Distance from Level 3” (DF3) California Department of Education

32. Distance from Level 3 • Distance between: • Student’s score on Smarter Balanced Summative Assessments, and • Lowest possible scale score* for Standard Met Achievement Level (Level 3) *Scale scores vary by content area and grade level. California Department of Education

33. DF3 Example: Grade Five Mathematics • Grade five student received a score of 2,505. Student scored 2,505. This is 23 points below the lowest possible score for Level 3. (2,528 – 2,505 = 23 points) California Department of Education

34. Volatility in Scale Scores • Performance standards based on two years of data (2015 and 2016) • Distributions positively skewed: Over 80 percent of LEAs had positive change. • 2017 Change distribution more symmetrical: 45 percent of LEAs had a positive change California Department of Education

35. Produces Dramatic Downward Swings • High performing schools must sustain continued improvement that may be untenable. • Some 200 schools at “High” or “Very High” Status level dropped two performance levels in 2017 (from Blue to Yellow), even as most of their students continued to meet or exceed state standards. California Department of Education

36. Produces Dramatic Upward Swings • Schools with Very Low Status that improve by just 15 points from prior year could jump two performance levels (i.e., Red to Yellow). • Applying current cut scores and methodology to 2017 DF3 would result in 1,117 schools (15 percent) moving up or down at least two performance levels in ELA. California Department of Education

37. California Department of Education

38. Rationale: Revisions to Change Cut Scores • Normal distribution supports symmetrical cut scores • 2016 cut scores (“Maintained” level): ˂ -1 to ˂ +7 (ELA) ˂ -1 to ˂ +5 (mathematics) • Revised Change cut scores: ˂ -3 to ˂ +3 (both ELA and mathematics) California Department of Education

39. Revisions to Mathematics Cut Scores California Department of Education

40. Rationale: Revisions to Status Cut Scores for Mathematics • Ensure that LEAs and schools cannot receive a “High” Status unless they have a positive DF3: • Addresses issues raised by advocacy groups after release of Spring 2017 Dashboard California Department of Education

41. New ELA Five-by-Five Colored Table California Department of Education * Change in color designation

42. New Mathematics Five-by-Five Colored Table California Department of Education * Change in color designation

43. Rationale: Revisions to Color Designation • LEAs and schools that maintain their Status can only be assigned one of two colors: • School with Very High Status can only move from Blue to Green if it declines. • School will Very Low Status can only move from Red to Orange if it increases.   California Department of Education

44. Feedback on Academic Indicator • What information should be included in a one-page flyer on the Academic Indicator: • Should contain graphics • Should be clear for general public California Department of Education

45. SBE Action on Chronic Absenteeism Indicator • Include information in Fall 2017 Dashboard that redirects users to Chronic Absenteeism reports on DataQuest. • Direct CDE staff to develop recommendation for: • March 2018 SBE meeting: Proposed Status cut scores to be used to update Fall 2017 Dashboard • September or November 2018 SBE meeting: Proposed Change cut scores California Department of Education

46. Feedback on College/Career Indicator (CCI) • On Equity page of the Dashboard, CCI row reflects “N/A” • Causing confusion and generating many questions. • Is there more explanatory language that we can use? (Example: Status Only) California Department of Education

47. California Department of Education

48. New Communication Materials California Department of Education

49. Flyers on the Dashboard • Sacramento County Office of Education is developing a series of one- and two-page flyers on the Dashboard: • Five-by-Five Colored Grids • Academic Indicator • CCI • Dashboard Alternative School Status (DASS) • English Learner Subgroups California Department of Education

50. Dashboard Release Schedule California Department of Education