Assessment and Accountability Update Chun-Wu Li, Ph.D .

1. Assessment and Accountability Update Chun-Wu Li, Ph.D. Administrator II, Assessment and Accountability Assessment Accountability Continuous Improvement Unit Division of Educational Services May 18, 2018

2. Assessment Update (See Handout “CDE Program Updates – May 2018”)

3. ELPAC Update (See Handout “ELPAC Update – May 2018”)

4. Timeline of Initial ELPAC Activities • February 2018––Initial English Language Proficiency Assessments for California (ELPAC) Standard Setting • May 2018––Threshold Scores approved by State Board of Education • July 2018 ––Finalize the student score report (SSR) with new reporting scale • July 9, 2018––Local educational agencies (LEAs) begin local scoring • Fall 2018––Educators to participate in threshold review process • May 2019––adjustments to threshold scores if necessary for implementation in 2019–20 California Department of Education

5. Parent/Guardian Notification of Testing LEAs must now notify parents/guardians in writing prior to administering the ELPAC. Three options for LEAs to notify parents/guardians: • California Department of Education (CDE)-produced Letter of Notification template • A copy of the home language survey highlighting appropriate sections • Student handbook or the kindergarten registration packets, which includes information regarding ELPAC testing California Department of Education

6. LST Process • The LEA submits the student’s enrollment and Student English Language Acquisition records to the California Longitudinal Pupil Achievement Data Systems (CALPADS). • The student is administered the Initial ELPAC. • The Test Examiner scores the Initial and completes the student Score Sheet in the back of the Answer Book. • Designated staff enters the raw scores into the local scoring tool (LST). • The Student Score Report (SSR) is produced by the LEA (printing is optional) with official scores. • Designated staff produce an exportable Excel file for merging into a required Title I/III letter that will be sent to parents/guardians. California Department of Education

7. Initial ELPAC SSR

8. Summer Scoring of Initials • Two options: • Request a Statewide Student Identifier and submit English language acquisition status record of TBD; enter raw scores into the LST to generate an official score • Score by hand and hold until the student arrives on the first day of school • A preliminary score template will be provided • Follow the steps in the first option to generate an official score California Department of Education

9. Rotating Score Validation Process • July 1–October 31, 2018––Collect Initial ELPAC test materials • September 3–December 31––Submit Initial ELPAC test materials • Participating LEAs to receive a comparison report • The report can be used as evidence collected for the Correction of Classification Process California Department of Education

10. Correction of Classification Errors Per the California Code of Regulations 11518.20: • A correction process may be requested by a parent/guardian or a certificated employee of the LEA after the administration of the Initial ELPAC, but before the Summative ELPAC. • The LEA must collect and review evidence about the student’s English language proficiency. • Based on this review, the LEA determines whether the pupil’s classification, initial fluent English proficient or English learner should remain unchanged, or be changed in CALPADS. • The LEA notifies the parent/guardian of the results. California Department of Education

11. Correction of Classification Errors: Three Scenarios • If a student is classified as English only, but the LEA has an indication that the pupil’s primary or native language is not English and the student is unable to perform ordinary classroom work in English • If an LEA administers an Initial or Summative ELPAC to a student who is not eligible for the assessment • After the administration of the Initial ELPAC to a student, but before the student takes the Summative ELPAC

12. New Parent/Guardian Resources • “Understanding Your Student Score Report” for the Summative ELPAC will be available by July 2018 • “Introduction to the Initial ELPAC for Parents” video posted by end of July 2018

13. Considerations to Increase Accessibility The CDE is evaluating several features based on • stakeholder feedback • cognitive lab studies • Every Student Succeeds Act regulations • Pausing between/repeating test items • Adding use of assistive devices • American Sign Language as a mode of delivery for prompts/questions • Domain exemptions • Federal government will be releasing Peer Review Guidance for evaluating a state’s English language proficiency assessment • Revised ELPAC Regulations are planned to be brought to the State Board of Education in July California Department of Education

14. Accountability Update (See Handout “Accountability Update – May 2018”)

15. Topics • Growth Model • Dashboard Reports versus Data Reported to the U.S. Department of Education (ED) • Changes to 2018 California School Dashboard (Dashboard) • Incorporation of participation rate into Academic Indicator

16. Student Level Growth Model

17. SBE Action • Approved Residual Gain for further exploration. • Requested further information at July SBE meeting: • What are the benefits of the residual gain model over the distance from level 3 already included in the Dashboard? • Should parent education level be included as a variable in the model? • How should the model be used (i.e.,in the Dashboard for accountability, reporting only, etc.)

18. Residual Gain (RG) • Represents difference between student’s predicted and actual test score • Predicts student’s current-year score in either mathematics or ELA using the student’s prior-year mathematics and ELA scores. RG = Student’s current year score – predicted score • Provides a relative measure of student growth on the current test scale.

19. How Does RG Work: Results for One Student

20. Next Steps (May – September) • May 30 2018 Webinar: Obtain feedback on the growth model interest • June 2018: • Obtain feedback on growth model from LCFF stakeholder group • Review RG model for needed technical refinements at the TDG meeting • July 2018 SBE Meeting: Present growth model simulation results and any recommended refinements and discussion around expected metrics and indicator placement. • September 2018 SBE Meeting: SBE approval of final model and methodology for possible inclusion in the 2018 Dashboard.

21. Feedback and Questions • Is there still interest in a student growth model? • If so, how would districts and county offices use the information in growth model to improve classroom instruction? • What is the best option for reporting the growth model results? • The Change component for DF3 • Separate Indicator • Detailed Reports Only • Other

22. One-Year Graduation Rate

23. SBE Action and Next Steps • At the May 2018 meeting, the SBE approved a one-year graduation rate for DASS schools for implementation beginning with the 2018 Dashboard. • Next Steps: Work with the Alternative Schools Task Force, the LCFF Stakeholder Group, and the Technical Design Group (TDG) to determine: • How to include the one-year graduation rate in the district’s Dashboard. • If other indicators need to have modified methods • If indicators for DASS schools need their own cut scores.

24. Data Reported in 2018 Dashboard Reports versus Data Reported to U.S. Department of Education

25. Every Student Succeeds Act (ESSA) Requirements • Three required indicators for which states must set goals: • Academic Achievement (ELA and Math assessments) • Graduation Rate • English Language Proficiency • ED’s statutory interpretation is that these indicators must be based only on current year data (Status)

26. Revisions to ESSA Plan to Meet Federal Requirements • Treat “Status” and “Change” components of the Dashboard Indicators as separate indicators for federal purposes • Status is the required indicator • Change is an additional indicator • Set goal relative to Status

27. ESSA Long-Term Goals • For federal purposes, only Status will be used to set long-term goal: • Graduation rate = 90 percent • ELA (grades 3-8) = +10 Distance From Level 3 (DF3) • Math (grades 3-8) = zero DF3 • English Learner Proficiency = 75 percent of students make progress • Technical work must be completed before determining ELA and math goals for grade 11

28. Elements of ESSA Plan that Remain Intact • Calculate Dashboard indicators using both status and change • No changes to the five-by-five color-coded grids that the SBE approved for the Dashboard Indicators • Use color-coded performance levels to identify the lowest performing five-percent of Title I schools • Based on both Status and Change • The goals for these indicators are not materially different in practice

29. Changes to the 2018 Dashboard

30. Chronic Absenteeism • Cut scores for both Status and Change will be recommended to the SBE in fall 2018 • 2018 Dashboard will report for the first time: • Status, • Change, and • Performance levels (colors) • Cut scores will be determined at the November SBE meeting

31. New Data Reported for CCI • 2018 Dashboard will report: • Status (Class of 2018) and Change • Performance levels (colors) • Possible new career measures for the Class of 2017 • State Seal of Biliteracy • Articulated Career Technical Education (CTE) courses • Golden State Seal Merit Diploma • Leadership/Military Science (formerly ROTC) • Change cut scores will be determined at the November SBE meeting

32. Graduation Rate Indicator • Calculation for Change based on one-year rate (rather than the three-year average). • Note: New federal requirements necessitate changes to the calculation of the four-year cohort rate. See Flash 136 at https://www.cde.ca.gov/ds/sp/cl/calpadsupdflash136.asp • Incorporation of five-year cohort rate: To be determined • Scheduled for consideration at the September SBE meeting

33. Safety Net Methodology Applied at Student Group Level (n ≥ 150) • For two state indicators: • Graduation Rate Indicator: If student group has fewer than 150 students in graduating cohort • Suspension Rate Indicator: If student group has fewer than 150 students who are cumulatively enrolled. • SBE will consider at the July SBE meeting

34. Incorporation of Participation Rate into Academic Indicator

35. Incorporation of Participation Rate into Academic Indicator • In accordance with ED direction: • Use of icons not sufficient to meet ESSA requirements • Participation rate must be factored into Academic Indicator • Specifically, for schools and student groups that did not test 95 percent, the number of students needed to bring the school and/or student group up to 95 percent shall be included in the Academic Indicator.

36. Proposed Option • Calculate the percentage points the school or student group is below the 95 percent participation rate target and reduce the DF3 by a fraction of the number of points short (i.e., ¼ the number of points short of participation rate)

37. ELA Example Emerald Elementary School (K-6): Current DF3 = 16.9

38. Example: Calculation of the Adjusted DF3 • For this example, the DF3 is being reduced by ¼ point for each percentage point the school is below the 95 percent participation rate target. • Adjusted DF3 Calculation 16 points shortage X .25 = 4 points 16.9 Current DF3 – 4 points reduction = 12.9 (High)

39. Pros and Cons—Using Adjusted DF3 • All entities receive equal penalty for missing the 95 percent participation rate. • The further away an entity is from the 95 percent participation rate, the greater the penalty. • Simple to calculate. • Minimizes the impact of the of incorporating the participation rate into the Academic Indicator. • May be more complicated to explain.

40. Assessment and Accountability Related Items

41. 2018-2019 RCAN Dates 2018-2019 RCAN Dates: • Friday, August 17, 2018 • Friday, October 5, 2018 • Friday, December 7, 2018 • Friday, January 18, 2019 • Friday, March 22, 2019 • Friday, May 31, 2019

42. Upcoming Conferences/Meetings/Trainings May 22, 2018: CAASPP “Principles of Scoring and Reporting,” Webcast. May 30, 2018: “2018 California School Dashboard,” Webinar. June 5, 2018: CAASPP “The Results Are In—Now What? Analyzing Evidence to Inform Teaching and Learning,” Webcast OR In-Person Training (May 30th– Irvine, May 31st– Downey, and June 13th– San Diego). September 6 or September 7, 2018: “ELPAC Academy,” RCOE. September 25-26, 2018: “New CAASPP and ELPAC Coordinator Training,”Ontario, 2018. September 27, 2018: “State Assessment and Accountability Information Meeting,” Ontario, 2018. October 9, 2018: “CAST Academy,” RCOE. October 23-26, 2018: “NGSS Exploring Performance Assessments in Science (EPAS) Workshop,” RCOE (10/23-24 for ES; 10/25-26 for MS/HS). October 29-30, 2018: “CAASPP Institute,” Riverside Convention Center. November 7-8, 2018: “K-12 BEAL Mathematics Training,” UCR-Extension. November 12-14, 2018: “CERA Conference,” Anaheim.