BCN Update for Transitioning to a New Assessment System

1. BCN Update for Transitioning to a New Assessment System Bilingual Coordinator’s NetworkNovember 15, 2012Patrick Traynor, Ph.D.Director, Assessment Development & Administration Division

2. The STS Pre AB 1521 Standards-Based Tests in Spanish (STS) • Required for Spanish-speaking English learners (ELs) who either: • Receive instruction in their primary language; or • Have been enrolled in a school in the U.S. less than 12 months • May also be administered, at the option of the school district, to any other Spanish-speaking ELs • Not available for non-ELs in dual immersion programs

3. AB 1521 (Brownley) Approved by Governor September 21, 2012 • Upon approval of the state board, the department may make the STS available to pupils enrolled in a dual language immersion program: • Nonlimited English proficient or • Redesignated fluent English proficient. • Administer the assessment at own expense and enter into an agreement with the state testing contractor • The cost for the assessment will be the same for all school districts and charter schools

4. AB 1521 (Brownley) ETS Options for Implementation: • Survey schools and districts to determine number of students • 2013 Possible Scenarios: • High risk making edits to order management systems • Flag assessments and answer documents • Fold into current order management process • Manual Operations • Add separate ordering process for dual immersion students • Return to ETS for hand scoring • Provide raw score results

5. AB 1521 (Brownley) ETS Options for Implementation: (cont) • 2014 Possible Scenarios: • Incorporate in current order management system • Provide On-line Pre-ID services • Monitor inventory controls for smooth deliveries • Provide a flag on the answer document to identify hand marked dual immersion test takers • Address change in law in manuals and directions for administration

6. Alternate Assessment Participation • California recently joined the National Center and State Collaborative (NCSC) as a Tier II state • Developing alternate assessments for students with significant cognitive disabilities • Based on alternate achievement standards

7. Reauthorization • Standardized Testing and Reporting (STAR) Program sunsets July 2014 • California Education Code 60604.5 amended • Extensive consultation with stakeholders • Outreach plan activities

8. Proposed Web Site RedesignHome Page

9. Proposed Web Site RedesignCurrent Testing Standards—Filtering Feature

10. Proposed Web Site RedesignCurrent Testing Standards—Information About the Standard

11. Proposed Web Site RedesignCurrent Testing Standards—Sample Question

12. Proposed Web Site RedesignCurrent Testing Standards—Related CCSS Information

13. Proposed Web Site RedesignCommon Core—Filtering Feature

14. Proposed Web Site RedesignCommon Core—Standards Information

15. Proposed Web Site RedesignCommon Core—Comparison to Current Testing Standards

16. Proposed Web Site RedesignCommon Core—Example Related Sample Question

17. Current Assessment System • Originally designed in 1997 • Modifications made over the years • Current STAR system includes: • California Standards Tests • California Modified Assessment • Standards-based Tests in Spanish • California Alternate Performance Assessment • Other parts of state assessment system: • California High School Exit Examination • California English Language Development Test • Physical Fitness Test

18. Smarter Balanced Participation • In June 2011, California joined the Smarter Balanced Assessment Consortium (SBAC) • Developing assessments in English–language arts (ELA) and mathematics in grades three through eight and grade eleven • Optional formative assessment tools and interim assessments

19. Purposes of the New Assessment System • Design an assessment system that includes a variety of assessment approaches and item types that has as its primary purpose to model and promote high-quality teaching and student learning activities.

20. Purposes of the New Assessment System (cont.) In accomplishing the purpose, the system can also: • Produce scores that can be aggregated for the purpose of holding schools and districts accountable for the progress of their students in learning the California academic content standards. • Provide assessments and/or assessment tools in multiple grades covering the full breadth of the curriculum to provide clear expectations and incentives for teaching the full curriculum.

21. Guiding Principles • Assess subjects and learning in ways that promote high-quality instruction • Conform to rigorous industry standards for test development • Use resources efficiently and effectively • Provide for inclusion of all students • Provide information on the assessment system that is readily available and understandable to parents, teachers, schools, and the public

22. Considerations for Developing California’s Future Assessment System • Adoption of the Common Core State Standards and sunset of the STAR Program • Presents a set of challenges and opportunities for the future assessment system • Important to decide what information is expected from the tests • Consider resources • Test subjects beyond ELA and mathematics • Progress or growth from year to year • Plan for how to use the results

23. Strengths of the Current System • Paper and pencil multiple-choice assessments inexpensively developed, administered, and scored • Reliable results • Provide secure measures of achievement • Use of multiple-choice approach has allowed for a wide variety of tests and a high level of reliability and validity in the accountability system

24. Limitations of the Current System • Multiple-choice assessments limits the types of knowledge and skills measured (less depth) • Limited types of items and formats • Criticized for negatively influencing instruction through narrowing of the curriculum • Not sufficient for providing diagnostic information

25. Determining What to Test • CCSS requires a more integrated approach to delivering instruction • SBAC assessments use multiple item types such as multiple-choice, constructed response, technology enhanced, and performance tasks • SBAC will provide optional formative assessment tools and interim assessments • Automated scoring

26. Determining What to Test (cont.) • Costs will likely be greater per student with SBAC assessments than the current system • Need to consider possibly allocating additional resources for assessment, finding more efficient ways to assess subjects not included in SBAC, and reduce the number of grades and subjects assessed • Utilizing various ways to administer, score, and report has the potential for realizing greater efficiencies

27. Determining What to Test (cont.) • Choices beyond SBAC assessments • Accountability considerations • Interim and formative assessment • Reducing redundant testing • Matrix testing • Balance costs of the system and the kind of assessment and reporting desired

28. English Language Arts and Mathematics, Grades 3–8 and High School BEGINNING OF YEAR END OF YEAR INTERIM ASSESSMENT INTERIM ASSESSMENT Computer Adaptive Assessment and Performance Tasks • PERFORMANCE • TASKS • Reading • Writing • Math END OF YEAR ADAPTIVE ASSESSMENT SBAC Timeline Last 12 weeks of year* Formative Assessments DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools. Computer Adaptive Assessment and Performance Tasks Scope, sequence, number, and timing of interim assessments locally determined Re-take option Optional Interim assessment system— Summative assessment for accountability * Time windows may be adjusted based on results from the research agenda and final implementation decisions. 28 Source: http://www.ets.org

29. SBAC Schedule of Activities • Sample item and performance task release • Oct. 9 – Early November 2012 • Small scale trial • Oct. 15 – Nov. 9, 2012 • Pilot test • Feb. 2013 – May 2013, Non-adaptive • Field test • Spring 2014, Adaptive • Operational system • Fall 2014, Adaptive

30. Pilot Test • Pilot Test in early 2013 • Open to all schools in the Consortium • 11% of students from each Governing State to be included in scientific sample 30

31. Purpose of Sample Items and Performance Tasks • Demonstrate rigor and complexity of ELA/literacy and mathematics questions • Showcase variety of item types: • Selected response • Constructed response • Technology enhanced • Performance tasks • Help teachers to begin planning for the shifts in instruction

32. Accessibility and Accommodations • Sample items do not include accessibility and accommodations features • Full range of accessibility tools and accommodations options under development guided by: • Magda Chia, Ph.D., Director of Support for Under-Represented Students • Accessibility and Accommodations Work Group • Students with Disabilities Advisory Committee • Chair: Martha Thurlow (NCEO) • English Language Learners Advisory Committee • Accessibility & Accommodations Framework • Learn more online: • http://www.smarterbalanced.org/parents-students/support-for-under-represented-students/

33. CDE SBAC Web page

34. Sample Items and Tasks SBAC Landing Page

35. Sample Items and Tasks Navigation View mathematics or ELA/literacy items Advance to next item, or go back to previous

36. Sample Items and Tasks Navigation (cont.) Content Claim Grade band

37. Sample Items and Tasks Navigation (cont.) Filter by item type, themes

38. Item Metadata About this item

39. Exploring the Sample Items Selected response and technology enhanced items are machine scorable

40. Exploring the Mathematics Sample Items

41. Exploring the Mathematics Sample Items (cont.)

42. Exploring the ELA Sample Items

43. Exploring the ELA Sample Items (cont.)

44. SBAC Timeline Formative Processes, Tools, and Practices Development Begins Field testing of summative assessment, training school- and district-level staff in formative tools Technology readiness tool available Teams of teachers evaluate formative assessment practices and curriculum resources Full implementation of assessment system Writing and Review of Pilot Items/Tasks (including Cognitive Labs and Small-Scale Trials) Formative tools available to teachers Content and Item Specifications Development Writing and Review Items/Tasks for Field Testing (throughout the school year) Pilot Testing of Summative and Interim Items/Tasks Conducted

45. Resources • CDE/SBAC presentations and an electronic mailing list are found on the CDE SBAC Web page at http://www.cde.ca.gov/sbac/ • Join the SBAC CDE electronic mailing list by sending a blank e-mail to subscribe-sbac@mlist.cde.ca.gov • Visit the SBAC Website at www.smarterbalanced.org • E-mail SBAC Questions to sbac@cde.ca.gov • Email IT Readiness Questions to sbac-itreadiness@cde.ca.gov

46. Patrick Traynor, PhD, Director Assessment Development and Administration Division E-mail: ptraynor@cde.ca.gov Jessica Barr, SBAC and Reauthorization Lead Consultant Assessment Development and Administration Division Phone: 916-319-0364 E-mail: jbarr@cde.ca.gov Jessica Valdez, SBAC Teacher Involvement Coordinator and NAEP Coordinator Assessment Development and Administration Division Phone: 916-319-0332 E-mail: jvaldez@cde.ca.gov Contact Information