Transitioning to New Assessments Common Core Assessment Workshop Los Angeles County Office of Education March 22, 2012

1. Transitioning to New Assessments Common Core Assessment WorkshopLos Angeles County Office of EducationMarch 22, 2012 Deborah V.H. Sigman Deputy Superintendent of Public Instruction District, School & Innovation Branch California Department of Education

2. Smarter Balanced Assessment Consortium (SBAC) An Ever Changing Landscape • 27 states representing 43% of K-12 students • 21 governing, 6 advisory states

3. Richer assessment of and for learning Use of technology as a tool Adaptive testing Universal access National expertise Preparation for 21st century skills College and career ready students Opportunities and Challenges of Transitioning to New Assessment System

4. Change is difficult Sequencing of activities Technology infrastructure Communication efforts Fiscal climate Opportunities and Challenges of Transitioning to New Assessment System (cont.)

5. SBAC Balanced System

6. Timeline Formative Processes, Tools, and Practices Development Begins Field Testing of Summative and Interim Items/Tasks Conducted Final Achievement Standards (Summative) Verified and Adopted Writing and Review of Pilot Items/Tasks (including Cognitive Labs and Small-Scale Trials) Writing and Review of Field Test Items/Tasks (throughout the school year) Summative Master Work Plan Developed and Work Groups Launched Procurement Plan Developed Content and Item Specifications Development Preliminary Achievement Standards (Summative) Proposed and Other Policy Definitions Adopted Operational Summative Assessment Administered Pilot Testing of Summative and Interim Items/Tasks Conducted Common Core State Standards Adopted by All Member States

7. Summative Assessments Today • Each state bears the burden of test development; no economies of scale Measure proficiency against state standards, not agreed-upon standards • Students often leave high school unprepared to succeed in entry-level college courses Usually heavy reliance on multiple choice questions • Inadequate measures of demonstration of skills and complex cognitive performance Each state procures its own assessment system Results often delivered months after tests are given • Tests cannot be used to immediately inform instruction or affect program decisions Accommodations for special education and ELL students vary • Difficult to interpret meaning of scores; concerns about access and fairness Most administered on paper • Costly, time consuming, and challenging to maintain security

8. Using Computer Adaptive Technology for Summative and Interim Assessments Faster results • Turnaround in weeks compared to months today Shorter test length • Fewer questions compared to fixed form tests Increased precision • Provides accurate measurements of student growth over time Tailored to student ability • Item difficulty based on student responses Greater security • Larger item banks mean that not all students receive the same questions Mature technology • GMAT, GRE, COMPASS (ACT), Measures of Academic Progress (MAP)

9. Assessment System Components Summative Assessment (Computer Adaptive) • Assesses the full range of Common Core in English language arts and mathematics for students in grades 3–8 and 11 (interim assessments can be used in grades 9 and 10) • Measures current student achievement and growth across time, showing progress toward college and career readiness, allowing for growth models • Includes a variety of question types: selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks

10. Assessment System Components Interim Assessment (Computer Adaptive) • Optional comprehensive and content-cluster assessment to help identify specific needs of each student • Can be administered throughout the year • Provides clear examples of expected performance on Common Core standards • Includes a variety of question types:selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks • Aligned to and reported on the same scale as the summative assessments • Fully accessible for instruction and professional development

11. Assessment System Components • Extended projects demonstrate real-world writing and analytical skills • May include online research, group projects, presentations • Require 1–2 class periods to complete • Included in both interim and summative assessments • Applicable in all grades being assessed • Evaluated by teachers using consistent scoring rubrics Performance Tasks The use of performance measures has been found to increase the intellectual challenge in classrooms and to support higher-quality teaching. -Linda Darling-Hammond and Frank Adamson, Stanford University “ ”

12. Assessment System Components Formative Assessment Practices • Research-based, on-demand tools and resources for teachers • Aligned to Common Core, focused on increasing student learning and enabling differentiation of instruction • Professional development materials include model units of instruction and publicly released assessment items, formative strategies “ ” Few initiatives are backed by evidence that they raise achievement. Formative assessment is one of the few approaches proven to make a difference. - Stephanie Hirsh, Learning Forward

13. Assessment System Components Online Reporting “ • Static and dynamic reports, secure and public views • Individual states retain jurisdiction over access and appearance of online reports • Dashboard gives parents, students, practitioners, and policymakers access to assessment information • Graphical display of learning progression status (interim assessment) • Feedback and evaluation mechanism provides surveys, open feedback, and vetting of materials ” Data are only useful if people are able to access, understand and use them… For information to be useful, it must be timely, readily available, and easy to understand. –Data Quality Campaign

14. Support for Special Populations Common- Core Tests to Have Built-in Accommodations - June 8, 2011 Accurate measures of progress for students with disabilities and English Language Learners Accessibility and Accommodations Work Group engaged throughout development Outreach and collaboration with relevant associations

15. How the System Fits Together

16. Administration and Accessibility Enhance the Vision • Support the Technology, Item Development and Test Design work groups as they incorporate the principles of accessibility and universal design into the design of the Smarter Balanced system • Identify the variables, attributes and components of tests that need to be dynamic to address the full range of student needs

17. Accessibility and Administration Build Consensus and Systems • Facilitate consensus among member states regarding common definitions of ELL and SWD, and common accommodations for ELL and SWD • Draft manuals and materials to support state implementation of pilot and field test as well as operational test • Materials will be used as part of an iterative design, build and revise approach to technology called agile development • They will also be used to support the development of professional learning modules and other formative tools

18. Technology Identify Technology Needs • Technology readiness application available for states, districts and schools to enter data regarding hardware, software, bandwidth, staffing, electrical systems and other infrastructure required for online testing • Data will be compared against minimum and recommended requirements • Application will support progress tracking • Data useful for state and national policymakers considering total cost of ownership of a high-quality assessment system

19. Update – California Technology Readiness Tool Rollout – Spring 2012 • Technology Readiness Data Collection Schedule – Spring 2012 • March 20 – April 9 ~ Pilot Districts Test Data Collection: • April 9 – 16 ~ Enhance Communications for the Technology Readiness Tool Survey: • April 16 – June 14 ~ Begin Statewide LEAs Data Collection • Technology Readiness Reporting – Spring 2012 the first data collection window will be a technology readiness inventory. Data collection summaries will be available April 9th. These reports will help improve data quality and to inform the consortia regarding which types of hardware and operating systems are currently being used in schools.

20. Technology Build the System • Provide the system based on the system architecture • Applications will include (subject to architecture): • Item bank (based on Michigan Item Bank) • Test delivery • Reporting / hub • Digital library with formative assessment practices resources, curriculum resources and interactive collaboration for Smarter Balanced users

21. Item and Test Design Write the Items • Item and test specifications will be used to drive item writing • Item specifications: ensure items are accessible and in the right form and format • Test specifications: ensure the right number of items will be written so the pool is sufficient • Item writing led by vendors, states and Smarter Balanced • Balance of item-writing burden will likely change from short-term to the long-term • Item writing in short-term needs to be aggressive to build the initial pool; time and volume will be a driving factor • Long-term, other priorities can take precedence

22. Draft Assessment Claims for English Language Arts/Literacy

23. Assessment Claims for English Language Arts/Literacy Cont.’d

24. Draft Assessment Claims for Mathematics

25. Draft Assessment Claims for Mathematics (cont.)

26. Sample Items: Technology Enhanced and Constructed Response 26

27. Test Question Types & Designs 27

28. Sample Items (cont.) 28

29. Sample Items (cont.) • Item a – Geometry – 7.4 – Know the formulas for the area and circumference of a circle and use them to solve problems; give an informal derivation of the relationship between the circumference and area of a circle. • Item b – Ratios and Proportional Relationships - 7.3 -Use proportional relationships to solve multistep ratio and percent problems. 29

30. Sample Performance Task • Phase 1/Day 1: Directions: Write an imaginary or real story telling about someone losing a tooth. Be sure to have a beginning, middle, and ending. • Phase 2/Day 2: Students will hear selected read-aloud text. Directions: Using your notes, write two to four paragraphs describing and comparing different tooth traditions. Be sure to • (1) tell some things that are the same about the traditions, • (2) tell some things that are different about the traditions, and • (3) explain something interesting that you learned OR tell about your tooth tradition. 30

31. Sample Performance Task (cont.) • Standards Assessed with this Task • Writing Standards: • W.3.2. Write informative/explanatory pieces (a-d) • W.3.3. Write narratives (a-d) • W.3.5. With guidance and support from peers and adults, strengthen writing as needed by revising and editing. (formative evidence) • W.3.6. With guidance and support from adults, use technology to produce and publish writing. • Speaking and Listening Standards: • SL.3.2. Identify the main ideas and supporting details of information presented graphically, visually, orally, or multimodality. • SL.3.4. Report on a topic or recount stories or experiences with appropriate facts and descriptive details. • Language Standards: • L.3.1. Observe conventions of grammar and usage. • L.3.2. Observe conventions of capitalization, punctuation, and spelling. • L.3.3. Make effective language choices. a. Use words for effect. 31

32. Formative Practices, Professional Learning and Implementation Understand the CCSS • Teams of teachers from each state will: • Participate in identifying formative assessment practices and curriculum resources to put in Digital Library • Participate on a committee to complete voluntary alignment review of publishers’ materials to the content specifications and develop a “Consumers Report” to upload to the Digital Library • National content experts to develop 54 (3 ELA and 3 math per grade) formative assessment practices exemplar modules that provide model products for Smarter Balanced teachers (housed in Digital Library) • Existing CCSS curriculum projects are adapted to align with the Smarter Balanced content specifications (and uploaded to the Digital Library)

33. Formative Practices, Professional Learning and Implementation Understand the Test • Produce high-quality test manuals that include administration guidelines and supports for teachers and students • Support administration of test consistent with its purpose and intended use of data • Offer trainings on how to administer the test, provide accommodations, use reporting system and other applications • Enhance assessment literacy by providing well articulated training on interpreting assessment results • Support connections with pre-service teachers

34. Formative Practices, Professional Learning and Implementation Improve Teaching and Learning • Provide comprehensive support for formative assessment, including instructional modules aligned with CCSS • Training modules help teachers focus their instruction on the CCSS and develop teaching practices that support more in- depth learning • Enhance assessment literacy by training teachers to use formative assessment tools and interim assessment to determine next steps in instruction • Provide supports for students to manage their own learning

35. Timeline Formative Processes, Tools, and Practices Development Begins Field Testing of Summative and Interim Items/Tasks Conducted Final Achievement Standards (Summative) Verified and Adopted Writing and Review of Pilot Items/Tasks (including Cognitive Labs and Small-Scale Trials) Writing and Review of Field Test Items/Tasks (throughout the school year) Summative Master Work Plan Developed and Work Groups Launched Procurement Plan Developed Content and Item Specifications Development Preliminary Achievement Standards (Summative) Proposed and Other Policy Definitions Adopted Operational Summative Assessment Administered Pilot Testing of Summative and Interim Items/Tasks Conducted Common Core State Standards Adopted by All Member States

36. Progress to Date • Major tasks / scope of work • Schedule and description of procurements • Two rounds of public comment • Review by Governing States Master Work Plan for Summative Assessment • RFPs issued for: item/task specifications, item/task materials development; test and CAT specifications; pilot test item/task development Content Specifications for ELA/Literacy & Math • Phase 1 development complete • Architecture review board established Item & Test Development • Delivered presentations to more than 130 groups and organizations IT Systems Architecture • COO; Lead psychometrician; Director of higher education collaboration; Director of strategic communications Communications Staffing

37. Next Six Months • Recommendations for procurement, administration, and maintenance post 2015 • Item/task specifications issued • Rapid prototyping & small scale trials for pilot test Sustainability Task Force • Major tasks / scope of work • Schedule and description of procurements Item/Task Development • Higher education collaboration; Research-based alignment of CCSS and credit-bearing courses Master Work Plans for Interim and Formative • Common accessibility guidelines issued • Convene advisory groups for ELL and SWD Definition of College/Career Readiness • Math and ELA/literacy content specialists Access and Accommodations Staffing

38. Transitioning to New Assessments - STAR Reauthorization Assembly Bill (AB) 250 modifies California Education Code (EC) to address the development and adoption of new curriculum frameworks, instructional materials, professional development practices, and high-quality assessments.

39. Statutory Authorization Requires that the State Superintendent of Public Instruction (SSPI) develop recommendations for the reauthorization of the statewide pupil assessment system, which includes a plan for transitioning to a system of “high-quality” assessments as defined in EC Section 60603.

40. High-Quality Assessments • Means an assessment designed to measure a pupil’s knowledge of, understanding of, and ability to apply critical concepts through the use of a variety of item types and formats, including, but not limited to, items that allow for open-ended responses and items that require the completion of performance-based tasks.

41. High-Quality Assessments • A high-quality assessment should have the following characteristics: • Enable measurement of pupil achievement and pupil growth • Be of high technical quality by being valid, reliable, fair, and aligned to standards • Incorporate technology where appropriate • Include the assessment of pupils with disabilities and English learners • Use, to the extent feasible, universal design principles, as defined in Section 3 of the federal Assistive Technology Act of 1998

42. Requirements of Bill Requires that the SSPI consult with The State Board of Education Public Schools Accountability Act (PSSA) Committee Measurement experts from California private and public universities Individuals with expertise working with students with disabilities and English learners Teachers, administrators, and governing boards from California’s local educational agencies. Parents

43. Requirements of Bill (cont.) Requires that the SSPI’s recommendations include a plan for transition for and address sixteen areas outlined in statute. The recommendations are to be reported to the Legislature on or before November 1, 2012.

44. 16 Areas of Consideration Aligning assessments to standards Implementing common assessments developed by state collaborative Conform to ESEA reauthorization Measurement of achievement at a point in time and over time for groups and subgroups of pupils and for individual pupils Allow for comparison from one year to the next as a reflection of growth over time Valid, reliable and fair for all students including English learners (EL) and students with disabilities

45. 16 Areas of Consideration (cont.) • Assessment of ELs using primary language assessments • Ensure no bias with respect to race, ethnicity, culture, religion, gender, or sexual orientation • Incorporate a variety of item types including open-ended and performance-based tasks • Generate multiple measures of pupil achievement, which, when combined with other measures, can be used to determine the effectiveness of instruction and the extent of learning • Assess science and history-social science in all grade levels at or above grade 4

46. 16 Areas of Consideration (cont.) • Assess understanding and ability to use technology necessary for success in the 21st century classroom and workplace • Formative and interim assessments that provide timely feedback for purposes of continually adjusting instruction to improve learning • Use test administration and scoring technologies that will allow the return of test results to parents and teachers as soon as is possible • Minimize testing time • Options for diagnostic assessments for pupils in grade 2

47. Reauthorization of Statewide Assessment System - CDE Contact Information Diane Hernandez Interim Division Director Assessment Development & Administration Division DHernand@cde.ca.gov 916-445-9449 Jessica Barr AB 250 Lead Assessment Development & Administration Division jbarr@cde.ca.gov 916-319-0364

48. SBAC - CDE Contact Information 48 Diane Hernandez Interim Division Director Assessment Development & Administration Division DHernand@cde.ca.gov 916-445-9449 Kristen Brown, PhD SBAC Coordinator Education and Research Evaluation Consultant Assessment Development & Administration Division kbrown@cde.ca.gov 916-319-0334 Join the SBAC CDE electronic mailing list by sending a blank e-mail to: subscribe-sbac@mlist.cde.ca.gov