Assessment Literacy Module

1. California Department of Education Tom Torlakson, State Superintendent of Public Instruction Assessment Literacy Module Unit 2: California State Assessment System

2. Welcome to Unit 2 The purpose of this unit is to increase your knowledge of the components of California’s new state assessment system.

3. Learning Objectives for Unit 2 By the end of this unit, participants will be able to: Explain the purpose and characteristics of the new assessment system for California, the Measurement of Academic Performance and Progress (MAPP) Describe plans for aligning components of the CA CCSS and new assessment system that are unique to English learners and students with disabilities

4. Standardized Testing and Reporting Program A brief review of California’s Standardized Testing and Reporting (STAR) Program (ending in the 2013–14 school year) provides a backdrop for a new CCSS-aligned statewide assessment system.

5. Standardized Testing and Reporting Program STAR Program assessments were first implemented in 1998 and are aligned to the 1997 academic content standards. The majority of test questions are multiple-choice, with some writing samples. The STAR Program consists of four components: The California Standards Tests (CSTs) The California Modified Assessment (CMA) The California Alternate Performance Assessment (CAPA) The Standards-based Tests in Spanish (STS)

6. Standardized Testing and Reporting Program STAR tests were designed to produce (1) school-, district-, county-, and state-level results that allow the state to monitor each school’s progress toward established state performance targets; and (2) results that allow the federal government to monitor the annual yearly progress of schools and local educational agencies (LEAs) toward meeting the accountability targets of the federal Elementary and Secondary Education Act (ESEA), also known as No Child Left Behind (NCLB). Additional information is available on the STAR Program Resources Web page at http://www.cde.ca.gov/ta/tg/sr/resources.asp.

7. New State Assessment System • The new statewide assessment system was informed by the efforts of the State Superintendent of Public Instruction (SSPI), with input from a statewide work group, through a publication titled, Recommendations for Transitioning California to a Future Assessment System (2013). • The new system is being developed by a multi-state assessment consortium with capabilities to meet those recommendations.

8. New State Assessment System The urgent calls for change were taken into consideration when the SSPI’s recommendations were developed and sent to the California State Legislature and the Governor for consideration in the re-authorization of the STAR Program. SSPI Torlakson (2013) specifically recommended an assessment system: Whose primary purpose would be to model and promote high-quality teaching and student learning activities That would include a variety of assessment approaches and item types That would be aligned with college and career readiness, address 21st Century Skills, and have meaning for students

9. New State Assessment System Read the following quotes. Notice the repeated references to 21st Century Learning Skills. “The California Common Core State Standards ask students to acquire a deeper knowledge of the subjects they study and be able to perform more complex tasks using what they have learned. It is critical that we have assessments that measure their progress toward these goals (iv). "The concept is simple but powerful: If our assessments require students to use problem solving and critical thinking skills to perform well, those same skills are much more likely to be taught in our classrooms day in and day out. The goals we set for our assessment system have profound implications for our students and our schools (iv).” –Torlakson2013

10. New State Assessment System "At the heart of the recommendations is a clear vision and commitment to establishing a bold and innovative assessment system that includes a variety of assessment approaches and item types that model and promote high-quality teaching and student learning and sets a course to ensure that all California students are well prepared to enter college and careers in today's competitive economy (x). "The CA CCSS call for a more integrated approach to delivering instruction across all subjects. The CA CCSS are designed to be robust and relevant to the real world…They require student collaboration; fluency with multimedia and technology; and the development of strong complex reasoning, problem solving, and communication skills. High-level skills such as these transcend subjects and demand a reexamination of the state's existing system of professional learning, curriculum development, assessment, and accountability (1)."

11. New State Assessment System Respond to the following: 1. What is one critical difference between the 1997 California academic content standards and the CA CCSS? 2. Summarize the purpose and characteristics of a CCSS-based assessment system in the overall instructional cycle as described in Recommendations for Transitioning California to a Future Assessment System.

12. New State Assessment System The entire report is available through the Web link below: Recommendations for Transitioning California to a Future Assessment System: A Report by State Superintendent of Public Instruction Tom Torlakson http://www.cde.ca.gov/ta/tg/sa/documents/suptrecrpt2013.pdf#search=Transition%20Report&view=FitH&pagemode=none

13. MAPP Legislation Assembly Bill 484 outlines a new CCSS-based assessment system for California, known as the Measurement of Academic Performance and Progress (MAPP). The text of the legislation emphasizes that the purpose of assessment is to promote quality instruction and learning that align with the CA CCSS, CCR Anchor Standards, and 21st Century Skills. “It is the intent of the Legislature in enacting this chapter to provide a system of assessments of pupils that has the primary purposes of assisting teachers, administrators, and pupils and their parents; improving teaching and learning; and promoting high-quality teaching and learning using a variety of assessment approaches and item types.” “California should adopt a coordinated and consolidated testing system that will develop statewide academically rigorous content and performance standards that reflect the knowledge and complex skills that pupils will need in order to succeed in the information-based, global economy of the 21st century.” —Assembly Bill 484 2013

14. MAPP Legislation The full text and status of the legislation is available through the Web link below: Assembly Bill 484 http://leginfo.legislature.ca.gov/faces/billNavClient.xhtml? bill_id=201320140AB484

15. MAPP Legislation As you watch this video, consider what 21st Century Skills the job applicant is missing that are important to the potential employer: “What's wrong with this picture?" http://www.youtube.com/watch?feature=player_embedded&v=dY2mRM4i6tY 1. What makes this video simultaneously humorous and realistic? 2. What implications does this video have for you and how you will support 21st Century Skills and implementation of the CA CCSS? 3. How do components of MAPP, based on Recommendations for Transitioning California to a Future Assessment System, support implementation of the CA CCSS?

16. Smarter Balanced Assessments Smarter Balanced Overview California is a governing state in the Smarter Balanced Assessment Consortium (SBAC). The goal of SBAC is to develop a balanced, coherent student assessment program that is aligned to the CCSS and measures student progress toward college and career readiness. Beginning in 2014–15, Smarter Balanced assessments will be administered to students annually in grades 3–8 in English language arts (ELA) and mathematics, and once in grade 11 as required by federal law. Unlike STAR assessments, they will incorporate formative, interim, and summative assessments designed to support instruction and learning.

17. Smarter Balanced Assessments Smarter Balanced Overview Map of participating Smarter Balanced states: http://www.smarterbalanced.org/about/member-states/ “Smarter Balanced is guided by the belief that a balanced, high-quality assessment system-including formative, interim, and summative components-can improve teaching and learning by providing information and tools for teachers and schools to help students succeed. Timely and meaningful assessment information can offer specific information about areas of performance so that teachers can follow up with targeted instruction, students can better target their own efforts, and administrators and policymakers can more fully understand what students know and can do, in order to guide curriculum and professional development decisions.” —Smarter Balanced Assessment Consortium 2012

18. Computer Adaptive Technology Moving away from fixed-form testing, Smarter Balanced assessments will capitalize on the precision and efficiency of computer adaptive testing (CAT) for both the mandatory summative assessment and the optional interim assessments (SBAC 2012).

19. Computer Adaptive Technology Read the excerpt below and note key pieces of information on how the Smarter Balanced assessments will differ from the STAR Program assessments. “Smarter Balanced assessments make use of computer adaptive technology, which is more precise and efficient than fixed-form testing. Teachers, principals, and parents can receive results from computerized assessments in weeks, not months. Faster results mean that teachers can use the information from optional interim assessments throughout the school year to differentiate instruction and better meet the unique needs of their students.” –Smarter Balanced 2013

20. Computer Adaptive Technology • Computer Adaptive Testing (CAT): Computer automatically scores responses and adjusts subsequent questions according to the ability demonstrated by the test taker on the prior questions. • Provides a more accurate measurement of student achievement, particularly for high- and low-performing students. • Every student receives a unique set of test questions, reducing the need to test students on particular content simultaneously. • Enables schools to group and assess students in a configuration that matches their specific capacity. For additional information, visit the Smarter Balanced Computer Adaptive Testing Web page at http://www.smarterbalanced.org/smarter-balanced-assessments/computer-adaptive-testing/.

21. Computer Adaptive Technology • Fixed-form testing: All students see the same set or sets of questions that can be varied through the use of different test forms or versions. • Most commonly represented by the traditional paper-and-pencil test but can also be administered on a computer (i.e., computer-based). • A computer can randomize questions or present them in different orders for different test takers.

22. A Balanced Assessment System This graphic illustrates the components of the Smarter Balanced assessment system. Notice the multiple assessment points included.

23. A Balanced Assessment System “The 'Balanced' in Smarter Balanced refers both to our model of an interconnected system of summative, interim, and formative components, as well as a balance between technologically advanced computer adaptive testing (CAT) and extended, thematically related performance tasks." —Smarter Balanced Preliminary Test Blueprints 2012 Read about possible outcomes from the perspectives of students, teachers, parents, and policy makers: What Will Smarter Balanced Assessments Mean for Me? Handout What Will Smarter Balanced Assessments Mean for Me?

24. A Balanced Assessment System 1. How does the purpose of California’s proposed assessment system as described by Smarter Balanced differ from the STAR system? 2. Describe benefits you anticipate for teachers, students, and parents as the Smarter Balanced computer adaptive assessment system is implemented in California.

25. Coherent System Components This table provides additional detail on each of the Smarter Balanced assessment system components. How does this system differ from the STAR Program? Components of the Smarter Balanced Assessment System

26. A Balanced Assessment System Read more about Smarter Balanced and computer adaptive testing in the document titled, Computer Adaptive Testing: Handout Computer Adaptive Testing Join the CDE Smarter Balanced electronic mailing list: subscribe-sbac@mlist.cde.ca.gov

27. Assessment Methods and Item Types Smarter Balanced assessments will be using several methods and item types to gather information about student learning. The descriptionson the next several slides are applicable not only to Smarter Balanced, but also to any statewide assessment system as well as to district-, school-, and classroom-level assessments. Handout Assessment Methods (optional)

28. Assessment Methods and Item Types Selected-Response Selected-response (SR) items prompt students to select one or more correct responses from a set of choices. Carefully constructed and reviewed SR items allow students to demonstrate their use of complex thinking skills, such as developing comparisons or contrasts; identifying cause and effect; identifying patterns or conflicting points of view; or categorizing, summarizing, or interpreting information.

29. Assessment Methods and Item Types Selected-Response Example SR items are available on the Smarter Balanced Web site: Grades 6–8 Math: Expressions and equations item 1 http://sampleitems.smarterbalanced.org/itempreview/ ModernShell.aspx?config=SBAC\Content\EEProgressions1.json High School ELA: Diamonds in the Sky 1 http://sampleitems.smarterbalanced.org/itempreview/ ModernShell.aspx?config=SBAC\Content\Diamonds1.json

30. Assessment Methods and Item Types Constructed-Response Constructed-response (CR) tasks prompt students to generate a text or numerical response in order to collect evidence about their knowledge or understanding of a given assessment target. Smarter Balanced assessments use both short and extended CR items. Short CR items may require test-takers to enter a single word, phrase, sentence, number, or set of numbers, whereas extended CR items require more elaborated answers and explanations of reasoning.

31. Assessment Methods and Item Types Constructed-Response CR tasks allow students to demonstrate complex thinking skills by formulating comparisons or contrasts; proposing causes and effects; identifying patterns or conflicting points of view; categorizing, summarizing, or interpreting information; and developing generalizations, explanations, justifications, or evidence-based conclusions (Darling-Hammond & Pecheone 2010). These complex thinking skills support development of the 4 Cs and college and career readiness.

32. Assessment Methods and Item Types Constructed-Response Example CR items are available on the Smarter Balanced Web site: Grades 3–5 Math: Rectangle 2 http://sampleitems.smarterbalanced.org/itempreview/ ModernShell.aspx?config=SBAC\Content\Garden2.json Grades 6–8 Writing: Cell Phones http://sampleitems.smarterbalanced.org/itempreview/ ModernShell.aspx?config=SBAC\Content\Writing1a.json

33. Assessment Methods and Item Types Technology-Enhanced Items Technology-enhanced (TE) items use specialized computer interactions for collecting response data that go beyond traditional SR or CR items. TE tasks require students to use technology skills to respond to the item prompt. For example, students may be required to draw a figure with a mouse, stylus, or finger; drag and drop text; highlight text; or reorder text.

34. Assessment Methods and Item Types Technology-Enhanced Items Example TE items are available on the Smarter Balanced Web site: Grades 3–5 ELA: Grandma Ruth 2 http://sampleitems.smarterbalanced.org/itempreview/sbac/ ELA.htm High School Math: Water Tank http://sampleitems.smarterbalanced.org/itempreview/ ModernShell.aspx?config=SBAC\Content\WaterTank.json

35. Assessment Methods and Item Types Performance Tasks Performance tasks (PTs) measure a student’s ability to integrate knowledge and skills across multiple standards—a key component of college and career readiness. PTs are used to measure capacities such as depth of understanding, research skills, and complex analysis, which cannot be adequately assessed with SR or CR items. These item types challenge students to apply their knowledge and skills to respond to complex real-world problems.

36. Assessment Methods and Item Types Performance Tasks Smarter Balanced interim and summative components of the assessment system will include PTs at each grade level tested. Additional extended tasks will be available in the Digital Library of Formative Assessments as part of the Smarter Balanced exemplar instructional modules and inventory of currently available resources. Smarter Balanced has released some sample performance tasks focused on assessing critical thinking, problem solving, and communication skills described in MAPP.

37. Assessment Methods and Item Types Performance Tasks Grades 3–5 ELA: Animal Defenses http://www.smarterbalanced.org/wordpress/wp-content/uploads/ 2012/09/performance-tasks/animal-performance.pdf Grades 3–5 Math: Planting Tulips http://www.smarterbalanced.org/wordpress/wp-content/uploads/ 2012/09/performance-tasks/tulips.pdf Grades 6–8 ELA: Garden of Learning http://www.smarterbalanced.org/wordpress/wp-content/uploads/ 2012/09/performance-tasks/garden.pdf

38. Assessment Methods and Item Types Performance Tasks Grades 6-8 Math: Taking a Field Trip http://www.smarterbalanced.org/wordpress/wp-content/uploads/ 2012/09/performance-tasks/fieldtrip.pdf High School ELA: Nuclear Power http://www.smarterbalanced.org/wordpress/wp-content/uploads/ 2012/09/performance-tasks/nuclear.pdf High School Math: Thermometer Crickets http://www.smarterbalanced.org/wordpress/wp-content/uploads/ 2012/09/performance-tasks/crickets.pdf

39. Assessment Methods and Item Types 1.Which of the essential 21st Century Learning Skills are required for the performance tasks that you reviewed? A. Critical thinking and problem solving B. Communication C. Collaboration D. Creativity E. Fluency with multi-media and/or technology 2. What are some of the things you noticed as you reviewed the performance task items? 3. Which assessment item type would you select to gather information on students’ ability to integrate knowledge and skills across standards and/or across content areas? 4.   Which would you choose to gather information on students’ ability to recall information? Describe your rationale.

40. Assessment Methods and Item Types Additional sample items available on the Smarter Balanced Sample Items and Performance Tasks Web page at http://www.smarterbalanced.org/sample-items-and-performance- tasks/. Learn more about California’s role in the development of the Smarter Balanced Digital Library of formative assessment practices and tools: http://www.cde.ca.gov/ta/tg/sa/diglib.asp

41. Smarter Balanced Preliminary Blueprints Preliminary blueprints for the Smarter Balanced summative assessment (released November 2012) provide detailed information on the CA CCSS content to be assessed and the item types to be used for assessment. “For those responsible at the instructional level, the test blueprint provides a guide to the relative importance of competing content demands and suggests how the content is intended to be demonstrated…” “The blueprints provide information on the Smarter Balanced assessment claims, assessment targets, item types, and depth of knowledge.” —Smarter Balanced General Item Specifications April 2012

42. Smarter Balanced Preliminary Blueprints Understand the following definitions: Assessment claims: Broad statements of the assessment system's learning outcomes. Assessment targets: Description of the expectations of what will be assessed by the items and tasks.

43. Smarter Balanced Preliminary Blueprints Assessment Targets Smarter Balanced assessment targets comprise an essential component of a quality assurance approach to item development referred to as evidence-centered design (ECD). Central to this approach is the idea of collecting evidence from a student’s response to an item or task that supports a claim about the extent to which a student has developed the knowledge, skill, and ability that is contained in a content standard (SBAC 2013).

44. Smarter Balanced Preliminary Blueprints Assessment Targets “Clear and rigorous prioritized assessment targets translate the grade-level Common Core standards into content frameworks along a learning continuum.” —Measured Progress/ETS Collaborative 2012 "These summative assessment targets (evidence) at each grade level represent the prioritized content for summative assessment...” —Smarter Balanced General Item Specifications April 2012 Additional information about the ECD approach is available on the SBAC Item Writing and Review Web page at http://www.smarterbalanced.org/smarter-balanced- assessments/item-writing-and-review/.

45. Smarter Balanced Preliminary Blueprints Depth of Knowledge The Smarter Balanced assessment system relies on the depth of knowledge (DOK) performance levels to aid in creating and scoring ELA and mathematics items. More basic levels of knowledge, such as recall, are assessed with SR items, while the deeper levels of knowledge, such as reasoning, are most often assessed with CR items or PTs. The CA CCSS “…require high-level cognitive demand, such as requiring students to demonstrate deeper conceptual understanding through the application of content knowledge and skills to new situations and sustained tasks. For each assessment target in ELA and mathematics, the depth(s) of knowledge that the student needs to bring to the item/task has been identified.” —Smarter Balanced General Item Specifications April 2012

46. Smarter Balanced Preliminary Blueprints Depth of Knowledge Level 1: Recall and Reproduction Requires eliciting information such as a fact, definition, term, or a simple procedure, as well as performing a simple algorithm or applying a formula. Level 2: Basic Skills and Concepts Requires the engagement of some mental processing beyond a recall of information. Level 3: Strategic Thinking and Reasoning Requires reasoning, planning, using evidence, and explanations of thinking. Level 4: Extended Thinking Requires complex reasoning, planning, developing, and thinking most likely over an extended period of time.

47. Smarter Balanced Preliminary Blueprints Explore the Smarter Balanced Assessment Consortium Preliminary Test Blueprints in greater depth: http://www.smarterbalanced.org/wordpress/wp-content/ uploads/2011/12/Smarter-Balanced-Preliminary-Test-Blueprints.pdf

48. Smarter Balanced Practice Tests Smarter Balanced practice tests contain sample questions aligned to the CCSS for ELA and mathematics in grades 3–8 and 11. Practice tests are not computer adaptive, but allow users to experience some of the features of online testing and to get a beginning understanding of how Smarter Balanced will assess students’ mastery of the CCSS (SBAC 2013). For additional information about Smarter Balanced pilot and practice tests, visit the following Web pages: http://www.smarterbalanced.org/pilot-test/http://sbac.portal.airast.org/Practice_Test/default.html

49. English Learners and Students with Disabilities in California’s Assessment System MAPP legislation specifies inclusive practices and checks for fairness when assessing English learners and students with disabilities. “The department shall develop a three-year plan of activities, with the approval of the state board, supporting the continuous improvement of the assessments developed … Activities may include, but not necessarily be limited to, a variety of internal and external studies such as validity studies, alignment studies, and studies evaluating test fairness, testing accommodations, testing policies, and reporting procedures, and consequential validity studies specific to pupil populations such as English learners and pupils with disabilities.” ‒Assembly Bill 484

50. English Language Assessment Students in kindergarten through grade twelve whose home language is not English are required to take an annual English language proficiency assessment based on English Language Development (ELD) standards. California’s current assessment is the California English Language Development Test (CELDT) and is based on the 1999 ELD standards. The English Learner Proficiency Assessment for California (ELPAC) will be first administered in 2015–16 and will be aligned to the 2012 ELD Standards for California Public Schools and the CA CCSS. The system’s goal is to maximize assessment information on language development to support English learners’ attainment of the CA CCSS. The 2012 ELD Standards for California Public Schools are available on the CDE English Language Development Standards Web page at http://www.cde.ca.gov/sp/el/er/eldstandards.asp.