State Assessment in 2015: Smarter Balanced Assessment Consortium

1. State Assessment in 2015:Smarter Balanced Assessment Consortium Wyoming State Board of Education and Select Committee on Educational Accountability May 7 – 10, 2013

2. EA116 “Notwithstanding [existing statute], the state board and the department of education shall investigate options available to the state for future assessment system development. The state board, through the department, shall periodically report to the select committee on statewide education accountability regarding the status of assessment development, investigation of options available to the state and the impact of existing law governing statewide assessments on future assessment development. The select committee shall report to the 2014 legislature on any necessary legislation supporting future assessment development.”

3. Next Generation Assessments The U.S. Department of Education funded two consortia of states to develop new assessments aligned to college- and career-ready standards • Rigorous assessment of progress toward “college and career readiness” • Common cut scores across all Consortium states • Provide both achievement and growth information • Valid, reliable, and fair for all students, except those with “significant cognitive disabilities” • Administer online • Use multiple measures • Operational in 2014-15 school year

4. Purpose of the Smarter Balanced Assessment Consortium (SBAC) • To develop a comprehensive and innovative assessment system for grades 3-8 and high school in English language arts and mathematics aligned to the Common Core State Standards, so that... • ...students leave high school prepared for postsecondary success in college or a career through increased student learning and improved teaching • The assessments shall be operational across Consortium states in the 2014-15 school year

5. SBAC: Consortium of States • 26 members representing over 39% of K-12 students • 21 Governing States, 4 Advisory States, 1 Affiliate Member • Washington state is fiscal agent • WestEd provides project management services

7. State-Led Governance States Join SBAC as Governing or Advisory States • Governors • Education Chief • State Legislatures • State Boards of Education State Reps Serve on Executive Committee • 2 Elected Co-Chairs • 4 Reps elected by Governing States • Lead Procurement State (WA) • Higher Ed Reps Smarter Balanced Staff WestEd, Project Mgmt Partner Work Groups andAdvisory Committees

8. Governing / Advisory • States join the Consortium at no cost until assessments are delivered in 2015 • Governing states influence direction of Consortium; they vote on matters of importance (e.g. blueprints), take leadership roles in SBAC, and are invited to participate in various activities (teacher PD, state-level item-writing, scientific pilot) • Change from Advisory to Governing gives Wyoming a voice in the Consortium work

9. SBAC Work Groups • Formative Assessment Practices and Professional Learning/Transition to Common Core State Standards • Item Development/Performance Tasks • Technology Approach/Reporting • Test Administration/Student Access • Validation and Psychometrics/Test Design

10. Balanced Assessment System Summative Assessments Benchmarked to college and career readiness Teachers and schools have information and tools they need to improve teaching and learning Common Core State Standards specify K-12 expectations for college and career readiness All students leave high school college and career ready Teacher resources for Formative Assessment Practices to improve instruction Interim Assessments Flexible, open, used for actionable feedback

11. Balanced Assessment System ELA/Literacy and Mathematics, Grades 3 – 8 & High School School Year Last 12 Weeks of the Year DIGITAL LIBRARY of formative tools, processes and exemplars; released items and tasks; curriculum unit models; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools Optional Interim Assessment Computer Adaptive & Performance Tasks Optional Interim Assessment Computer Adaptive & Performance Tasks Summative Assessment for Accountability Computer Adaptive & Performance Tasks Scope, sequence, number, and timing of interim assessments is locally determined.

12. Assessment Components Summative Assessment (Computer Adaptive) • Assesses the full range of Common Core in English language arts and mathematics for students in grades 3–8 and 11 (interim assessments can be used in grades 9 and 10) • Measures current student achievement and growth across time, showing progress toward college and career readiness • Can be given once or twice a year (mandatory testing window within the last 12 weeks of the instructional year) • Includes a variety of question types: selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks

13. Assessment Components Interim Assessment (Computer Adaptive) • Optional comprehensive and content-cluster assessment to help identify specific needs of each student • Can be administered throughout the year • Provides clear examples of expected performance on Common Core standards • Includes a variety of question types: selected response, short constructed response, extended constructed response, technology enhanced, and performance tasks • Aligned to and reported on the same scale as the summative assessments • Fully accessible for instruction and professional development – Could replace MAP and save districts approximately $10/student

14. Assessment Components Formative Assessment Practices • Research-based, on-demand tools and resources for teachers • Aligned to Common Core, focused on increasing student learning and enabling differentiation of instruction • Professional development materials include model units of instruction and publicly released assessment items, formative strategies

15. Understand the CCSS Formative Practices, Professional Learning and Implementation • Teams of teachers from each state will: • Participate in identifying formative assessment practices and curriculum resources to put in Digital Library • Participate on a committee to complete voluntary alignment review of publishers’ materials to the content specifications and develop a “Consumers Report” to upload to the Digital Library • National content experts to develop 54 (3 ELA and 3 math per grade) formative assessment practices exemplar modules that provide model products for Smarter Balanced teachers (housed in Digital Library) • Existing CCSS curriculum projects are adapted to align with the Smarter Balanced content specifications (and uploaded to the Digital Library)

16. Improve Teaching and Learning Formative Practices, Professional Learning and Implementation • Provide comprehensive support for formative assessment, including instructional modules aligned with CCSS • Training modules help teachers focus their instruction on the CCSS and develop teaching practices that support more in- depth learning • Enhance assessment literacy by training teachers to use formative assessment tools and interim assessment to determine next steps in instruction • Provide supports for students to manage their own learning

17. Understand the Test Formative Practices, Professional Learning and Implementation • Produce high-quality test manuals that include administration guidelines and supports for teachers and students • Support administration of test consistent with its purpose and intended use of data • Offer trainings on how to administer the test, provide accommodations, use reporting system and other applications • Enhance assessment literacy by providing well articulated training on interpreting assessment results • Support connections with pre-service teachers

18. CAT Advantages • Increased Precision: provides accurate measurements of student growth over time • Tailored for Each Student: item difficulty is based on student responses • Increased Security: larger item banks mean that not all students receive the same questions • Shorter Test Length: fewer questions compared to fixed form tests • Faster Results: turnaround time is significantly reduced

19. Assessment Blueprints • Literacy • Reading • Writing • Speaking/Listening • Research Mathematics • Concepts and Procedures • Problem Solving • Modeling and Data Analysis • Communicating Reasoning

20. SBAC Sample Items • Grade 4 Literacy: Reading http://sampleitems.smarterbalanced.org/itempreview/sbac/ELA.htm • Grade 4 Math: Problem-Solvinghttp://sampleitems.smarterbalanced.org/itempreview/sbac/index.htm

21. Higher Education Collaboration • Involved 175 public and 13 private systems/institutions of higher education in application • Two higher education representatives on the Executive Committee • Higher education lead in each state and higher education faculty participating in work groups • Goal: The high school assessment qualifies students for entry-level, credit-bearing coursework in college or university

22. College Content Readiness (draft) • English Language Arts/LiteracyStudents who perform at the College Content-Ready level in ELA demonstrate reading, writing, listening, and research skills necessary for introductory courses in a variety of disciplines and have knowledge and skills for entry-level, transferable, credit-bearing English and composition courses • Mathematics Students who perform at the College Content-Ready level in math demonstrate foundational mathematical knowledge and quantitative reasoning skills necessary for introductory courses in a variety of disciplines and have knowledge and skills for entry-level, transferable, credit-bearing mathematics and statistics courses

23. Framework for Readiness

24. Framework for Readiness

25. K-12 Teacher Involvement • Support for implementation of the Common Core State Standards (2011-12) • Write and review items/tasks for the pilot test (2012-13) and field test (2013-14) • Development of teacher leader teams in each state (2012-14) • Evaluate formative assessment practices and curriculum tools for inclusion in digital library (2013-14) • Score portions of the interim and summative assessments (2014-15 & beyond)

26. Timeline Formative Processes, Tools, and Practices Development Begins Summative Master Work Plan Developed and Work Groups Launched Writing and Review of Pilot Items/Tasks (including Cognitive Labs and Small-Scale Trials) Writing and Review of Field Test Items/Tasks (throughout the school year) Field Testing of Summative and Interim Items/Tasks Conducted Final Achievement Standards (Summative) Verified and Adopted Procurement Plan Developed Content and Item Specifications Development Pilot Testing of Summative and Interim Items/Tasks Conducted Preliminary Achievement Standards (Summative) Proposed and Other Policy Definitions Adopted Operational Summative Assessment Administered

27. Timeline -- 2009-10 • December 2009: States from Hawaii to Kansas collaborate to create dynamic interim and formative assessments that could be scaled to a common summative assessment • March 2010: At the Next Generation K-12 Assessment Systems meeting, the Consortium joins forces with a collaboration of New England states and West Virginia working to create a balanced assessment system • April 2010: ED announces availability of funds for assessment development • June 2010: SBAC submits grant to ED

28. Timeline – 2010-11 • October 2010: ED awards grant to SBAC • October 2010: Wyoming signs MOU to become an Advisory state in SBAC • July 2011: Governing States approve master work plan for summative assessment

29. Timeline – 2011-12 • Fall 2011: SBAC releases content specifications in mathematics and English language arts/literacy • January 2012: Information technology systems architecture available, providing a roadmap for how the SBAC technology vision will be realized • March 2012: Technology readiness tool available to assist with transition to online assessments • April 2012: SBAC awards contract for item writing for the Pilot Test and small-scale trials • May 2012: SBAC awards item authoring and item pool application contract

30. Timeline – 2011-12 • Spring 2012: Development of item and task specifications; accessibility and accommodations guidelines; and bias and sensitivity guidelines for assessment system • June 2012: Governing States approve the Formative Assessment Master Work Plan, which outlines the development of a digital library of resources • August 2012: Advisory committees on English language learners and students with disabilities convened • Summer/Fall 2012: Teachers from Governing States help write pilot items and tasks, review for alignment with CCSS, and check for bias/sensitivity; cognitive labs and small-scale trials are conducted • September 2012: SBAC awards test delivery system contract

31. Timeline – 2012-13 • September 2012: SBAC holds twice-yearly Collaboration Conference, bringing together K-12 state leads, higher education leads, work groups, and contractors • October 2012: SBAC releases online, interactive sample of items and performance tasks • November 2012: Governing States approve preliminary blueprint for summative assessment • November 2012: SBAC awards contract for the development of online reporting system • November 2012: SBAC releases draft initial achievement level descriptors and college content-readiness policy

32. Timeline – 2012-13 • December 2012: SBAC releases technology strategy framework and system requirements specifications. The framework provides minimum hardware specifications and basic bandwidth calculations • Winter 2013: Teachers and teacher educators from Governing States recruited to develop digital library of formative assessment tools and practices • Early 2013: Begin development of Exemplar Instructional Modules across grades in ELA/literacy in collaboration with State Networks of Educators in Governing States (through May 2014)

33. Timeline – 2012-13 • February – May 2013: Pilot Test of assessment items and performance tasks; research sample includes 10 percent of public school students in Governing States • Winter/Spring 2013: Planning for long-term sustainability of the SBAC • Spring 2013: Teacher teams from Governing States write Field Test items and tasks, review for alignment with CCSS, and check for bias (through fall 2013) • Spring 2013: Pilot Test scoring/sensitivity • Spring 2013: Development of Field Test items and tasks; reporting system developed

34. Timeline – 2013-14 • Summer/Fall 2013: Field Test items and tasks are reviewed for content and bias/sensitivity • Spring 2014: SBAC conducts Field test of summative and interim assessment items and performance tasks • Spring 2014: Field test is scored • Summer 2014: Preliminary standard- setting is held

35. Timeline -- 2014-15 • Summer/Fall 2014: Training materials available for all users on interpreting interim and summative assessment reports, searching resources, and using collaboration tools in the Digital Library • Fall 2014: SBAC interim assessments and Digital Library ready for use • Spring 2015: States administer summative assessment during last 12 weeks of the school year • Summer 2015: Final achievement standards for the summative assessment verified and adopted

36. Technology Framework

37. Technology Readiness • SBAC has deployed a Technology Readiness Tool to help states plan for SBAC • Assesses current capacity in each school across the states • Compares capacity to required technology • Devices • Device to tester ratio • Network infrastructure • Staff and personnel • WY will release the Tool this Spring and will provide readiness data ASAP

38. Sustainability • Current organizational host/fiscal agent is the state of Washington • Future hosting organization will be the UCLA National Center for Research on Evaluation, Standards, and Student Testing (CRESST) • Procurement will be via RFP, preferably with multiple states, to achieve greater economies of scale (approx 1M students) • Initial discussion with some western states has been held to determine interest in collaborating on procurement

39. SBAC Costs • Package (summative, interim, & formative) = approx $27.30 per pupil $9.55 SBAC fee + $17.75 vendor fee • Summative only = approx $22.50$6.20 SBAC fee + $16.30 vendor fee • Available for purchase: additional assessments in high school grades

40. SBAC Costs • Summative only cost is approximately 1/3 of current cost of PAWS ($23 vs $76) • Entire package of integrated assessments and resources is less than ½ the current cost of PAWS ($28 vs $76) • Entire package could also relieve districts of cost of on-line, adaptive assessments (approx $10 per pupil) • Grade 11 cost is approximately ½ that of ACT Plus Writing ($23 vs $47.50)

41. Benefits of SBAC • Balanced: summative, interim, and formative tools/resources – digital library • Aligned: deeper coverage of standards with multiple item types, including CAT multiple choice, constructed response, and extended response/ performance tasks • Cost effective: at under $30/student, it’s less than ½ the current cost

42. Benefits of SBAC • Returns results to schools two weeks after testing window • Permits state to state comparisons • Facilitates cross-state collaboration (partnership w/other states) • Eases peer review of assessment system • Improves test security with online administration

43. Benefits of SBAC • Measures growth over time with interim and summative integrated scale • Provides resources for school staff to improve teaching and learning and foster school improvement • Ensures local control of instructional programs and curriculum while establishing common performance outcomes • Supports WAEA by providing measures of achievement, growth, and college and career readiness

44. 2015 Statutory Impact • Separate reading and writing assessments • Writing assessment in grades 3, 5 and 7 only • Multiple choice items only in reading and math • High school assessment system

45. Other Assessments • Ongoing need for annual science assessments in grades 4, 8, and 11 • Ongoing need for annual alternate assessments for students with significant cognitive disabilities (1%) • Ongoing need for annual assessments of English proficiency for English learners