Designing an Assessment System for the Race to the Top

1. Designing an Assessment System for the Race to the Top Edward Haertel School of Education Stanford University RTTT Public Meeting, General Assessment Denver, Colorado December 1, 2009

2. What’s Different This Time? In the absence of well-defined and demanding standards, education in the United States has gravitated toward de facto national minimum expectations, with curricula focusing on low-level reading and arithmetic skills and on small amounts of factual material in other content areas. Most current assessment methods reinforce the emphasis on these low-level skills and processing bits of information rather than on problem solving and critical thinking. The adoption of world-class standards would force the Nation to confront today's educational performance expectations that are simply too low. NCEST Report, 1992

3. New Ideas • Common Standards • fewer, clearer, higher? • Aligned assessments of individual growth • Explicit attention to range of assessment types / formats, content sampling, item release plans, etc. • 21st century technology • computer-based testing, longitudinal growth models, Evidence-Centered Design, …

4. Old Ideas • If assessments are aligned to standards, then growth in test scores will track intended schooling outcomes • Essentially all valued schooling outcomes can be captured by test performances of • students working alone • without “smart workplace” resources • answering clear questions with clear answers

5. Old Ideas • One assessment system will “do it all” • track progress • of students / schools / … • inform decisions • about graduation / college readiness / career readiness / college course placement • support improvement • in teaching / learning / instructional programs • determine effectiveness • of schools / principals / teachers

6. Theories of Action are, if anything, less clear than in 1992 A Theory of Action is a chain of logical propositions that “connects the dots,” explaining step by step, in commonsense terms, how the assessment system, accountability system, and related reforms will lead to improved learning outcomes

7. Rethinking Old Ideas To Make New Ideas Work • Aligning tests and curricula to common standards is not sufficient to create a coherent system ? Curriculum & Instruction Assessments Content Standards

8. Rethinking Old Ideas To Make New Ideas Work • Aligning tests and curricula to common standards is not sufficient to create a coherent system Curriculum & Instruction Assessments ? Content Standards

9. 11th Grade History /Social Studies US11.11.2. Discuss the significant domestic policy speeches of Truman, Eisenhower, Kennedy, Johnson, Nixon, Carter, Reagan, Bush, and Clinton (e.g., education, civil rights, economic policy, environmental policy).

10. Item Testing US11.11.2

11. High School Biology BI6.f Students know at each link in a food web some energy is stored in newly made structures but much energy is dissipated into the environment as heat. This dissipation may be represented in an energy pyramid.

12. Item Testing BI6.f

13. Rethinking Alignment • It is not enough for each item to be somehow connected to one or more standards • Assessments, standards, and curriculum and instruction must all be considered together • Curriculum-embedded formative assessments • Portfolios incorporating well-specified, externally scorable components

14. Rethinking Old Ideas To Make New Ideas Work • Testing need not imply that students work alone, without references or resources, on small tasks with one right answer • Well-structured problems(Frederiksen, 1984) • Content-lean, process-constrained tasks(Baxter & Glaser, 1998) • Sequestered Problem Solving(Bransford & Schwartz, 1999)

15. Rethinking What It Means to Take A Test • A comprehensive assessment system must also include • ill-structured problems with multiple solutions that are better or worse, not right or wrong • content-rich, process-open tasks requiring flexible, adaptive application of coherent knowledge structures to solve novel problems • tests of preparation for future learning – measuring skill in approaching new problems, “knowing with” as well as “knowing how” or “knowing that”

16. Rethinking Old Ideas To Make New Ideas Work • A single form of testing cannot satisfy the full set of RTTT design requirements • Sole reliance on external, on-demand tests will fall short, even if constructed-response items are included • Sole reliance on external, on-demand tests will fall short, even if interim / benchmark assessments are included

17. Rethinking Multiple Indicators • A comprehensive assessment system must use multiple forms of evidence • curriculum-embedded assessments • extended projects • a range of student performances including • finding, organizing, and evaluating information • analyzing and synthesizing data • planning and carrying out investigations • solving unfamiliar, open-ended problems • written reports and oral presentations

18. Rethinking Multiple Indicators • Different forms of assessment, organized in different ways, may be required for • improving instructional decision making in the classroom • tracking student growth and predicting college or career readiness • accountability

19. Rethinking Multiple Indicators • Standards, assessments, curriculum, and instruction must be aligned and coherent • horizontal links across content areas • vertical links across grade levels • Assessment at the classroom or school level may employ matrix sampling • different students answer different questions to provide better group-level information • some assessment system components may not yield individual scores

20. Summary • We must RETHINK Alignment • not just test-to-standards, but instead a coherent, simultaneous consideration of standards, assessments, curriculum and instruction, and I might add, teacher preparation

21. Summary • We must RETHINK what it means to take a test • not just sequestered problem solving, with tidy questions and routine problems, but also complex tasks • with multiple solutions judged by multiple criteria • sometimes worked out over an extended time • sometimes involving collaboration • sometimes requiring students to figure out what information they need and where to find it

22. Summary • We must RETHINK multiple indicators • not just a broader mix of item formats • not just interim plus end-of-year tests • some portion of classroom assessment must be sufficiently documented and structured so as to be incorporated into comprehensive assessment systems to meet multiple needs

23. Thank you