Connecting the Dots: Formative, Interim, and Summative Assessment

1. Connecting the Dots:Formative, Interim, and Summative Assessment G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11th Annual MARCES/MSDE Conference University of Maryland, October, 2011

2. The purpose of education Thomas Jefferson “the ideal of offering all children the opportunity to succeed, regardless of who their parents happen to be” (Hirsch, 2006, p 30) George Washington Carver “Education is the key to unlock the golden door of freedom.” Malcolm Forbes “Education's purpose is to replace an empty mind with an open one.”

3. Assumption: The mission of an educational system To provide each student with an opportunity to learn what life has available, to help them decide what interests them, and to help them learn as much as they can to take them in their desired direction.

4. Stakeholders and Assessment Needs • Students • Am I doing well enough? • Can I reach my goals? • Teachers • What is next for this student? • How am I doing? • Parents • Is my child growing well? • How is my child doing relative to his/her peers? • School Administrators • Is my school doing as well as it could? • Is my school meeting requirements?

5. Stakeholders and Assessment Needs (cont.) • Legislators • Are we creating a strong workforce? • Will our next generation be ready to lead? • The general public • Is my money being spent well? • Can I hire good employees?

6. Assessment tools for education • Processes that allow a teacher to capture learning as it occurs (“formative”; ”diagnostic”) • Procedures that allow the identification of student achievement and growth relative to the trait of primary interest during instruction (“interim”; “benchmark”). • Procedures that allow identification of achievement at the end of a course of study (“summative”; “evaluative”).

7. Using these tools together • Currently, the primary focus of federal regulation is summative/evaluative assessment. • This focus creates an imbalance in the classroom. • We need to find balance, so each assessment tool can help us provide the educational outcomes we seek. • Each approach provides only a portion of the information that stakeholders need • Here we will describe the characteristics of each assessment tool, and how they might work together to provide more or the needed information

8. The Annual Achievement Assessment (U.S. Version) • Assesses student performance towards the end of a school year. • Is primarily designed to make evaluative statements about groups of students. • Measurement accuracy in the aggregate is crucial. • Can be designed to make statements about individual students, but this requires longer tests.

9. Characteristics • Neither detailed content coverage nor full-sample testing is required. • For most purposes, immediacy of results is neither required nor available. • Information shelf life (i.e., for how long can these data support the intended inferences?) is long. • Needed frequency of data collection low

10. Stakeholder Needs • School administrators can use results to chart trends in student performance. • Legislators can use results to identify funding needs. • The general public can use results to gauge the effectiveness of the educational system. • Teachers may be able to use results to help in curriculum planning for future cohorts.

11. Unmet Needs • Students: typically little to no actionable information about their instructional needs • Teachers: little information about the instructional needs of this year’s individual students. • Parents: little information about their student’s academic growth.

12. Interim Assessment • Focus on student achievement and growth relative to a trait of primary interest during instruction. • Assesses student performance at multiple points during a year of instruction. • Designed to support inferences about the academic growth of individual students. • Measurement accuracy for the individual student is crucial. CAT is useful here.

13. Characteristics • All students are assessed. • Immediacy of results is important. • Can be interpreted normatively or relative to long-term goal (i.e., college readiness). • Information shelf life is short. • Frequency of data collection is moderate.

14. Stakeholder Needs • Students can gauge their growth relative to normative or aspirational goals. • Parents can assess their child’s growth. • Teachers can use results to make instructional decisions for the current cohort. • School administrators can aggregate student results to assess trends in growth and to evaluate teacher effectiveness. • Legislators can use results to evaluate policy. • The general public can use results to gauge the effectiveness of the educational system.

15. “Formative” Assessment • Processes that allow a teacher to capture learning as it occurs, and to make appropriate instructional adjustments.

16. Characteristics • All students are assessed. • Immediacy of results is important. • Interpreted in terms of instructional decisions. • Information shelf life is short. • Frequency of data collection is high.

17. Which of These is Formative? • District science supervisor uses test results to plan professional development workshops for teachers. • Teachers doing item-by-item analysis of 5th grade math tests to review their 5th grade curriculum. • A school tests students every 10 weeks to predict which students are “on course” to pass the state test in March. • Three-fourths of the way through a unit test. • Students who fail a test on Friday have to come back on Saturday. • Exit pass question: “What is the difference between mass and weight?” • “Sketch the graph of y equals one over one plus x squared on your mini-white boards.”

18. Formative Assessment:An Inclusive Definition An assessment functions formatively to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited.

19. Mapping Out the Terrain Timescale Academic promotion High-stakes accountability End-of-course exams Annual Benchmark Growth Interim Common formative assessments End-of-unit tests Weekly Before the end-of-unit tests Daily Exit pass Hourly Hinge-point questions Instructional Guidance (“formative”) Describing Individuals (“summative”) Institutional Accountability (“evaluative”) Function

20. Hinge-point questions Sheena leaves a wooden block, a glass flask, a woolly hat, and a metal stapler on a table overnight. What can she say about their temperatures the next morning? • The stapler will be colder than the other objects • The woolly hat will be warmer than the other objects • The temperatures of all four objects will be different • The temperatures of all four objects will be the same

21. Connecting (some of) the dots Development of science skills in eighth grade • Use of laboratory equipment • Metric unit conversion • Density calculations • Density applications • Density as a characteristic property • Phases of matter • Gas laws • Communication (graphing) • Communication (lab reports) • Inquiry skills

22. Assessment matrix

24. Stakeholder Needs • Students can gauge their growth relative to normative or aspirational goals. • Parents can assess their child’s growth. • Teachers can use results to make instructional decisions for the current cohort. • School administrators can aggregate student results to assess trends in growth and to evaluate teacher effectiveness.

25. Unmet Needs • Teachers: different teachers may be acting effectively in their own classrooms, but their judgments may differ from those of their colleagues • School administrators need to know that achievement scores are comparable from class to class. • Parents need assurance that the grades reported to them are meaningful.

26. Connecting (more of) the dots • Common assessments allow teachers to compare their judgments about the meaning of standards, and thus align their judgments. • Although these instruments are often called “common formative assessments” their main use may be in aligning teachers’ judgments for summative purposes.

28. A strong assessment system • will provide students with immediate feedback concerning their progress • will provide teachers with information concerning their student’s needs • will provide teachers with information useful in long-range instructional planning • will provide school administrators with information about the school’s progress • will provide the public with information about student achievement and growth

29. A strong assessment system • needs to be designed to have an impact in the classroom • needs to communicate needed information clearly to teachers and students • needs a strong measurement scale to measure growth • needs normative, criterion, and content references to make meaning of performance • needs a strong measurement design to measure growth well

30. A combined system • Before school starts, an interim assessment allows accurate class placement, suggests a place to start for each student, and sets individual growth targets. • During the first term formative assessment processes allow the teacher to adjust the original placement and help each student start to move forward. • At the end of the first term, an interim assessment gives each teacher and student a first look at achievement during the year and progress toward growth targets.

31. A combined system continued • During the second term formative assessment allows each teacher to adjust content as the students progress, and allows regrouping of students using teams of teachers. • Toward the end of school, the summative assessment identifies the overall achievement of the students in the class to help determine what there is to celebrate, and what might be done better in subsequent years. • At the end of school, an interim assessment gives each teacher and student a look at achievement during the year and attainment of growth targets.

32. Thank you for your attention. Questions?gage.kingsbury@nwea.orgdylanwiliam@mac.comsteve.wise@nwea.org