CURRENT TRENDS, PRACTICES, AND ISSUES IN CLASSROOM ASSESSMENT

1. Lorrie A. Shepard University of Colorado at Boulder Classroom Assessment SIG American Educational Research Association San Francisco April 7, 2006 CURRENT TRENDS, PRACTICES, AND ISSUES IN CLASSROOM ASSESSMENT

2. Big Ideas The power of theory Assessment’s role in motivation as well as learning Changing the culture of grading Formative program evaluation vs. formative assessment Hijacking Teacher education & teacher professional development

3. [Classroom Assessment SIG – prior knowledge] • Knowing What Students Know Pellegrino, Chudowsky, & Glaser NRC, 2001 Cognitive science findings on key aspects of learning processes can be translated into targeted features of formative assessment: • Accessing prior knowledge • Strategic use of feedback • Teaching and assessing for • transfer • Meta-cognitive benefits of self- • assessment

4. The Power of Theory • Learning theory provides coherence and big-picture understandings when trying to change teaching practices. • LT identifies effective practices (when fully implemented). • LT helps us reason about what to do when we can’t rely on experience. • Sociocultural learning theory helps us see that formative assessment and instructional scaffolding are essentially the same thing.

5. Research on Teacher Learning & Teacher Professional Development • As with all novices, teachers learning about formative assessment are likely to attend to surface features. • Without conceptual support, teachers may adapt FA to fit traditional practices, e.g., making self assessment about grading, and miss its core features. • e.g., 1/3 of feedback studies show negative effects (Kluger & DeNisi meta-analysis) • Fundamental change requires conceptual and strategic support (Putnam & Borko, 2000). • In Assessment for Learning, teachers are treated as learners, trying to make sense of a new theory (Black, Harrison, Lee, Marshall, & Wiliam, 2003).

6. Motivation and Participation Are Inextricably Entwined with Learning • Historically, research on motivation was separate from research on learning until studies of metacognition showed import of motivation (Resnick & Klopfer, 1989). • “Performance-oriented” students work for grades, to please the teacher, and to appear competent. Normative grading practices and extrinsic rewards induce these behaviors. (Stipek, 1996). • Activity theory and Vygotsky’s notion of socially supported learning provides a wholly different view of what might “motivate” students to devote their hearts and minds to learning (Shepard, 2001).

7. “We have to change the social meaning of evaluation.” Developing an identity of mastery occurs as learners participate in a community of practice (Lave & Wenger, 1991). Self-assessment also increases students’ responsibility for their own learning and makes the student-teacher relationship more collaborative (Gipps, 1999). Shepard (2000). “The Role of Assessment in a Learning Culture.”

8. Student-Involved Classroom Assessment • Stiggins (2001). • Taylor & Nolen, Classroom Assessment: Supporting Teaching and Learning in Real Classrooms (2004). • Assessment textbooks with chapters on student motivation • Rewards and punishments: the rich getting richer and the poor getting poorer (Stiggins) • Task involvement and the willingness to persist (Taylor & Nolen)

9. Summative Assessment and Grading Summative assessment and grading pose a serious threat to the learning purposes avowed for formative assessment. If tests diverge from valued learning goals, students focus only on the graded portion of the curriculum. The use of grades as rewards and punishments undermines intrinsic motivation to learn. Using effort grades to manage student behavior is not the same thing as creating a learning environment that is motivating to students.

10. Grading Practices that Emphasize Learning Goals • Grades should represent achievement, not normative comparisons. • Achievement-based grades will be more transparently aligned with feedback on the same standards used for formative assessment and will communicate better to external audiences. • A commitment to achievement-based grading would mean doing away with compliance grading practices such as extra credit points, points for turning in homework that is never graded, and so forth. • Other ways to soften student worries about grades can be allowed so long as they provide opportunities for students to demonstrate mastery – e.g., replacement assignments and replacement tests or throwing out test scores when learning is verified by later assessments.

11. Grading Practices that Support Formative Assessment • To make formative assessment truly for learning, teachers may need to postpone grading, or use only peer and student self-assessment in early grades, or give “as if” grades to keep students focused on substantive feedback. • For feedback to help with new learning, there have to be opportunities to use that knowledge again.

12. Formative Program Evaluation vs. • Formative (Student) Assessment • By convention, “assessment refers to judgments of student performance, while evaluation refers to judgements of programme or organisational effectiveness.” • -OECD (2005) Formative assessment: Improving learning in secondary classrooms • Formative program evaluation data can be used to identify systematic strengths and weaknesses in instructional programs. Adjustments can then be made to curriculum or instructional strategies.

13. To Be Effective, Formative Program Evaluation Tools Must Meet 3 Criteria • More than simplistic alignment, they must “embody” learning goals. • They should be timed to be instructionally-linked. • They should meet a cost/benefit test, instructional insights > instructional time lost. (At a minimum must yield new insights beyond high-stakes test.) • Instructional insights should lead to coherent, theoretically-sound improvements in teaching.

14. To Be Effective, Formative Student Assessments Must • “Embody” learning goals. • Be curriculum-embedded (both in timing and substance). Tasks are instructional tasks so no instructional time is lost; occurs “midstream”* to inform instruction not as a unit summative test. • Enable the supportive learning processes invoked in the formative assessment literature. • Instructional insights should lead to coherent, theoretically-sound improvements in teaching. • *see Stuart Kahl, Ed Week, 9.21.05

15. Substantively, Good Formative Assessment Tools • Can never be all multiple-choice. • Provide qualitative insights about understandings and misconceptions not just a numeric score. • Have immediate implications for what to do besides re-teaching every missed item. (the 1000 mini-lessons problem).

16. As Processes, Good Formative Assessment Practices • Elicit prior knowledge, both prerequisite and background knowledge, and develop classroom routines to help students become aware of knowledge use. • Provide feedback that enables students to see how to improve performance over time. • Develop classroom expectations that call for application and generalization, rather than repetitive practice. • Use self-assessment as a means to support internalization of criteria and personal ownership of the learning process.

17. “Déjà vu All Over Again.”

18. The research-based concept of formative assessment, closely grounded in classroom instructional processes has been taken over (hijacked) by commercial test publishers and used instead to refer to formal testing systems called benchmark or interim assessment systems.

19. What’s the Harm of Imperfect Formative Tools? • In contrast to these criteria for good formative assessment, typical commercial products trade content quality for the efficiency of computer-delivered multiple-choice items. Some clearly provide only an impoverished rendition of the content domain thus exacerbating the effects of teaching the test. • Expert teachers will know better, but novice teachers are likely to take officially adopted formative instruments as sufficient representations of what students need to know. (e.g., teachers teaching nonsense syllables for DIBELS)

20. Sample Question Imitating A “Formative Assessment” Website Cats are fun animals to have as pets. Read the following passage and fill in the missing words. Cats make great pets. They are soft and cuddly. They play with toys. Sometimes they get tired and need to rest. They {1} . There are many kinds of cats. Cats have four legs. Cats like to play outside. Dogs chase them. Cats have to climb trees to get away. They get {2} . {1} O bite {2} Oscared Osleep Ohurry Oski Ocold Oclose Ohungry These reading passages resemble a cloze technique but, in fact, are based on unnatural paragraphs. Instead of inference, they invite learning the strategy that the answer is nearly always in the sentence before.

21. Negative Effects on Learning Exacerbated by More Teaching to Benchmark Tests • Test-score inflation (non-generalization of test-score gains, increased proficiency on exact formats without conceptual understanding) • Curriculum distortion (both what is taught and how it is taught) • Greatest impacts for poor and minority children (The poorer the school, the more time devoted to instruction that looks like the test.)

22. Negative Effects of Teaching the Test on Motivation • Stipek (1996). • Emphasis on evaluation reduces students’ intrinsic motivation and interest in material for its own sake. Students focused on extrinsic rewards pick easy tasks and worry about looking good instead of trying to understand a task and wanting to become more competent. • Lave & Wenger (1991). • Testing in schools leads to the commoditization of learning because it shifts effort and purpose from learning “to know” to learning so as “to display one’s knowledge.”

23. Potential Solutions • Commercial test publishers could change the labels of their data systems from formative assessment to formative evaluation (previous examples: CTB divested of Scoring High & ETS argued against use of teacher test for practicing teachers). • Commercial publishers could develop rich curricular units tied to standards with embedded assessments that modeled best practice in both formative and classroom summative assessment.

24. Potential Solutions • States and districts with curricular authority could invest in development of curriculum embedded assessments and accompanying learning progressions. • Districts could invest in professional development for teachers emphasizing the use of formative assessment specifically tied to the district’s curriculum (examining student work, identifying typical misconceptions, developing learning progressions, sharing related instructional practices, etc.).

25. Putnam & Borko (2000). Research on Teacher Learning and Teacher Professional Development We know that teachers are most likely to make and sustain fundamental changes in instructional practice if given • time, • conceptual and strategic support, • and opportunities to tryout new • practices in the context of their own • teaching.

26. Lessons from Research on Teaching Given the layers of assessment-related demands already faced by teachers, efforts to improve classroom assessment strategies should be woven into standards-based professional development and curriculum development. Teachers need better access to materials that model teaching for understanding – with extended instructional activities, formative assessment tasks, scoring rubrics, and summative assessments built in. And they need extended support while attempting to use these materials and teach in new ways.

27. Black, Harrison, Lee, Marshall, & William, (2003). Assessment for Learning: Putting It into Practice. • Offers a model of effective classroom assessment and professional development by which teachers can simultaneously transform their assessment and instructional practices to focus more on learning. • A key theme is the focus on student thinking and “mindfulness,” which formative assessment helps to develop. • Teachers are treated as learners, trying to make sense of a new theory.

28. Preparing Teachers for a Changing World • Darling-Hammond & Bransford (2005) • What kinds of teacher learning opportunities are associated with teaching practices that, in turn, influence student learning? • Students are better able to acquire complex skills when their teachers – • help them understand underlying concepts and patterns that tie together ideas they are studying, • provide models of how to approach the task and reason through problems, • provide scaffolds or structured steps that support the learning process, • coach students as they apply their knowledge to real-world tasks, • help students learn to evaluate and regulate their own learning.

29. Pre-Service Teachers’ Grounded Experience with Assessment Analyzing student work to inform instructional choices Documenting how student learning develops over time Designing an assessment-infused instructional unit Developing performance assessments and paper-and-pencil tests Designing rubrics and feedback with attention to issues of intrinsic and extrinsic motivation

30. In order for formative assessment not to be just another hijacked reform – implementation efforts must attend to both the substance of what is assessed and the social processes. Reformers must have an explicit plan for teacher learning and an evaluation cycle to check for fidelity of implementation.

31. Happily Ever After