Do You Know What Your Students Know? Examining Student Work to Inform Instruction

1. Do You Know What Your Students Know?Examining Student Work to Inform Instruction Lani Seikaly, partner Hillcrest and Main, Inc. and project director of MDK12 http://mdk12.org lani@mdk12.org

2. What do principals need to put in place to improve their student achievement?How do teachers and schools identify what students know and still need to learn in relation to state standards? Lani Seikaly, Hillcrest & Main, Inc.

3. State assessments only inform schools where their students are performing at the time of testing. Teachers must know where their students are at any point in the school year.

4. AYP provides us a clear target. However, to meet AYP, we need to monitor student level data at the classroom level on an ongoing basis. Lani Seikaly, Hillcrest & Main, Inc.

5. How do state standards and NCLB change expectations for what happens in our schools? Lani Seikaly, Hillcrest & Main, Inc.

6. Standards-based education changes everything. • Defines what we expect all students to know and be able to do. • Expects educators to take all students to proficiency on those standards. Lani Seikaly, Hillcrest & Main, Inc.

7. Before Standards Teachers taught what they thought was important. After Standards Teachers are expected to teach the content standards. Lani Seikaly, Hillcrest & Main, Inc.

8. Before Standards Teachers had different expectations for different groups of students. After Standards Teachers are expected to take all students to proficiency. Lani Seikaly, Hillcrest & Main, Inc.

9. Before Standards The focus was on how well teachers taught. After Standards The focus is on how well students learn. Lani Seikaly, Hillcrest & Main, Inc.

10. Before Standards Only selected students had access to higher level instruction. After Standards All students are expected to have equity of opportunity for higher level instruction. Lani Seikaly, Hillcrest & Main, Inc.

11. What activities do teachers need to engage in today that they might not have needed to do 15 years ago? Lani Seikaly, Hillcrest & Main, Inc.

12. Align instruction and assessment with state content standard indicators Know where their students are performing on the indicators Work toward a common understanding of content standards and proficient work Diagnose what students know and still need to learn in relation to those indicators and the criteria for proficient work Engage in grade level team examinations of student work Lani Seikaly, Hillcrest & Main, Inc.

13. How do we get answers to these questions… • Where are each of our students in relation to the content standards they must attain? • What do they know and are able to do? • What do they still need to learn? Lani Seikaly, Hillcrest & Main, Inc.

14. We have a good deal of research to draw on in identifying what effective schools and leaders do to improve student achieve-ment. Lani Seikaly, Hillcrest & Main, Inc.

15. Black and Wiliam in their 1998 Phi Delta Kappan article, “In the Black Box: Raising Standards through Classroom Assessment,” assert, “There is a body of firm evidence that formative assessment is an essential component of classroom work and that its development can raise standards of achievement. We know of no other way of raising standards for which such a strong prima facie case can be made.” Lani Seikaly, Hillcrest & Main, Inc.

16. If formative assessments are so critical, then how do we create good assessments? Lani Seikaly, Hillcrest & Main, Inc.

17. We examined the National Research Council book, Knowing What Students Know, which identified the key concepts on which good assessments are built.From Knowing What Students Know by James W. Pellegrino, Naomi Chudowsky, and Robert Glasser/ Lani Seikaly, Hillcrest & Main, Inc.

18. Every assessment, regardless of purpose, rests on three pillars: • Cognition, a model of how students represent knowledge and develop competence in the subject domain • Observation, tasks or situations that allow one to observe students’ performance • Interpretation, an interpretation method for drawing inferences from the performance evidence thus obtained. James W. Pellegrino, Naomi Chudowsky, and Robert Glasser Lani Seikaly, Hillcrest & Main, Inc.

19. To understand what students know, staff need to understand • the knowledge and cognitive domains of the content standard indicators • how students learn • how to provide students opportunity to demonstrate what they know • how to interpret student responses Lani Seikaly, Hillcrest & Main, Inc.

20. Knowing What Students Know also suggests that “Student work should focus adult-student and adult-adult conversations to discuss standards.” Lani Seikaly, Hillcrest & Main, Inc.

21. We have also read about the importance of collaborative examinations of student work. Lani Seikaly, Hillcrest & Main, Inc.

22. Joan Richardson, editor of the National Staff Development Council newsletter, believes that“The practice of having teachers work together to study student work is one of the most promising professional development strategies in recent years. Examining student work helps teachers intimately understand how state and local standards apply to their teaching practice and to student work. Lani Seikaly, Hillcrest & Main, Inc.

23. “As they see what students produce in response to their assignments, they can see the successes as well as the situations where there are gaps. In exploring those gaps, they can improve their practice in order to reach all students.” Lani Seikaly, Hillcrest & Main, Inc.

24. The Aspen Workshop on High Schools recommended in its summary report for the Transforming High Schools Task Force that the continuous and collaborative examination of student work along with the personalization of schooling are the two critical strategies for transforming high schools at the local level. Lani Seikaly, Hillcrest & Main, Inc.

25. “At the risk of sounding overly simplistic, the use of student work as the unrelenting focus of adult conversations can be the catalyst of fundamental changes in the educational experience of adolescents, and the transformation of teaching and learning at the high school level.” Lani Seikaly, Hillcrest & Main, Inc.

26. Kate Nolan, Director of Re-Thinking Accountability for the Annenberg Institute of School Reform, believes “Rich, complex work samples show us how students are thinking, the fullness of their factual knowledge, the connections they are making. Talking about them together in an accountable way helps us to learn how to adjust instruction to meet the needs of our students.” Lani Seikaly, Hillcrest & Main, Inc.

27. So we understand that to know where each of our students is in relation to content standards, we need to regularly examine student performance to inform our instruction and to monitor student progress. What would that look like? How would we operationalize that? Lani Seikaly, Hillcrest & Main, Inc.

28. What does a team examination of student work look like? • How do teachers define proficiency? • How do teachers diagnose strengths and needs? • How do they record the diagnostic information? • Who leads the discussion? • What questions need to be asked? Lani Seikaly, Hillcrest & Main, Inc.

29. Examining Student Work Though teachers have always examined student work as part of their grading process, the new focus on accountability and standards has driven a more structured and collaborative examination of student work. Lani Seikaly, Hillcrest & Main, Inc.

30. Examining Student Work The focus of the examination has shifted from a summative evaluation of student performance to a diagnostic evaluation of student performance, teacher assignment, and implications for instruction. Lani Seikaly, Hillcrest & Main, Inc.

31. The Examining Student Work Protocol asks teachers to • Reach consensus about what makes a student response proficient on anassignment or assessment • Diagnose student strengths and needs on the performance • Determine next instructional steps based on the diagnosis Lani Seikaly, Hillcrest & Main, Inc.

32. In the first part of the protocol, a team of teachers work through the process of reaching consensus on what the team believes constitutes a proficient response on a selected text and question. Lani Seikaly, Hillcrest & Main, Inc.

33. Stiggins argues that we really can’t assess accurately if we don’t understand the target: “To assess student achievement accurately, teachers and administrators must understand the achievement targets their students are to master. They cannot assess (let alone teach) achievement that has not been defined”.Stiggins, Richard J. 2001. “The Principal’s Leadership Role in Assessment.” NASSP Bulletin (January 2001): 13–26. Lani Seikaly, Hillcrest & Main, Inc.

34. Part 1: Reaching Consensus about Proficiency • What did you ask the students to do? • Which content standard indicator were you assessing? • What did you consider proficient performance on this assignment? • Exactly what did students need to say or write for you to consider their work proficient? Lani Seikaly, Hillcrest & Main, Inc.

35. It is not enough that an individual teacher defines proficiency. It is critical that at least a grade level team has reached consensus on the definition of proficiency to ensure that all students are held to the same performance expectations. Lani Seikaly, Hillcrest & Main, Inc.

36. Teachers were surprised at how “off the page” their teammates were. Think of all the mixed messages our students are receiving when we haven’t defined proficiency on a standard / indicator in the same way. Lani Seikaly, Hillcrest & Main, Inc.

37. Let’s take a look at a team of 3rd grade teachers trying to get on the same page about what a proficient answer would need to include for a reading comprehension assessment. Lani Seikaly, Hillcrest & Main, Inc.

38. Only after the team has agreed on what constitutes a proficient response are they able to diagnose student strengths and needs. Lani Seikaly, Hillcrest & Main, Inc.

39. In the second part of the protocol, the team examines three student papers to determine if the response is proficient and to identify strengths, needs and instructional next steps. Lani Seikaly, Hillcrest & Main, Inc.

40. Each teacher is asked to bring three samples of student work from the same assignment or assessment: a response at the top of the class, a response at the bottom of the class and a response in the middle of the class. Lani Seikaly, Hillcrest & Main, Inc.

41. Part 2: Diagnosing Student Strengths and Needs • What did the student demonstrate that he/she knew? • What misconceptions or wrong information did the student have? • What did the student not demonstrate? • How would you find out if he/she knew it? Lani Seikaly, Hillcrest & Main, Inc.

42. Teachers must shift their mindset from scoring (a summative examination) to diagnosing (a formative examination) student performance. Lani Seikaly, Hillcrest & Main, Inc.

43. In many cases teachers have spent a great deal of time sorting student responses (either by letter grades or by rubric scores) and virtually no time diagnosing what students know and still need to learn. Lani Seikaly, Hillcrest & Main, Inc.

44. Let’s take a look at our team of 3rd grade teachers diagnosing their student responses. Lani Seikaly, Hillcrest & Main, Inc.

45. Part 3: Identifying Instructional Next Steps • Based on the team's diagnosis of the student performance, what would you do next with that student? • What questions might you ask the student? • What feedback would you give? • Do you need to re-teach anything to the class or a subset of the class? Lani Seikaly, Hillcrest & Main, Inc.

46. How have teams captured the data? Lani Seikaly, Hillcrest & Main, Inc.

47. Lani Seikaly, Hillcrest & Main, Inc.

48. What can you learn from examining student work? • What do teachers learn about students? • What do teachers learn about their instruction? • What do school teams learn about their team’s understanding of content standards? Lani Seikaly, Hillcrest & Main, Inc.

49. Teachers came out of this examining student work process with • Grade level consensus of what constitutes proficient work on the assignment • Formative assessment data • Specific information to inform their instruction • Strategies for re-teaching • Deeper understanding of the intent of the standard / indicator they were assessing. • Probing questions to ask students to better understand where they were Lani Seikaly, Hillcrest & Main, Inc.

50. Andrew Nelson, teacher at Harmony Hills E.S. “I wish I had caught onto this earlier but at one of the meetings, it became apparent that kids weren’t reading the question. That was a big awakening to me because I was so focused on how to write the answer, we hadn’t spent time unpacking the question.” Lani Seikaly, Hillcrest & Main, Inc.