Craft Knowledge: The Road to Transforming Schools by Deanna Burney

1. Revelations on Formative Assessment and FeedbackFred HarwoodHugh McRoberts Secondary and SFU’s Today’s Classrooms, Tomorrow’s FutureBCAMT Conference October 24th 2008

2. “I crave the opportunity to discuss my reflections on what I’ve been learning in my classroom, to write about it and to crystallize my learning on teaching and learning. This PLC has invigorated my teaching when others get excited about the things that are exciting me, when they share on what they’ve observed and how the ideas connect with their practice!” • (Helen Sipsas- SFU grad student) • So I want to share with you what I’ve observed and learned in my research on and through formative assessment with my classes and in the PLC. • I have 30 years of experience and have continually been invigorated with how much others, especially my students, can teach me about learning. • May some of these ideas connect with you and your practice and may you be invigorated.

3. . . . we need to rethink what we mean by "research" and "practice." The traditional model, based on separate and distinct research and practice communities, with the former possessing knowledge that will inform the latter, is no longer valid. We need to think of the road between research and practice as a two-way street, where both are valid sources of knowledge about teaching. To be sure, teachers need to study and conduct research on learning and cognition and to incorporate up-to-date findings of research into their practice. At the same time, they can also gain important knowledge about teaching through their own and one another's experience. Researchers, for their part, need to carefully examine the knowledge that teachers and school leaders have acquired. Only by recognizing and using both sources of knowledge can educators truly transform our schools and turn teaching into • a true profession. Craft Knowledge: The Road to Transforming Schools by Deanna Burney

4. Usually, though, craft knowledge is confined to isolated classrooms, where individual teachers keep a tight grip on instruction and student learning. Our education system, quite simply, does not invest in the cultivation and dissemination of craft knowledge. Schools and school systems are not learning communities. • But teachers have a right to investments in their professional development as well as a responsibility to reflect on their work, build their knowledge, share it with others, and pay attention to what others are learning. School systems have an obligation to provide the conditions that will foster this learning, because it is the only way we will continuously improve instruction instead of spinning our wheels. Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

5. . . . teachers have come to regard autonomy and creativity -- not rigorous, shared knowledge -- • as the badge of professionalism. • A great many teachers would be threatened • or offended by any suggestion that they should consult, collaborate, observe other teachers' practice, • or open their own classrooms to their colleagues. • Instead of always working to expand and refine • a body of shared knowledge, • they keep what they know • and what they don't know • virtually secret. Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

6. Recent research on how people learn points to another reason why the current structure of our schools is dysfunctional.2 • This research has demonstrated that robust, fluid, and usable knowledge must be grown by learners through highly active engagement with ideas and their interconnections. Knowledge is neither acquired nor applied mechanically or in piecemeal fashion. It evolves into ever more complex, integrated bodies of thought and skill. Knowledge does not just sit there, waiting to be retrieved; it must be tended, fed, and used. In fact, the way people learn anything -- from the ABCs to cooking to astrophysics -- is by energetically connecting ideas with action. • 2. John D. Bradford, Ann L. Brown, and Rodney R. Cocking, eds., • How People Learn: Brain, Mind, Experience, and School: Expanded Edition (Washington, D.C.: National Academy Press, 2000). Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

7. Learning those kinds of skills is not a solitary endeavor; rather, it needs to be a highly social one. • It depends on continual discussion and demonstration. People learn by watching one another, • seeing various ways of solving a single problem, • sharing their different "takes" on a concept or struggle, and developing a common language with which to talk about their goals, their work, and their ways of monitoring their progress or diagnosing their difficulties. When teachers publicly display what they are thinking, they learn from one another, but they also learn through articulating their ideas, justifying their views, and making valid arguments. Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

8. This type of learning involves unlearning some of what teachers already thought they knew. For example, teachers need to understand that a student's failure to learn may reflect a failure of instruction rather than a lack of motivation or ability on the student's part. This is profoundly countercultural. It requires teachers to radically transform the way they think about instruction and learning, thus radically transforming the system in which these actions take place. • The point is to replace isolation with challenging and supportive relationships and fragmentation with a coherent culture organized entirely around instruction. • This is a monumental challenge. Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

9. Transforming thinking and practice requires people to take risks. They can develop their expertise only if they are willing to experiment, make mistakes, and analyze those mistakes -- with everyone else and in front of everyone else. There is no other way for new knowledge to infuse the system and create stronger instructional practice. This will require everyone in the process to place a high level of trust in everyone else -- colleagues, leaders, and outside experts. Teachers must be assured that they will not be blamed for what they do not know, only for what they refuse to learn. By making mistakes, engaging in constructive dialogue -- however tentative or unsure at first -- and observing and being observed, teachers can measure their practices against the expectations for high-quality teaching in a safe and mutually supportive way. When people do not have to worry about unjust criticism or unfair consequences, they quickly gain confidence and become willing to strive for a very high standard of practice. Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

10. Transforming thinking and practice requires people to take risks. They can develop their expertise only if they are willing to experiment, make mistakes, and analyze those mistakes -- with everyone else and in front of everyone else. There is no other way for new knowledge to infuse the system and create stronger instructional practice. This will require everyone in the process to place a high level of trust in everyone else -- colleagues, leaders, and outside experts. Teachers must be assured that they will not be blamed for what they do not know, only for what they refuse to learn. By making mistakes, engaging in constructive dialogue -- however tentative or unsure at first -- and observing and being observed, teachers can measure their practices against the expectations for high-quality teaching in a safe and mutually supportive way. When people do not have to worry about unjust criticism or unfair consequences, they quickly gain confidence and become willing to strive for a very high standard of practice. Craft Knowledge: The Road to Transforming Schoolsby Deanna Burney

11. SFU summer course’s assignment for the first class in August 2007: “read this chapter and bring a mindmap.” • I was preparing for cooking at camp for the week before the course and was under some time pressure but I’d read the chapter already and I had plenty of mindmaps from a variety of other things I’d done. • I wasn’t properly prepared a week later since I’d missed the words “of the chapter” after mindmap. • Not all homework being done wrong or not being done is due to a lack of effort. I was sensitized to this but noticed that the mentor running the activity based on the assignment had made it possible for me to take part despite not having a mind map of the chapter. • Revelation: How often do I have students, who have not done their assignment for what ever reason, sitting relatively passively while the rest of the class • checks how successfully the assignment was done? Revelation 1

12. ”All assessment should improve learning”. • How many times have I been checking homework in my math classes with students who had not done their homework? How often have they sat fairly passively doing little to learn while I reviewed homework from theoverhead? • I resolved to make sure that even if a student hadn’t done his/her homework that they wouldn’t be sitting non- engaged. • My plan was to prepare a false student assignment to be given to students without homework. It would contain a mixture of mistakes that were common to Grade 10 students that highlight some of the key conceptual ideas. Students would be assessing this assignment in place of another student’s. I was curious as to how well it would be assessed as now it would act as a control factor for consistency Revelation Response

13. These were marked just like the other students’ peer assessments of completed homework. All students were prepped to monitor for mistakes and were led through the process by me at the overhead. I’d demonstrate how the question could have been done and where part marks should be given. . . • just as I’m sure many of you have done. Revelation Response

14. There were 10 students in two classes of Principles of Math 10 who ‘marked’ the fake assignment. • An overview of the results • show scores out of 18 of • 13, 14, 14, 14, 15, 15, 15 1/2, 16, 16, 17. • The distribution of scores with a range of 4 in such a short assessment surprised me. Revelation Response

15. The fact that no student had assessed • the 14.5/18 • that I was expecting shocked me! • . . .the only point of asking questions is to raise issues about which • a teacher needs information or • about which the students need to think. • Black & Wiliam, Working Inside the Black Box Revelation Response

16. Why were these discrepancies present? • I had been assuming that peer assessments were fairly good since I’d tell them when to give part marks and for what reasons. • How many other teachers are assuming the same thing? Have you been including these types of marks in your letter grade reporting? • Can we correct these weaknesses through more awareness, metacognition and further training in assessment for learning? • I am certainly learning that I shouldn’t assume students are attending to me equally with the same care, verbal ability, processing speed, motivation and physical well-being Revelation II

17. “The main problem is that pupils can assess themselves only when they have a sufficiently clear picture of the targets that their learning is meant to attain. • Surprisingly, and sadly, many pupils do not have such a picture, and they appear to have become accustomed to receiving classroom teaching as an arbitrary sequence of exercises with no overarching rationale. • To overcome this pattern of passive reception requires hard and sustained work. When pupils do acquire such an overview, they then become more committed and more effective as learners. Moreover, their own assessments become an object of discussion with their teachers and with one another, and this discussion further promotes the reflection on one's own thinking that is essential to good learning.” • Black & Wiliam, Inside the Black Box Research Thought

18. I I had another created student assignment. “Ima Checkov” completed the homework assignment for my grade 10 classes on Pages 90 – 91 of MathPower 10. Every student who hadn’t done this assignment was going to give her feedback on where she was making mistakes and to help her to improve. Eighteen students from four classes assessed Ima’s homework. I had instructed them to give feedback only and not scores. Five of the eighteen students still gave a score. These self determined scores were • 9/18, 11/17, 11/17, 12/18, 13.5/18. • The two 11/17 scores were from ESL level 1 or 2 students suggesting verbal instructions were insufficient to change their practice. They only marked the questions right or wrong and didn’t give much help to Ima. • I noticed that, despite marking questions differently, they had the same score. The Next Step

19. I need to pursue them further but have a language problem to get at this perceived lack of caring and complying. They both missed a copying error in question 13 that I had highlighted on the overhead. I believe they were out of sync with the marking. Was this due to language issues? • The other three score givers were below passing status at the time of this check. The 13.5 scorer and the 9/18 scorer had given similar feedback and caught all the mark deduction mistakes but they had set their own mark determinations and this is where their scoring differed. • [Writer’s Note: Two of them have raised their percentages to over 57% for the term with summative assessments on the topics of these assessments. The Next Step

20. The thirteen other students just gave feedback. • I assessed the evaluations for helpfulness, error and omission The quality of their feedback had definitely changed with the students being more aware of our goal for improving performance and with them knowing I was looking for more consistency in their evaluation. Students in general made a concerted effort to highlight the places that errors took place but few actually were analyzing and feeding back why the mistakes were being made and how they could improve them • “. . . to be effective, feedback should cause thinking to take place.” • Black & Wiliam, Working Inside the Black Box • So now I am wondering what their thinking is? What is causing these discrepancies? The Next Step

21. I set up the process by having them complete a survey photocopied on the front side of a sheet. I asked the students to think about why the scores on the last group assessment of one student’s assignment varied so greatly and to answer the survey’s questions as honestly as possible to give me good data. • The back side contained another Ima Checkov homework assignment on Page 102 – 3 of MathPower 10 with built in errors. • The difference is that I assessed everyone marking the same assignment and not just those who hadn’t completed homework. The Next Step

22. A sift of the surveys showed some general trends and some excellent observations. From the students who were involved in marking one or two of the fake student assignments I got five main categories of reasons for the discrepancies. • Care (taken), • Motivation (to do it well), • Speed (of processing information), • Verbal Ability (ESL considerations) • Physical Well-being. The Next Step

23. I am learning that there is a definite indoctrinated culture around assessment that must be overcome and can be overcome by a greater awareness of purpose, vocabulary and a focus on assessment for learning and as learning. • “It is often said that feedback is the breakfast of champions. All kinds of learning, whether on the practice field or in the classroom, require feedback based on formative assessments. Ironically, the quality feedback necessary to enhance learning is limited or nonexistent in many classrooms.” • McTighe & O’Connor, Seven Practices for Effective Learning Revelation 3

24. I utilized a variation on the feedback for learning theme. I asked everyone in the Math 9/10 class and the Math 10 Enriched class to partner up and to go through a common (imaginary) student assignment together. • They were to utilize their own answers, the text book answers and to give this student feedback on what types of mistakes they tend to make and what they do well on. • My goal was to involve them in an assessment for learning of the person being marked but knew that it would be assessment as learning for many of them. • It also increased their communication and reasoning while deepening their awareness and understanding of the key ideas and sources of errors. • I also asked them to provide two metacognitions on two seed statements: • “I ______ this process because . . .” and • “During this process I learned . . .” Next Steps

25. “I like this process because it allows me to reflect onto my own work habits and makes me think of the careless mistakes I’ve done on tests and homework for the past years. It also allows me to find a way to correct the habit and try to correct as many answers as possible.” • “I like this process because we learn many things at the same time. Instead of just working on problems, and solving them, we learn and review concepts as we correct someone’s work. As well, by working with a partner, we are able to correct one another, if we make an error. By correcting someone’s work in this method, • I have to a) redo the question b) find possible areas with trouble and c) if they did the question wrong, I have to correct it and give feedback, such as the formulas they used wrong. The process is useful because at the same time, I solve questions and I review the formulas to prevent my own errors in the future.” Observe some student statements from these metacogs:

26. I am learning that collectively students know assessment wisdom but this needs to be activated and shepherded through a refocusing of thinking and work habits. • Some of the evidences of this are: - students saying, • “List things to improve on instead of spreading them out” • “Tell me how and what the student did wrong. Give encouragements. Teach the student slowly step by step why the student did it wrong.” • “Don’t give a score just the feedback.” • “Maybe they can just give feedback and not simply put a “X” mark. Most people only care about the overall score, so they don’t give as much feedback.” • “Mr. Harwood marks our homework once, by himself and gives us feedbacks. We will learn from that example.” • “Make it more clear how much each question is worth.” • “Stop giving us tests and give more assignments or don’t give a test until you’re sure everyone knows the material.” • “Remind me of the important concepts, common mistakes and key ideas.” • “Go slower – make sure everyone is paying attention.” Revelation 4

27. I was a guest teacher in a French Immersion math class out at UBC for an hour. I was allowed to teach in English • BUT then spent three hours as an FSL Level 1 student • as part of the class. • WHAT AN EYE OPENER! <-- Read REVELATION • I highly recommend that you spend some time in alternative settings where you aren’t an expert. We forget what it is like to be a weaker student for whatever reason. • This class in French was an incredible struggle for me. I had 4 years of French in BC high schools 35 years ago but I’ve taught at a French Immersion high school for over 14 years so my ear for French has improved and I’ve seen written output daily. I had little idea of what difficulties I’d have processing the class even though it was on a topic of expertise for me personally – math. I found when anyone spoke English in my vicinity, I’d gravitate to them naturally and ‘fall into’ that conversation. It was a rest for my brain. I thought of all my ESL students taking a rest by listening (and participating) in their own languages. I thought of the math challenged students ‘taking a rest’ in discussing things off topic that were easier to process.I also found four big ideas that would have helped me in the French class that I want to do in my own classes to hel NEXT STEPS TO REVELATION 5

28. I also found four big ideas that would have helped me in the French class that I want to do in my own classes to help those who are language challenged: • 1. Speak slower - I told one student that I liked his French the best because he spoke the slowest. • 2. Use more written instructions - the class instructor at UBC spoke all directions in French and many times I wasn’t totally clear on what to do. I process written French much more effectively by having time to fit things into context and to fill in the unknown words between the known words. When instructions were in spoken French, I didn’t have time to do this well and I was usually lost. • 3. Use pictures more - When one student was presenting his idea for his project, I kept thinking, “If only you would have had a picture/example, I could have made sense of your project so much easier.” • 4. Make sure the big idea is clear - Another student presented his project which I thought was on “Discovery Guides” and struggled with how he talked about his work. I asked in English and found the topic was “Guided Discovery” a totally different concept! • A resource teacher has challenged me with another big idea. • 5. Talk less - many students are just overwhelmed with input and once their brains are full, no more processing is possible. Next Steps

29. Nancy Atwell (1998) said, • “In addition to creating veritable smog banks of jargon in my earlier descriptions of writing workshop • (I pledge never, ever again to empower students with ownership), I know I continued to view my students and their writing through the clouds of my own needs as a teacher and human being, however sincerely I was convinced I had cast off all assumptions and was seeing my student writers with clear eyes.“ From my readings

30. As I read this, in light of our lens on students • as diverse learners, • I realized that I have said on many occasions • and with knowing nods from the other teachers of experience, • “There is more diversity in my essentials class than there is in my regular classes!” Buildup to Revelation 6

31. This is a bulloney • (a Pentacostal phrase like ‘holy crap’ • meaning ‘ridiculous statement’) • that I can’t believe I have said and believed! • I only had 16 students last year in Essentials 10 • and over 30 in my regular 10 blocks. • This perception of a greater spread came from • the fact that I spent more time • in individual instruction with the smaller class • and got to know them more deeply. • The larger blocks were perceived more as a totality with only the vocal few • exposing their learning differences. Revelation 6

32. The biggest mistake of past centuries • in teaching has been to treat all • children as if they were variants of the • same individual, and thus to feel • justified in teaching them the same • subjects in the same ways. • (Howard Gardner in Siegel & Shaughnessy, 1994; Phi Delta Kappan) Revelation 7

33. These revelations make me want to utilize • my research into formative assessment • and involve my students more • in the process of assessment for learning. • As I proceed into the study, many assumptions • I had been making may surface and • more powerful revelations may occur. • More barriers existed then I had expected. • My plan this year is to further this study • by exposing and reducing these barriers • for students. • I know other teachers will also be challenged • in the way that I was. Past Research

34. Last year’s focus question was, • “How can I get my students to give quality feedback to each other and themselves • to improve learning?” • This year’s focus question will be, • “How can I engage, enable and equip • my students to give quality feedback • to each other and themselves • to improve learning for all students?” • What are the differences and • why do you think they were made? ? Essential Research Questions