Exploring design features for effective teaching cases

1. Exploring design features for effective teaching cases Matthew J. Koehler Michigan State University

2. Context for my interest • A circuitous route to this research area • Mathematics  Computer Science  Artificial Intelligence  Cognitive Psychology  Educational Psychology • Work in mathematics and science classrooms exposed me to research projects interested in conveying models of student thinking to teachers so that they could become better teachers (Carpenter & Fennema; Lehrer; Lehrer & Schauble). • Each of these projects wrestled with how to develop materials and professional development experiences that helped teachers accomplish this.

3. Context for my interest • Some common elements across efforts: • Complex, but structured models to be conveyed. • A belief that pieces of video of student reasoning, or classroom practices were the fertile ground for this learning to take place. • What could I offer? • Thinking about the ways that recent advances in technology (e.g. video and hypermedia) might be designed to scaffold this process of teacher development.

4. Ancient History (e.g., 1998) • Koehler, M.J., & Lehrer, R. (1998). Designing a hypermedia tool for learning about children’s mathematical cognition. Journal of Educational Computing Research, 18(2), pp. 123-145. • My first work in this area was developing teacher resources for Cognitively Guided Instruction (Carpenter & Fennema, 1992) • They had a handbook for teachers that described a model of children’s thinking about simple arithmetic, rich with text-based examples of children’s solution strategies • Could materials adapted to a hypermedia format, using video vignettes of children’s reasoning show any advantage in teacher learning?

5. Ancient History (e.g., 1998) • ANSWER: Yes • The study used pre-service teachers studying with the text-only materials and hypermedia, video-rich materials. • Results showed that the hypermedia / video materials were more effective in two areas of teacher learning: • A taxonomy of word problems for addition and subtraction that had implications for instruction and the development of children’s reasoning • Identifying the developmental sequence of solution strategies that children bring to bear for each of the problem types.

6. Ancient History -- Summary • At the very least, well-designed hypermedia that used video to exemplify children’s thinking has something to offer in the realm of teacher learning. • Important here, is that the way in which video was used was only to exemplify children’s thinking. Exemplification, is of course, important for understanding in general, but seemed to be particularly useful here for developing teachers knowledge of children’s thinking (even though text described the same “episodes” of children’s thinking)

7. Broadening the questions • Given that the approach of using video and hypermedia showed some potential, what were the next steps? • Moved to a consideration of what else you could do with video and hypermedia. What about teacher performances? What about classroom video? What to put in and what to leave out? • Enter :: “Case” and “Case-methods” • “One common definition suggests that a case is a descriptive research document, often presented in narrative form, that is based on a real-life situation or event. It attempts to convey a balanced, multidimensional representation of the context, participants, and reality of the situation” (Merseth, 1994).

8. Broadening the questions • Also enter a research project by Lehrer (geometry); and Lehrer & Schauble (modeling in mathematics and science). • Up to 40 practicing teachers at one point. • Regular teacher meetings that use text, classroom video, examples of children’s work, and teacher writings. • New curriculum, being design and implemented by teacher-researchers • A *LOT* of video from a *LOT* of classrooms. • QUESTION: Given the complexity in these projects (student models, teaching models, curriculum, examples of student work, teacher writings) … • HOW to develop a case that would be effective at helping learning teachers “see” and understand all of this.

9. Recent History (2002) • Koehler, M.J. (2002). Designing case-based hypermedia for developing understanding of children’s mathematical reasoning. Cognition and Instruction, 20(2), 151-195. • Most of the remaining talk explains this paper in more detail. • High level overview: Tests the effectiveness of a number of decisions about how to use video (and cases) effectively.

10. Core Research Questions • What makes for a good case? • What elements should be present? • How should elements be sequenced? • Do you follow the teacher or students? • How do you design a case that meets your goals? • When to link to cases? • When (and how) to link *from* cases? • How to help teachers “see” what is important in the cases. • Given that there are different ways to accomplish the same goals, which ways are most effective for teacher learning?

11. Some caveats Primarily talking about what makes a good case in K-6 mathematics teaching Keeping in mind the kind of materials that are used in the professional development communities that I described (video,children’s work, texts, etc.) Five general principles to guide development. Good cases are: Situated in Practice Layered with Annotation Annotated with Big Ideas “Criss-cross” the domain Anchor exploration What makes for a good case?

12. Elements of good casesSituated in practice • Since teaching is situated in classroom practice, cases of teaching should also be situated in classroom practices. • Advocate use of classroom video • Video is more engaging and facilitates remembering (CTGV). • Video is more like “being there” than text • Written accounts of a classroom assume that textual expression can completely express the dynamics of classroom activity.

13. Elements of good casesLayered with annotation • Video does not speak for itself. • Any two viewers of a classroom video are likely to see different things, especially if they differ in experience, perspective, or expertise. • Classroom events are often subtle and difficult to interpret. • Therefore, video cases should be layered with annotation that helps teachers interpret classroom situations, so that teachers understand what the video is “a case of.”

14. Elements of good casesAnnotated with “Big Ideas” • Big ideas in mathematics are important landmarks in teaching based on models of student thinking (Schifter, 1996; Lehrer & Schauble, in press). • Accordingly, annotation should help teachers “lift out” and interpret the “big ideas” of the domain as they occur in the case. • Like big mathematical ideas • norms for argument (Yackel & Cobb, 1996) • general trajectories of student thinking (Carpenter & Fennema, 1992).

15. Elements of good cases“Criss-cross” the domain • Teaching and learning comprises a complex, ill-structured domain, cases often embody more than one “big idea.” • The same episode can be related to the “big ideas” in mathematics, children’s thinking, the use of tools and notations, and the classroom norms of teaching. • Good teaching requires not only understanding these ideas in isolation, but also orchestrating them to design effective classroom environments. • Cognitive Flexibility Theory (Spiro, Coulson, Feltovich, & Anderson, 1988) suggests that cases should “criss-cross” the conceptual landscape.

16. Elements of good casesAnchor Exploration • Cases that portray complex, ill-structured classroom situations often raise several important issues • For example, the same episode bring up “big ideas” in mathematics, children’s thinking, the use of tools and notations, and the classroom norms of teaching. • Cases should situate, or anchor (CTGV, 1990), explorations into these important ideas by providing access to further information (e.g., text, interpretation, related case, etc.) as issues arise in the case. • In contrast, if cases only represent the main story line, teachers may come to understand children’s development as a fixed progression through stages.

17. Tried to use these ideas in the development of a case-based tool for teachers about length and area Measurement (based on the work of Lehrer et. al). Often taught and understood procedurally Instead, instruction should help children to understand the mathematical ideas that underlie measurement (e.g., all the units are the same size) Goal was to build a case-based hypermedia that emphasized 6 strands of teaching and learning: Key Mathematical Ideas (e.g., Identical Units) Classroom Norms (e.g., make thinking visible) Children’s thinking (e.g., measurement = rulers) Connections to other Math ideas (e.g., fractions) Classroom activities (e.g., building tape measures) Tools and Notations (e.g., graph paper) The Domain of Measurement

18. Candidates for good cases • Given: • the domain to cover (The six strands of measurement) • A set of design principles for good cases • There a still a lot of possible designs to explore. • This study focused on two possibilities …

19. Two types of casesExemplification • One emphasized exemplification (like in the previous work) • Mini-Demo of this type of case • Uses all five principles • Situated in practice • Layered with Annotation • Annotated with “Big Ideas” • “Criss-crosses” the domain • Anchors exploration

20. Two types of casesNarrative • The other type of case emphasizes narrative structure • Cognitive reasons in general - • Narrative structure facilitates remembering and structures knowledge organization (Mandler, 1984) • Teacher learning reasons • Has the same footage as the examples do, but shows how they are structured over time. • How students’ ideas are initially raised, developed, and recalled in later classroom episodes. • Shows how episodes of student thinking are sequences to form a lesson, how lessons are used to design activities, and how activities and lessons are sequence to develop big ideas in the curriculum.

21. Two types of casesNarrative • Mini-Demo of this type of case • Uses all five principles • Situated in practice • Layered with Annotation • Annotated with “Big Ideas” • “Criss-crosses” the domain • Anchors exploration

22. An ExperimentRationale • Wanted to contrast the type of learning afforded by these two types of cases • Believed that the advantage of narrative cases lies in the causal structure that ties stories together (van den Broek & Trabasso, 1998). • The ability to apply knowledge relies, in part, on understanding the causal relationships between situations and actions that need to be taken (Eylon & Reif, 1984). • Therefore, I expected that narrative cases would be more likely to lead to knowledge that could be applied.

23. An ExperimentProcedure • Made two versions of the hypermedia tool • One version had exemplification cases only. • The other version had exemplification AND narrative cases. • Twenty-four pre-service teachers, randomly assigned to study with one version of the hypermedia tool. • Measures before study, after study, and 6 weeks after study • Speak aloud to video - participants saw short classroom segments. Following each clip, participants were asked to identify any important elements of teaching or learning about measurement that they saw. This was used to track the type of knowledge that participants acquired. • Analysis of student work - Participants were asked to apply their knowledge to an analysis of student work. Interviews addressed what the sample student understood (or did not), what the student needed to understand, and what classroom activities would most likely help this student gain understanding. This measured the ability to apply knowledge.

24. An ExperimentResults • Speak aloud video interviews showed that: • Both groups gained knowledge about the mathematics of measurement and about the teaching norms in place in the classes illustrated in the hypermedia. • Before, after, and six-weeks after instruction, both groups were equally able to name the big ideas (e.g., “identical units”) when they saw them in the videos • Conclusion: There were no difference in knowledge acquisition • Analyses of student work showed that: • The group who had access to narrative cases did better at applying their knowledge to their analyses of student work • More about this ...

25. An ExperimentAnalysis of student work • Measuring Application of Knowledge … • What does the following student understand about measurement? • Before instruction • Both groups tended to give procedural explanations • After instruction • Both groups improved … more so for the narrative group

26. An ExperimentOther findings • This trend towards better application of knowledge by the narrative group shows up in other questions of the student work interview • Better at listing all the requisite knowledge a student would need to understand the problem • Better at suggesting appropriate follow-up activities • Have better memory for the classroom activities outlined in the hypermedia tool • Analysis of their time allocation during study supports the view that the narrative cases were responsible for these differences • Tended to read less text than their exemplar-only counterparts • Tended to watch less exemplar cases • The more time spent watching narrative cases was predictive of better analysis of student work (up to a point).

27. Take-away message • The nature of cases, how they should be crafted, and the consequences of different knowledge structuring are all important questions to investigate. • This work shows that even given some guiding principles for design (the five), competing designs have different affordances for learning • Cases used for exemplification are pretty good at helping students acquire declarative knowledge. • Cases organized around narrative of classroom events has some potential for fostering the application of that knowledge.

28. More questions … / Future work .. • Since then, my work has opened up considerably … • Most broadly, I’m interested in how the many design decisions that go into making and delivering cases impact learners. • Here are few specifics of things that I’m working on …

29. Cases for Different Domains • Science Learning (with Lehrer and Schauble) • Different way to highlight ideas in the video cases • Different subject matter (science) • http://mkoehler.educ.msu.edu/growth/ • Early literacy teaching (K-2) (Hopefully w/ Nell Duke) • What elements are needed in a case about literacy instruction • Overall, interested in learning what elements of effective design are domain dependent and which are general purpose.

30. Future work - Student-authored cases • Working with a team at MSU on how to design a system that scaffolds student-authored cases of classroom video • What a student might learn from crafting a case • What kinds of video do they use as evidence for their claims? • What types of sequences would students build? • What would they learn? Interface design by Charlie Ruggiero

31. Future Work - When is video advantageous? And for what? • Aimed at figuring out what the cognitive affordances are for video vs. text treatments for various types of information/stories/genres in the teacher ed population (those 18-21 yr olds) • Currently collecting data • Looking at text and video versions of the same material • Video lectures vs. text transcripts • Poetry vs. Poetry readings • Stories from the nightly news. • Trying to figure out what the effect of video treatments are in terms of motivation, interest, affect, fact learning, persuasion, recall, and “gist” learning. • Measures

32. Future Work - Video vs. Text • Preliminary results are “interesting” • Poetry reading vs. text of poem • Video has, if anything, a negative impact on students interest level • Doesn’t seem to help their understanding of poem at all (and perhaps hurt it) • News story on college drinking (fact-laden story, very serious) • Seems to be no difference in text vs. video treatments on students’ interest, their ability to recall facts from the story, or a change in their beliefs about the problems of student drinking. • Story about AIBO, the robotic dog. • “Medium-size effects” for people’s interest level, change in emotional state, change in belief that robotic dogs could make a good companion, and ability to recall information • BIG differences in the quality of students’ written summary of the story. • WHY???

33. Future work - Genres differences • Looking at the “same classroom video” footage rendered in different story or TV genres • Case-format (from before) • News-story • Documentary • Drama • Reality TV (e.g., MTV’s “The Real World” … no narrator) • Looking for the different learning affordances of each for the teacher candidate population (e.g., 18-21 yr olds). Looking at factual recall, interest levels, “gist learning,” opinion changes, etc. • Hopefully some correspondance to previous slide

34. Conclusions and Questions • In general, I’m keenly interested in unpacking the learning affordances of the elements of effective case design. • Questions? Comments? Critiques?

35. Sample Data from speak aloud video Lesson has good use of visual manipulatives Teaches about the need for same-size units Teaches about units that are not whole Teaches that things that look different may be the same size Teaches about breaking big measurements into smaller ones Teaches about reusing units Activity is hands on Teacher uses a variety of units Good introduction to tools such as rulers Teacher uses open-ended questions Students lead class discussions Teacher takes input from the students Using more than just standard units Teachers build understanding by restating students ideas Teachers bring up history Children understand that you cannot have gaps between units Children understand the need for identical units Children account for fractional units Teachers help students see consequences of their ideas Children understand that different objects can have the same measure Children understand that units can be reused Episode Only (Post-test) Episode Plus Narrative (Post-test)

36. What would a student need to know to understand the problem Not using the diagram and 1x1 squares correctly Doesn’t understand all the units have to be the same size Student would need to know the diff between square and rectangle All the units of a measurement need to be the same size There can be no spaces between the units Episode Only (6-weeks later) Episode Plus Narrative (6-weeks later)

37. Follow up activities suggested

38. Very Early Results