Learning Progressions for Describing Teachers’ Use of Curriculum Materials

1. Learning Progressions for Describing Teachers’ Use of Curriculum Materials Christina Schwarz, MSU; Betsy Davis, UM; David Kanter, Northwestern; Sean Smith, Horizon CCMS KSI July, 2006 This material is based on work supported by the National Science Foundation under Grant No. ESI-0227557. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

2. What Do We Mean by a Teacher Learning Progression? • Not a stage theory model of teacher development • A learning progression that can help us: • Clarify beginning, middle, and end goals for teacher curriculum use • Think about teacher use of CMs with and without intervention and guidance • Think about what types of intervention and guidance are needed and when • Think about the role of mediating factors such as context, teacher knowledge, skills, and beliefs

3. Session Rationale • Why a teacher learning progression around CMs? • CMs are important in teacher practice, and teachers’ use of CMs evolves and changes over time. • Beginning teachers are highly dependent on CMs, and it’s important for them to know how to evaluate, select, and modify CMs in a principled, reform-based manner. Important to help beginning and experienced teachers develop a participatory relationship with CMs that enables effective curriculum modification, learning, and use. • Lack of knowledge around teacher progressions hampers teacher educators from designing effective teacher education and curriculum designers from creating effective materials for a range of teachers along the continuum.

4. Session Goal Have a conversation and map out a tentative teacher learning progression or set of progressions for pre-service, beginning, and experienced teachers around curriculum materials use, including: • Curriculum evaluation/choice (including finding and evaluating CMs) • Curriculum modification/planning • Curriculum enactment • Reflection with CMs (looking at student learning and outcomes) • Learning with CMs • Identity with respect to CMs (including professional role as teacher) Mediating factors: Context (learning community and tools), teacher knowledge (content and other), pedagogical skills (management, etc.), and teacher beliefs.

5. Session Questions • How do teachers interact with curriculum materials with and without support? What sorts of participatory relationships do they have with respect to CMs? • How do those aspects change over teachers’ careers? • What might we want teachers to know and be able to do with respect to CMs? • How might we help teachers get there? • What kind of research might we need to determine the answers to these questions?

6. Teacher Learning Progressions

7. Organization of Session • Introduction (10 min) • Presentation of findings from session organizers (50 min) • Break (15 min) • Small Group work on mapping out one dimension of LP (50 min) • Sharing of LPs (30 min) • Discussion (25 min)

8. Christina Schwarz, ETCM group at MSU • Focus on helping pre-service elementary teachers learn to evaluate and modify curriculum materials for effective science teaching. • Use P2061 criteria to guide pre-service teachers in evaluating materials (e.g. providing a sense of purpose, building on students’ ideas, engaging students in real world phenomena, etc.). • Used criteria to evaluate materials • Used criteria to evaluate and modify set of lessons taught in elementary classrooms • Trials of implementation of approach in 3 methods sections -- iterations over 3 years. Year 2 studied extensively. • Analyzed scaffolded and spontaneous use of P2061 and own criteria. Looked at attitudes towards P2061 criteria use and modification.

9. ETCM work

10. ETCM findings • Pre-service teachers evaluations of materials based on strong intuitive notions of what makes good material. Often, a practical orientation -- age appropriate activities that can be easily managed, fun, with hands-on activities, and provides clear science explanations for both teachers and students. • Efforts to focus pre-service teachers on evaluating materials with P2061 criteria a challenge because the criteria did not match intuitive notions and the task sometimes seemed inauthentic, unnecessary, and sometimes destabilizing. Needed more support in modifying materials to meet criteria. • Some indication that PSTs felt lack of power and knowledge for modifying materials, but were willing to consider modification when related to practical management concerns or to address concerns about age appropriateness.

11. Other ETCM Findings and Recommendations • Better results when focusing on helping PST evaluate and modify CMs with an instructional framework that helped them see HOW they can modify curriculum materials and brings in an overall guideline for how to do this and an image for this kind of instruction. • Engage: Establish a problem. Ask a question. Elicit student ideas. • Explore and Investigate: Explore phenomena and students’ ideas. [Ask students to derive patterns and explanations based on evidence.] • Explain: Introduce scientific ideas. Compare student ideas. • Apply: Model concept use and application, coach, and fade.

12. Other ETCM Findings and Recommendations • Providing several experiences evaluating, modifying, and enacting CMs in the pre-service stage may be very important, as well as discussing related issues. (When can and should I modify? What if my CT or principal tells me I have to stick to the book and worksheets?) • Important to match the PST communities with one-another so that they see this practice in action. Have CTs discuss CM evaluation and modification to make this invisible task more explicit. • Create tasks or cases related to curriculum evaluation and modification in teacher education courses that are engaging and authentic for PSTs.

13. End Goal for PSTs Where do we want PST’s to be at the end of their methods courses? • Knowing that CMs can be a useful resource, but they need to be evaluated and carefully modified to meet learning goals and student needs • Having gained experiences and skills in evaluating, modifying, and enacting of CMs in a principled, reform-based manner • Having gained experience working with a community (colleagues, developers, teacher educators) in evaluating and modifying CMs. And to a greater or lesser extent, enacting CMs and reflecting on that enactment.

14. Betsy Davis and the CASES group at UM • Preservice teacher education (several studies have been published) • Example research question: What is the basis for preservice elementary teachers’ critique of instructional materials in science? • Participants: Preservice elementary teachers at UM • Example data sources: work on critique & adaptation assignments • Longitudinal study (ongoing; no studies have been published) • One overarching research question: How do new elementary teachers’ knowledge and practice change over time? Specifically we’re interested in inquiry-oriented science teaching and related dimensions. • Participants: 7 elementary teachers, UM preservice through sixth year of teaching, nationwide • Multiple data sources: interviews, log files, reflections, etc. bd

15. Betsy Davis and the CASES group at UM bd

16. Betsy Davis and the CASES group at UM: What have we learned about… • Preservice teachers’ evaluation and modification of CM (A&B) (see Davis, 2006, Science Education; Forbes & Davis, 2006 ASTE / submitted) • The preservice elementary teachers held a sophisticated set of criteria for critiquing instructional materials; for example, they paid attention to scientific inquiry and instructional goals. • Even with explicit support, the preservice teachers did not engage in substantive critique about how scientific content is represented. • The preservice elementary teachers critiqued and modified curriculum materials dealing with socioscientific issues in ways that reflected their own orientations toward meaningful science learning and their developing understanding of inquiry-oriented science. bd

17. Betsy Davis and the CASES group at UM: What have we learned about… • Preservice teachers’ learning from and with CM (E) (see Smithey & Davis, 2004, ICLS; Dietz & Davis, in progress) • Since this is very specific to supports we use in CASES, we won’t discuss this today. • When responding to narrative images of inquiry, half of the preservice teachers studied reported agreement with instructional decisions of the image teacher as the only rationale for identifying with them. The other half also considered issues of similarity of circumstance and whether or not they could relate to the image teacher. • Also when responding to narrative images of inquiry, the preservice teachers reflect on five common themes (including identity, students’ ideas, and teacher modifications to CM), while also reflecting on some themes that are specific to particular narratives (including inquiry and learning goals). bd

18. Betsy Davis and the CASES group at UM: What have we learned about… • Preservice teachers’ curricular role identity (F)(see Forbes & Davis, 2006 ASTE / submitted for review; Forbes & Davis, in progress) • In critiquing and adapting CM dealing with socioscientific issues, the preservice teachers’ SMK, informal reasoning about socioscientific issues, and their role identity mediated their efforts. • They adopted a value-neutral approach to practice. • They were attuned to standards and learning goals. The challenge was to navigate conceptually-oriented goals and SSI-oriented ones. • Curricular role identity development is tied to preservice teachers’ practicum experiences. Many rarely interact, or observe cooperating teachers interact, with CM and don’t perceive CM a fundamental part of classroom-based practice. This is a function of what materials are available. • Preservice teachers view CM as always helpful for inexperienced teachers but less so for experienced teachers, except in cases where the teachers are learning to use new CM or teaching unfamiliar content. cf

19. Betsy Davis and the CASES group at UM: What have we learned about… • Beginning teachers’ evaluation and modification of CM (A&B)(talk to Forbes, Stevens, Beyer, Smithey, and/or Davis for different slants on this) • In general, beginning elementary teachers do engage in critique and adaptation of existing CM. They draw from a variety of resources but experience a tension between an investigation-orientation and a text-orientation. This tension is constructed and articulated differently by each teacher. • Time or experience isn’t the only factor in change. Context matters in how beginning teachers critique, adapt, and use CM. • The teachers have a wide variety of science CM to work with. • Their available CM influences the development of their science teaching practice over time. • Having and using a stable set of CM over time may help promote the development of PCK and pedagogical design capacity. • Some beginning teachers show increasing sophistication with regard to the criteria along which they critique instructional representations in CM and consider inquiry practices supported by the CM. cf

20. Betsy Davis and the CASES group at UM: What have we learned about… • Beginning teachers’ enactment of CM (C)(see Beyer, in progress) • Since this is a case study of a single teacher, we won’t discuss this today. • In enacting the educative materials, one third-year, second grade teacher developed a more sophisticated understanding of scientific explanation, adopted learning goals for some lessons that emphasized this inquiry practice, and developed instructional practices to foster students’ explanation construction. • However, she tended to emphasize the importance of learning factual content above the importance of generating explanations in some of her learning goals and in her instructional and assessment practices. She did not see this inquiry practice as an instructional strategy for facilitating students’ understanding of the science content nor did she see this inquiry practice as an educational goal in its own right. cf

21. Betsy Davis and the CASES group at UM:An Overarching Question • How can we make sense of longitudinal data like these to make claims about a learning progression? cf

22. David Kanter, Northwestern • In-service middle and high school Biology teachers • Curriculum-driven, practice-based professional development(Learning and Teaching Human Biology graduate course) • Concurrent with first enactment of project-based inquiry science (PbIS) I, Bio or Disease Detectives Biology curricula • Split between beginning and experienced teachers,although none particularly experienced with pedagogy or content • 12 weeks during school year, 3 hours, 1 evening/week • Co-taught with practicing teacher experienced with the curriculum

23. PbIS curricula designed around big Biology content ideas support opportunities to figure out students’ initial ideas about the big ideas, support students changing their ideas, and figure out students’ final ideas about the big ideas • Teachers learn in order to use pedagogical content knowledge + content knowledge to plan and reflect on puzzling through their students’ conceptual change • New big ideas added each week as we work toward enacting the next lesson • Reflecting and planning is structured by organizer that requiresdesign/ identify/ interpret (+ reasoning)/ design (+ reasoning)when clarifying or changing students’ ideas about (stated) big idea

24. David Kanter, Northwestern

25. Big Science Ideas Design/Plan • Provide “organizer” Identify Interpret “Teaching Jobs” Design/Plan

26. Use organizer for homework to reflect on enacted lesson that “was” (review during class) • Use organizer for homework to plan to enact lesson that “will be” (review during class) with further planning support during class, prior to enactment. • Planning and reflecting supported by content and conceptions readings and lecture for big ideas in each new lesson, video of other teachers’ enacting that lesson and personally experiencing that lesson. • Weekly, use one organizer per lesson. Update one reflecting on lesson that “was.” Begin another to plan for lesson that “will be.” • Flesh out organizer into paper for two lessons. Classroom video of teachers’ enactment provided to support this task. • (Pre-service teachers in same “methods” type course; complete papers based on observations in in-service teachers’ classrooms)

27. Analyze organizer-structured papers. Compare to pre-measure administered for big ideas from target lessons. • Chunk =self-contained use of “teaching jobs”to clarify or change a big science idea(i.e. column on the organizer) • Four (organizer-supported) qualities • Content • Connected • Correct • Justified

28. GREEN CHUNK -Content focused -Connected -Correct -Justified YELLOW CHUNK -Content focused -Connected -INCORRECT (e.g over estimation; best chunks are correct) -NOT Justified ORANGE CHUNK (pick 2) -SHIFTS AWAY from Content focus -Often UNCONNECTED -INCORRECT (pedagogical) -NOT Justified RED CHUNK -NO Content focus -UNCONNECTED -INCORRECT (pedagogical) -NOT Justified

29. Teacher A Teacher C Teacher B

30. Teacher Learning Progressions of Curriculum Materials Use • Pre: PREDOMINATELY RED OR ORANGE CHUNKS unable to identify relevant student comments/writings/actions unable to make interpretations about what students’ ideas are let alone justify (reasoning) unable to suggest designs to clarify or change students’ ideas let alone justify (reasoning) • Post: FEWER RED OR ORANGE CHUNKS MORE YELLOW AND SOME GREEN CHUNKS able to identify relevant student comments/writings/actions able to make correct interpretations about students’ ideas (although over/under estimation possible) some justify (reasoning) able to suggest designs to clarify or change students’ ideas some justify (reasoning) ALTHOUGH DIFFERENCES AMONG TEACHERS/CONTENT

31. Sean Smith, Horizon Research, Inc. (HRI) • HRI was the external evaluator for an Instructional Materials Development project; observed several teachers enacting the curriculum over 4 years. • The materials • InterActions in Physical Science (formerly known as CIPS—Constructing Ideas in Physical Science) • Year-long physical science curriculum targeting 8th grade; scaffolding of science concepts and science practices across units. • Highly structured curriculum based on a learning cycle approach—lessons within cycles within units. • Instruction heavily based on scientific argumentation and explanation. • Developed by team of physicists and physics educators from SDSU, U of Minn, and WMU. • Designed with P2061 criteria in mind; solicited P2061 feedback regularly during development.

32. Differences in Research Focus • HRI’s focus was on fidelity of implementation. • To what extent was the teacher’s enactment aligned with the developers’ vision? • What factors contributed to a high (or low) fidelity implementation? • How did teachers change over time in their enactment, particularly in relation to curriculum-related PD? • Focus of curriculum-related PD was on enabling teachers to implement with fidelity. • Not about principles for evaluating and choosing curricula. • Not about modifying the curriculum.

33. Teacher Support During Pilot and Field Test • Extensive educative materials to accompany student materials. • Week long summer workshop. • 4 day long workshops during the year. • Afternoon meetings every other week (pilot phase only)

35. Findings • Findings are limited to enactment: how using the materials changed teachers and how teachers changed the materials. • How the materials changed the teachers and their classrooms • All teachers made progress in terms of fidelity, some much more than others. • Teachers spoke of major changes in: • Classroom culture: from teacher dominated to collaborative; from no student talk or disruptive talk to constructive, respectful talk among students. • Who does the intellectual work in the classroom; shift from teacher only to students and teacher sharing the load. • The kind of intellectual work the teacher does in the classroom; from information dispensing to facilitating, making sense of evidence, and anticipating, analyzing, and reacting to student thinking. • Their relationship to the curriculum after the first year of enactment; from an activity-by-activity approach to a unit- and even curriculum-level approach.

36. How Teachers Changed the Materials:A Progression of Sorts • Managing classroom behavior and managing logistical aspects of the curriculum: • Teachers with no class control struggled. Those who could not manage the material and paper demands of the curriculum also struggled. • These teachers implemented with the least fidelity; no constructive discussions, no making sense of evidence. • Managing the pedagogy: InterActions involves a relatively sophisticated pedagogy that depends, among other things, on having certain kinds of discussions at certain points in the cycle. • Teachers focused heavily on the what of the pedagogy, but not the why. More emphasis on the form of the pedagogy than the substance. • Had discussions but these often did not build toward a conclusion, or teachers rushed to conclusion. • A mechanical enactment at best. • Managing the learning: Some teachers eventually saw the why of the pedagogy, and became comfortable with pedagogy, as well as the scope and sequence of the curriculum. • Had discussions guided by students’ ideas and the pedagogy. • Did not rush to closure or short-circuit the inquiries. • A purposeful enactment.

37. Create Learning Progression Along Dimensions • Group 1 - CM choice and evaluation • Group 2 - CM modification and planning • Group 3 - CM enactment • Group 4 - Reflection with CMs • Group 5 - Learning with and from CMs • Group 6 - Identity and CMs We acknowledge these categories overlap. Feel free to modify the framework. Mediating factors within these dimensions include: context (learning community, tools), teacher knowledge (content, etc.), pedagogical skills (classroom management, etc.), teacher vision/beliefs/orientations.

38. Discussion Questions for LPs • Where are teachers? (What do they know and what are they able to do? Unpack the information from these findings and your own.) Where are the teachers with and without scaffolding? What are some of the challenges and opportunities at those particular levels and foci? • Where do we want teachers to be with respect to CMs? When? • How can we help teachers get there? (e.g. educative CMs, teacher education, professional development) • What research might we need to figure this out?

39. Share Learning Progressions; Discuss Main Issues • Share Learning Progressions • Discuss main issues • Where do we want teachers to be? When? • What kind of experiences, scaffolding, or education is needed? (e.g. educative CMs, TE, PD) When? • What should our next steps be?