Promise and Problems of Learning Progression-guided Interventions

1. Promise and Problems of Learning Progression-guided Interventions Hui Jin,HyoJeongShin, Michele Johnson, Jinho Kim

2. Overview • Carbon Cycle Teaching Experiment: Using knowledge of photosynthesis & cellular respiration to explain plant growth • Tracing matter • Tracing energy • Connecting scales • Motivations • Validation LPs vs. Evolutionary LPs (Duschl, Maeng, & Sezen, 2011).Evolutionary LPs can be aimed at either knowledge enrichment or conceptual change depending on “different conditions of prior knowledge” (Chi, 2009).  Conceptual change-oriented learning progression • Very few learning progression studies explore teacher’s role in interventions (e.g. Furtak, 2012; Thompson, Braaten, & Windschitl, 2012). Teachers are agents who enact curriculum in class.  Explore teachers’ role in LP-guided interventions.

3. Research Questions • Student Outcomes: What are students’ learning outcomes in an intervention guided by a conceptual change-oriented Learning Progression Framework (LPF)? What learning difficulties do students have? • Teacher Knowledge: How do we develop LPF-based measures of teachers’ CK (content knowledge) and PCK (Pedagogical content knowledge)? What is teachers’ achievement in the CK and PCK assessments? • Teachers’ Impact on Student Outcomes: Is teachers’ CK and PCK linked to students’ outcomes? How? What are some other factors affecting student outcomes?

4. Development Process • The LPF served as a guide in the design of a coordinated set of resources: • Student Assessments • Teacher Assessment • Teaching unit • Professional Development programs and resources

5. Explanation Practice: Learning Progression Framework (LPF) Phenomena Plant Growth Plants gaining mass Gas exchange (CO2 and O2) Level 4. Tracing Matter; Tracing Energy; Connecting Scales 2. Teacher Assessment CK Items: 6 PCK Items: Analyze responses at Levels 1, 2, and 3; 3 Knowledge of student thinking items; 3 Next instructional move items Level 3. Reasoning about Matter and Energy Unsuccessfully 1. Student Assessment 7 Tracing Matter Items 3 Tracing Energy Items 4 Connecting Scales Items Level 2. Hidden Mechanisms Reasoning Level 1. Force-dynamic Reasoning

6. Explanation Practice Learning Progression Framework (LPF) 3. TEACHING UNIT To promote conceptual change Tracing Matter; Tracing Energy Connecting Scales Level 4. Tracing Matter; Tracing Energy; Connecting Scales To engage students in scientific practices Level 3. Reasoning about Matter and Energy Unsuccessfully 4. PROFESSIONAL DEVELOPMENT http://www.pathwaysproject.kbs.msu.edu/?page_id=59 Level 2. Hidden Mechanisms Reasoning Level 1. Force-dynamic Reasoning

7. Data Sources & Analysis

8. LPF-based PCK Rubrics

9. Findings • Student Outcomes • Teacher Knowledge • Teachers’ Impact on student outcomes

10. Student Learning Gains IRT Analyses Distribution of Students’ Responses ** p<0.001

11. Teacher Knowledge: General Pattern

12. CK PCK Teacher Knowledge: The Most Difficult Items CK PCK

13. Teacher Knowledge: The most difficult PCK items • The easiest item (Identifying incorrect description of content): During a discussion about how plants and animals get energy, one student says, “I know animals break down food to get energy, but I don’t think plants break down food for energy because they get light energy from the Sun.” What, if anything, is wrong with this student’s statement? • The most difficult item (Identifying naïve ideas of students): A teacher asks students where plants get their food. A student responds, “Along with soil, plants use carbon dioxide, sunlight, and water to help them make food.” In order to find out more fully how the student’s ideas of how matter is transformed when plants grow, which of the following question would you ask next?

14. Teacher Knowledge and Student Learning Gains • The association of teacher knowledge and student learning gains is statistically significant • Teachers withaverage knowledge level (combined CK & PCK scores) produced a significant learning gain in their students: • 0.59 logits (p<0.001) for 2011-12 • 1.13 logits (p<0.001) for 2012-13 • Teachers who had one logit higher knowledge produced an additional increase in student learning gain: • 0.36 logits (p<0.001) for 2011-12 • 0.34 logits (p<0.001) for 2012-13.

15. Identify High/Low performing Teachers 2011-12 Data Average learning gain Zero learning gain

16. Identify High/Low performing Teachers 2012-13 Data Average learning gain Zero learning gain

17. Compare High-performing teachers with low-performing teachers • Feedback forms from 7 High performing teachers & 4 Low performing teachers • Low-performing teachers taught 6, 7, or 8 lessons • High-performing teachers taught 10 or 11 lessons • Teacher H1 (highest learning gain) taught only 5 activities, but produced the highest learning gain in 2012-13. Feedback from suggests that the teacher used similar activities to replace activities in the curriculum • The coverage of curriculum is positively associated with student learning gains.

18. Implications • The LPF was the basis for developing an intervention that enabled students to learn significant knowledge and practices in an important domain. • We developed LPF-based measures of teachers’ CK and PCK, and those independently contributed to students’ learning. • Teachers’ classroom practices also made a difference, at least in terms of the coverage of curriculum. We are currently analyzing teachers’ classroom teaching videos to examine how teaching practice affect learning outcomes.

19. Questions? • Find manuscript, presentations, curriculum and PD resources at: • www.pathwaysproject.kbs.msu.edu • Further questions, contact: • Hui Jin: jin.249@osu.edu; Michele Johnson: mkiss@ucsb.edu • This grant was funded by the National Science Foundation (NSF) under grant number DUE-0832173. The views expressed here are those of the authors and do not necessarily reflect those of NSF.