A Study of Mathematics Coaching in (U. S.) Elementary Classrooms

1. A Study of Mathematics Coaching in (U. S.) Elementary Classrooms Elizabeth A. Burroughs Montana State University Department of Mathematical Sciences RCT Conference University of York, York, U.K. 11 September 2014

2. Research contributors Principal Investigators Beth Burroughs, Montana State University John Sutton, RMC Research Corp. David Yopp, University of Idaho Contributing Researchers Mark Greenwood, Megan Higgs, and Jennifer Luebeck (Montana State University); Brandie Good, Clare Heidema, Dan Jesse, and Arlene Mitchell (RMC Research Corp.). Funded under NSF Award No. 0918326. Any opinions expressed herein are those of the authors and do not necessarily represent the views of the National Science Foundation.

3. Context • I’m a U.S. mathematics education scholar, spending the autumn and spring terms at the University of York in the Department of Education as a Fulbright Scholar. • My home is in Bozeman, Montana, at Montana State University

4. Mathematics classroom coaching • A recent development in mathematics professional development for practicing teachers. • Built on a foundation of coaching in other professions, like business and medicine. • Used by school districts nationwide and encouraged by the U.S. National Mathematics Advisory Panel. • There are a variety of educational coaching models coaches might follow.

5. Organizing question What are next steps in mathematics coaching research, professional development, or other collaborations between researchers and coaches?

6. Examining Mathematics Coaching EMC is a 5-year research and development project examining the effects of a coach’s knowledge for coaching on a diverse population of grades K-8 teachers *(K – 8 is students aged 5 – 13, in U.S. elementary and middle schools, or sometimes a K-8 school.)

7. Mathematics coach: EMC definition A mathematics coach is an on-site professional developer who enhances teacher quality through collaboration, focusing on research-based, reform-based, and standards-based instructional strategies and mathematics content that include the why, what, and how of teaching mathematics.

8. Coaching cycle for EMC Project There are three distinct parts to each coaching cycle designed to examine mathematics instruction. • Pre-Lesson Conference (~15 minutes) • Lesson Observation (entire class period) • Post-Lesson Conference (~30 minutes) Coaches conduct 8 cycles per year, with 4 focused on number and operations.*

9. Limitations * This is not a study of coaching as an intervention (although it does produce some results along those lines). Though we gave guidelines, school and district priorities superseded our research wishes. What this leaves is a study of coaching as it is enacted. Certainly this introduces limitations to our findings, but increases our inferential abilities.

10. EMC research hypothesis The effectiveness of a mathematics classroom coach is linked to several domains of knowledge. Coaching knowledge and mathematics content knowledge both contribute to a coach’s effectiveness as measured by positive impact on teacher practice, attitudes, and knowledge.

11. Knowledge domains Mathematics Content Knowledge Coaching Knowledge Knowledge of Student Learning Knowledge of Teacher Learning

12. Impacts of EMC study • Understanding of knowledge needed for effective mathematics coaching. • Understanding of what practices contribute to effective mathematics coaching. • Instruments to evaluate and monitor mathematics coaching

13. Research design • An observational design will answer: To what extent does a coach’s depth of content knowledge in coaching knowledge and mathematics content knowledge correlate to coaching effectiveness? • An experimental design randomly assigns coaches to one of two groups to answer: To what extent does professional development targeting these two knowledge domains improve coaching effectiveness? and To what extent are the effects of the targeted professional development explained by increases in knowledge?

14. Crossover design

15. EMC participants Montana Coaches: 19 Teachers: 54 N. Dakota Coaches: 3 Teachers: 8 Wisconsin Coaches: 4 Teachers: 11 Washington Coaches: 2 Teachers: 4 Idaho Coaches: 15 Teachers: 44 Colorado Coaches: 11 Teachers: 31 Nebraska Coaches: 2 Teachers: 6

16. Coaching Effectiveness Project variables and measures

17. Mathematics Content Knowledge Coaching Effectiveness Implementation of Coaching Model Coaching beliefs, knowledge, skills, and practice Project variables and measures

18. Teacher Variables Mathematics Content Knowledge Mathematics Content Knowledge Classroom practice Teacher anxiety, efficacy, engagement, and preparedness Coaching Effectiveness Implementation of Coaching Model Coaching emphasis Coaching beliefs, knowledge, skills, and practice Coaching impact Project variables and measures

19. Teacher Variables Mathematics Content Knowledge Mathematics Content Knowledge Classroom practice Teacher anxiety, efficacy, engagement, and preparedness Coaching Effectiveness Implementation of Coaching Model Coaching emphasis Coaching beliefs, knowledge, skills, and practice Coaching impact Project variables and measures Mathematical Knowledge for Teaching Measures Inside the Classroom Observation Protocol Coach Reflection and Impact Teacher Survey Teacher Needs Inventory Teacher Reflection and Impact Survey Coaching Knowledge Survey & Coaching Skills Inventory

20. Classroom practices: Observational study

21. Research question 1 • To what extent does a coach’s depth of knowledge in coaching knowledge and mathematics content knowledge influence coaching effectiveness? • Models examine how variation in these aspects of the coaches propagates into teachers’ measures. • Four years of data included in the analysis.

22. Summary of findings for RQ1 Improvements in coaches’ coaching knowledge scores and self-efficacy measure of coaching skills scores are related to increases in teachers’ mathematics knowledge coached teachers perform better on mathematics assessments As coaches learn more about coaching, as measured both by how much they align with what coaching authors recommend and by their self-reports of effectiveness,

23. Summary of findings for RQ1 Coaches with higher mathematics for teaching scores are associated with teachers with higher mathematics for teaching scores. We suspect that this is a relic of how teachers were chosen.

24. Research question 2 • To what extent does professional development targeting these two knowledge domains improve coaching effectiveness? • Control for coaching intensity and outside PD • Effects are examined on changes in teachers’ mathematics for teaching scores, teachers’ attitudes, and teachers’ practice • Hierarchical linear models • Four years of data

25. Summary of findings for RQ2 • No detected coach-level PD effects on teacher content knowledge or teacher attitude • Some evidence of PD effects on teacher practice Coaching Coaching Math

26. Summary of findings for RQ2 • For all models, there are changes over time (Observational results that “coaching works”) • Suggestive evidence that changes happened in the different groups at different times; follow-up analyses are being conducted

27. Research question 3 • To what extent are the effects of targeted professional development on coaching effectiveness explained by increases in coaching knowledge and mathematics content knowledge? • Analysis uses 51coaches randomly assigned to PD groups; 5 years of data • Analysis uses linear modeling, and controls for outside mathematics or coaching training

28. Summary of findings for RQ3 • No evidence for direct effects of professional development on coaches’ mathematics for teaching scores either in terms of differences in groups or differences in changes over time. • There is evidence of a change over time in mathematics for teaching scores of the coaches in the study, with the highest average score in the last year of the study. • There is evidence of a time effect and a PD effect on the mean scores of coaching knowledge (that is, increases in coaching knowledge that have an effect at the teacher level can be inferred to result from the PD).

29. Some thoughts about experiments • Here, we’re using the model of RCTS, which measure interventions, to measure effects of knowledge • Hope that this is one study to contribute to overall understanding – too complicated to expect a single study to determine causality • Ongoing struggle: what gets funded (experiments), but an appropriate next step is qualitative descriptions about what coaching entails in vivo.

30. What we learn from participants • Coaches want to learn how to have hard conversations with teachers about mathematics content — • And about student learning. • Coaches expend a lot of energy on resistant teachers. • Professional development in coaching knowledge is needed, and our model shows promise.

31. Organizing question What are next steps in mathematics coaching research, professional development, or other collaborations between researchers and coaches?

32. Thank you! Beth Burroughs burroughs@math.montana.edu www.math.montana.edu/~emc