Research Findings about Elementary Mathematics Coaches and their Impact on Student Achievement and Teachers

1. Research Findings about Elementary Mathematics Coaches and their Impact on Student Achievement and Teachers Patricia F. Campbell University of Maryland College Park, MD 20742 patc@umd.edu This work was developed through the support of a grant from the National Science Foundation. Any opinions, findings and conclusions or recommendations expressed are those of the authors and do not necessarily reflect the views of the National Science Foundation.

2. Mathematics Specialist Former classroom teachers with strong backgrounds in math content, instructional strategies, and school leadership who are responsible for: • promoting enhanced mathematics instruction and student learning throughout their schools; • strengthening classroom teachers’ understanding of mathematics content; • helping teachers develop more effective mathematics teaching practices that allow all students to reach high standards; and • sharing research addressing how students learn mathematics. Virginia Mathematics and Science Coalition

3. Masters Program Number and Operations Geometry and Measurement Probability and Statistics Algebra and Functions Rational Numbers and Proportional Reasoning Education/Leadership I, II& III Implications of Mathematics Education Research Mathematics for Diverse Populations Externship (Masters Project)

4. Research Questions • What activities did coaches engage in and what proportion of their time did they spend completing those duties? • What is the effect of coaches on student achievement? • What is the effect of coaches on teachers’ beliefs about mathematics teaching and learning?

5. Data Sources • Coaches’ daily activity logs and weekly interaction logs recorded on a PDA • Coaches’ knowledge of mathematics content and knowledge of mathematics for teaching • Coaches’ and teachers’ beliefs surveys addressing mathematics teaching and learning • Coaches and teachers’ professional development surveys • Student achievement data from Virginia’s NCLB assessment (SOL)

6. Research Challenge: High Variability Within and Across Coaches’ Daily Activities • Develop menu-oriented software (a branching program) that operates on a PDA • Coaches provide a record of their daily activity

7. ISAM Time Entry • ISAM manager sets the contract day for each coach’s PDA. • The coach indicates the duration of the day’s first activity, setting up a listing of entries ordered by time.

8. ISAM Activity Menu • The coach selects the category of activity for that time period.

9. Branch to Detailed Choices • ISAM is based on a branching network. • Selection of activities of interest triggers the presentation of more detailed sub-choices.

10. Permits Self-Help • “Tap and hold” feature provides a more detailed statement of choices.

11. Built-in Reminders • “Tap and hold” feature provides a more detailed statement of choices.

12. Day’s End Review and Confirm/Save • At the end of the day, the coach reviews the data as entered, retrieves notes, makes needed changes in entries (not times), and then confirms (saves) the data.

13. Transmit • The coach transmits confirmed data over the Internet to a server.

14. Coaches Work Outside of Their Contract Day

15. Proportion of Cohort 1’s Contract-day Time

16. Proportion of Cohorts’ Contract-day Time

17. SOL Scales: Descriptive Statistics by Grade(Range: 200-600; ≥ 400 Passing; ≥ 500 Advanced)

18. High Engagement Values from ISAM Weekly Interaction Logs

19. Sample “Traditional” Beliefs Items • It is not very productive for students to work together during math time. • If students use calculators, they won’t master the basic math skills they need to know. • The best way to teach students to solve mathematics problems is to model how to solve one kind of problem at a time.

20. Sample “Making Sense” Beliefs Items • I often learn from my students during math time because my students come up with ingenious ways of solving problems that I have never thought of. • I don’t necessarily answer students’ math questions but rather let them puzzle things out for themselves. • Students can figure out how to solve many mathematics problems without being told what to do.

21. Conceptual Map: Hierarchical Analysis of Math Coaches’ Effects on Student Achievement Test Scores Level 3 -Schools: -Coach in School -Coach Experience -High Minority School Level 2- Teachers: -High Engagement with Coach -Beliefs -Traditional -Making Sense -Experience -High Minority Class Level 1-Students -Age -Gender -Special Education -SES (FARMs) -Minority Student SOL Total Scale Score and Subscale Scores

22. SOL Scale Score Regression Coefficients by Grade

23. SOL Scale Score Regression Coefficients by Grade

24. Coaches and Grade 3 Achievement • On average, students’ mean score on the SOL’s in Grade 3 was 492.8 points. • Just placing a coach in a school did not significantly effect the Grade 3 SOL scores. • But as coaches gained experience in their position, it did significantly influence Grade 3 SOL scores across all Grade 3 classrooms. • Grade 3 SOL scores went up on average 8.8 points during a coach’s second year in the school and 17.7 points during a coach’s third year in the school.

25. SOL Scale Score Regression Coefficients by Grade

26. Coaches and Grade 4 Achievement • On average, students’ mean score on the SOL’s in Grade 4 was 470 points. • Just placing a coach in a school did not significantly effect the Grade 4 SOL scores. • As coaches gained experience in their position, it did not significantly influence SOL scores across all Grade 4 classrooms. • But, Grade 4 students in classrooms with a teacher who had high engagement with a coach had SOL scores that were, on average, 11 points higher (a significant difference). • Grade 4 students in classrooms with a teachers who held a “making sense” perspective about mathematics teaching and learning had SOL scores that were, on average, 5.5 points higher (a significant difference).

27. SOL Scale Score Regression Coefficients by Grade

28. Coaches and Grade 5 Achievement • On average, students’ mean score on the SOL’s in Grade 5 was 494.9 points. • Just placing a coach in a school did not significantly effect the Grade 5 SOL scores. • But as coaches gained experience in their position, it did significantly influence Grade 5 SOL scores across all Grade 5 classrooms. • Grade 5 SOL scores went up on average 12 points during a coach’s second year in the school and 24 points during a coach’s third year in the school.

29. Teaching Experience Impacts Grade 5 SOL Achievement • On average, Grade 5 students in classrooms with teachers with only 1 or 2 years of experience scored 27 points lower on the SOL. • On average, Grade 5 minority students scored 28 points lower on the SOL. • On average, minority students in Grade 5 classrooms with teachers with only 1 or 2 years of experience score 73 (27 + 28 + 18) points lower on the SOL. • The negative impact of early career teachers on Grade 5 SOL achievement dissipates by teachers’ third year of placement.

30. Traditional Beliefs: Regression Coefficients *p <.05 ** p <.01 *** p <.001

31. Making Sense Beliefs: Regression Coefficients *p <.05 ** p <.01 *** p <.001

32. Findings • As coaches gained experience in their position, they had a significant positive effect on student math achievement in Grade 3 and Grade 5. • In Grade 4, this significant positive effect on student math achievement was limited to the students in classrooms of teachers who had a high level of engagement with the coach. • In Grade 4, teachers who agreed with a “making sense” perspective had significantly higher student math achievement. When coupled with high engagement with a coach, there were cumulative positive benefits significantly impacting their students’ achievement.

33. Findings (Continued) • In Grades 4 and 5, students in classrooms with teachers with ≥ 10 years of teaching experience had significantly higher math achievement. • In Grade 5, students in classrooms with teachers with only 1 or 2 years of teaching experience had significantly lower math achievement. • Teachers with ≤ 2 years of experience had an additional negative effect on minority students’ math achievement in Grade 5.

34. Findings (Continued) • As teachers gain experience, they steadily become less traditional in their beliefs about teaching and learning math. • Teachers in predominantly minority schools are overwhelmingly more traditional with respect to their beliefs about teaching and learning math. • The beliefs of teachers in schools with math coaches were significantly less traditional than those of teachers in control sites and this became more pronounced as the coaches gained experience.

35. Findings (Continued) • As teachers gain experience, they steadily become more likely to agree with a “making sense” perspective about teaching and learning math. • Teachers in predominantly minority schools are significantly less likely to agree with a “making sense” perspective about teaching and learning math. • The beliefs of teachers in schools with math coaches were no more in agreement with a “making sense” perspective than those of teachers in control schools, unless the teacher had high engagement with a coach.

36. Cautions • The coaches in this study engaged in a high degree of professional coursework addressing math content, pedagogy, and coaching prior to and during at least their first year of placement. Do not generalize these results to “anointed” coaches. • The significant positive effects of coaches on student achievement did not occur simply with the placement of a coach in a school. This impact emerged as the coaches gained experience -- as a knowledgeable coach and a school’s instructional and administrative staffs learned and worked together.

37. Cautions (Continued) • There is no evidence that the coaches in this study impacted the math achievement discrepancies frequently associated with race or poverty. • These coaches had differing responsibilities than that discussed for “math specialists” as “specialized teachers” in the National Mathematics Advisory Panel report.