Thomas Smith Laura Neergaard Vanderbilt University

1. Impact of Organizational Supports for Math Instruction on the Instructional Quality of Beginning Teachers Thomas Smith Laura Neergaard Vanderbilt University Eric Hochberg University of Pennsylvania

2. Project Details Four-year longitudinal study of beginning middle school math teachers’ induction and mentoring experiences Primary research questions: What is an appropriate framework and set of tools useful to study the quality of mathematics teacher induction programs and their effects on teacher content knowledge, instruction, and student learning? To what extent are teachers’ induction, mentoring, and professional development experiences associated with improved (a) knowledge for teaching mathematics, (b) alignment and quality of instruction, and (c) student learning?

3. Purpose of this Study Explore the relationships between beginning teachers educational background, organizational supports for further developing their math content knowledge and math knowledge for teaching, and how the quality of their instructional practices evolve over time. Do teachers with mathematics or mathematics education degrees implement higher quality lessons? Do teachers who have more math-focused support in their mentoring relationships improve the quality of their instructional practices? Do teachers who participate in more math focused PD improve the quality of their instructional practices? Do teachers who receive more support from their principals have greater improvement in the quality of their practices?

4. Rationale PD focused on specific content is associated with changes in teacher beliefs about instruction (Luft, Roehrig, & Patterson, 2003), self reported changes in instructional practice (Desimone et al, 2002; Garet et al, 2001), and improved student achievement (Cohen & Hill, 2000; Kennedy, 1998). Little is known about how the content of new teachers’ interactions with their mentors, principal, and other teachers in their school influences the content and quality of their instruction. Research is needed to identify whether the current forms of supports are effective in improving the quality of beginning teachers’ instruction.  

5. Participants & Data Sixty-two 7th and 8th grade mathematics teachers from 11 districts in 4 northeastern and southern states Mix of urban, suburban, and rural Mix of curricula (CMP2, Prentice Hall, Glencoe) Study uses 3 years of data from Cohort 1 (beginning in 2007-08), 2 years of data from Cohort 2 (beginning in 2008-09), and 1 year of data from Cohort 3 (2009-10) Surveys and classroom observations administered at four points: Winter of first year Spring of first year Spring of second year Spring of third year

6. Instructional Quality Assessment (IQA) Intended for use with math lessons that involve students in a problem-solving activity and subsequent whole-class discussion (Matsumura et al, 2006) Assesses the quality of observed classroom instruction on three dimensions: (1) task potential, (2) task implementation, (3) class discussion Four point scale for tasks potential and implementation Score of 3 or 4: instruction characterized by open-ended tasks, multiple representations of mathematical concepts, and connections among mathematical ideas Score of 2: instruction emphasizing unambiguous application of procedures and single representations of concepts Score of 1: instruction emphasizing facts and memorization Score of 0: absence of mathematical activity or discussion

7. IQA Coding At each time point, videotaped two lessons in the same math class on consecutive days Each lesson rated by two independent raters who had previously participated in at least two days of training in the IQA Ratings averaged across coders and the consecutive days to provide one set of IQA ratings per teacher for each of the four periods Multiple methods to assess the reliability of IQA coding One-point agreement at 88% Generalizability studies at three time points during the coding process supported decision to use scores from two raters across two observations

8. Supports for Math Instruction Teacher reported mentoring hours focused on math-specific instructional support (includes support from both formal and informal mentors) PD hours devoted to math-specific instructional support Math-specific instructional support may include emphasis on: Deepening teacher’s math content knowledge How students learn math (e.g., common misconceptions) Individualized instruction in math Limited variability across teachers regarding leadership support so not included

9. IQA Scores • Task potential & Implementation - emphasis on procedures and single representations of concepts • Not much change over time • Implementation averages lower than task selection • Discussion - students provided brief responses

10. Math Supports • Degrees • Math: 8% (n=5) • Math-education 24% (n=15) • Education 42% (n=26) • No math or education degrees 26% (n=16)

11. IQA Scores over Time by Degree Type Green = Education Red= Math Education Blue = Mathematics Purple = No Math or Education

12. Sensitivity Analyses Highest coders’ IQA scores For discussion, time is positive & significant (0.03) Education degree still positive for task Math degree still negative for implementation Ed degree still negative for discussion Coded math education degree holders as having education degrees rather than having math degrees Positive impact of education degree on initial task quality reduced (0.32 to 0.21) Negative impact of math degree on implementation quality no longer significant Substitute measure of math knowledge for teaching (MKT) scores for math degree Small positive relationship of MKT over time on implementation (0.01)

13. Task-Implementation Maintenance High-High: 3 or 4 on Task & 3 or 4 on Implementation High-Low: 3 or 4 on Task & 2 or lower on Implementation Low-Low: 2 or lower on Task & 2 or lower on Implementation • Most teachers in Low-Low in all four time periods • More teachers in High-High in 2nd & 3rd years

14. Summary and Conclusions Beginning teachers with education degrees tend to implement higher level tasks than teachers with only a math background Teacher with math degree have slightly higher scores on the discussion Little change over time in instructional quality, teachers degrading high level tasks appears to lessen No relationship between quantity of mathematics support from mentorship or PD and change in instructional quality Results indicate that organizational supports, as they are currently delivered, are not sufficient for increasing beginning teachers’ instructional quality growth.

15. Next steps • We are seeing a slight average increase over time in math knowledge for teaching, although this is not related amount of content focus in mentoring or PD • What is happening in these cases? • Why is MKT impacting task implementation the most? • Analysis of the interview data suggest that few mentors or teachers have very advanced visions of what high quality instruction is • To what extent is their greater press for HQMI in places where principals or mentors have more advanced visions • Does this result in improvements in instruction?

16. Contact Information: Tom Smith Vanderbilt University thomas.smith@vanderbilt.edu

17. References Cohen, D. K., & Hill, H. C. (2000). Instructional policy and classroom performance: The mathematics reform in California. Teachers College Record, 102, 294-343. Desimone, L., Porter, A.C., Garet, M., Suk Yoon, K., & Birman, B. (2002b). Effects of professional development on teachers’ instruction: Results from a three-year study. Educational Evaluation and Policy Analysis, 24(2), 81-112. Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. (2001). What makes professional development effective? Analysis of a national sample of teachers. American Education Research Journal, 38 (4), 915-945. Kennedy, M. M. (1998). Form and substance in in-service teacher education [Monograph 13]. Arlington, VA: National Science Foundation.

18. References • Luft, J. A., Roehrig, G. H., & Patterson, N. C. (2003). Contrasting landscapes: A comparison of the impact of different induction programs on beginning secondary science teachers’ practices, beliefs, and experiences. Journal of Research in Science Teaching, 40(1), 77-97. • Matsumura, L. C., Slater, S. C., Junker, B., Peterson,M., Boston, M., Steele, M., et al. (2006). Measuring reading comprehension and mathematics instruction in urban middle schools: A pilot study of the Instructional Quality Assessment. (CSE Technical Report 681). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).