Assessing Induction and Mentoring in Middle School Mathematics

1. Assessing Induction and Mentoring in Middle School Mathematics Thomas Smith Vanderbilt University

2. Motivation U.S. middle school students struggle in mathematics 66% below proficient on 8th grade NAEP in 2013; 27% below basic In TN, 72% below proficiency, 31% below basic TN teachers expected to fully implement Common Core Standards in Math in 2014-2015 Research suggests that current teachers are not well prepared to implement the forms of instruction intended in the Common Core Many teachers lack a deep understanding of math content and do not have sufficient knowledge about how students learn and how to teach particular topics

3. What are the biggest differences between what is expected in the Common Core compared to earlier standards?

4. What does a typical TCAP item look like? What do students need to know to answer this problem?

5. What might a test item based on Common Core look like? What do students need to know to answer this problem?

6. TCAP item What do students need to know to answer this problem?

7. Common core: What do students need to know to answer this problem?

8. One component of the Teacher Learning Continuum: Induction • Designed to help acclimate new teachers to their school and to the profession • Often begins with activities such as orientation seminars and workshops • Mentoring is the most common component • Need rigorous, empirical research to identify which features of induction programs make them effective in: • (1) increasing teachers’ math content and pedagogical content knowledge • (2) improving the quality of their instructional practices • (3) fostering subsequent student learning

9. Our Conjectures • Orienting induction activities to focus more on developing teachers subject-area, content knowledge and how students learn that content has the potential to serve as a pivotal policy mechanism for teacher learning • Making content an increased focus of new teacher induction could also help retain successful teachers, since a primary reason new teachers leave the profession is because they do not believe their instruction is helping students learn

10. Assessing Induction and Mentoring (AIM) Four-year longitudinal study of beginning middle school math teachers’ induction and mentoring experience Vanderbilt University University of Pennsylvania Funded by National Science Foundation Study of Natural Variation

11. Research Questions What forms of support do beginning teachers receive? To what extent do teachers’ induction experiences focus on mathematics content and how students learn that content? To what extent are teachers’ induction, mentoring, and professional development experiences associated with improved (a) mathematics knowledge for teaching, (b) quality of instruction, c) retention, and (d) student learning?

12. Participants 62 middle school mathematics teachers from 11 districts in 4 states Teachers invited to participate if serve as teacher of record for at least one 7th or 8th grade math class no prior experience as teacher of record Follow three cohorts of teachers 3 years of data from Cohort 1 (beginning in 2007-08) and Cohort 2 (beginning in 2008-09) 2 years of data from Cohort 3 (2009-10)

13. Data • Types • Teacher, mentor, and principal interviews • Surveys • Classroom observations (coded using Instructional Quality Assessment, IQA) • Math Knowledge for Teaching (MKT) assessment • Survey of Enacted Curriculum (SEC) • Collected at four points: • Winter of first year & Spring of first year • Spring of second year • Spring of third year

14. School District Characteristics

15. Mentoring Program Features

16. Beginning Teachers’ Backgrounds Mostly female (67%), white (85%), in 20s (73%) 71% had student teaching experience 59% had student teaching experience in a middle school math 29% obtained certification through an alternate route program Many of these had other careers prior to teaching Some not eligible for state funded induction program, although they had a mentor from certification program 22% began teaching having already earned a master’s degree

17. Subject area of degrees Most had bachelor’s or master’s degree in education not specific to math, although just over one-third of the sample had a degree either in math or math education. Math Math Education Education Other

18. About a quarter of the beginning 7th and 8th grade teachers had not had a math methods course • Only about a third had taken three or more higher level math classes

19. Beginning Teacher’s School Context Teachers tended to teach either 4 or 5 classes in their first year. First-year teachers’ class sizes ranged from 7 to 37, with an average of 23 per class. On average, teachers had about 2 different course preparations in their first year. On average, teachers in their first year spent about 20 hours per week teaching mathematics, and had approximately 1 hour per school day for planning.

21. Mentor match characteristics 25 percent of teachers did not have a formal mentor located primarily in the school where they taught. 18 percent did not have a mentor who had ever taught mathematics at the middle or high school level.

22. Time Engaged In both of the first two years, teachers spent more time interacting with informal mentors than with formal mentors. The amount of time teachers spent interacting with mentors decreased substantially between the first and second years, but remained relatively consistent between years two and three.

23. Content of Interactions (Formal)

24. Content of Interactions (Informal)

25. What do mentors and beginning teachers do together?

26. Pedagogical Content Knowledge Measured by the Mathematics Knowledge for Teaching (MKT) Assessment developed at the University of Michigan The MKT is designed to measure teachers’ knowledge about explaining terms and concepts to students, interpreting students’ statements and solutions, selecting and using appropriate representations of mathematical concepts in the classroom, and providing students with examples (Hill, Rowan & Ball, 2005). Current measures consist of multiple-choice prompts, achieve reliability of .70 or above, and can be used as a pre-/post-test to assess teachers' knowledge growth

27. Hill, H.C., Schilling, S.G., & Ball, D.L. (2004) Developing measures of teachers’ mathematics knowledge for teaching. Elementary School Journal 105, 11-30.

28. How does MKT change over time? On average, teachers in our study began their first year of teaching with below-average MKT scores. With the exception of a small decline between teachers’ second and third years, teachers’ MKT increased over time.

29. How did we measure instructional quality? Measured by Instructional Quality Assessment (IQA) developed at the University of Pittsburgh Focus on the rigor of lessons and activities Rigor of tasks Maintaining the rigor of tasks during implementation Quality of class discussions Teacher press Teacher/student linking Useful for quantifying instructional quality across classrooms

30. Academic Rigor: Task Potential and Implementation • The Potential of the Task rubric asks: “Did the task have potential to engage students in rigorous thinking about challenging content?” • The Implementation of the Task rubric asks: “At what level did the teacher guide students to engage with the task in implementation?” HIGH LOW

31. Beginning instructional quality • In the fall of their first year, teachers tended to choose lessons with low task potential (2), indicating an emphasis on procedures and single representations of concepts, with some (23%) providing tasks connected to underlying math concepts. • 90% had task implementation scores during their first semester that averaged to a score of 2—signifying an emphasis on procedures and single representations of concepts. Only 10% had task implementation scores of 3 or above--maintenance of a high-level task. • Most had low discussion scores (average = 1) indicating that students primarily provided brief responses rather than having more elaborate discussions that involved sharing different strategies or justifying solutions

32. Relationships between teacher background and school supports on Instructional Quality Teachers with degrees in math tended to introduce tasks of lower cognitive demand and implement those tasks in a more procedural way than those with education degrees only. No MKT relationship Teachers in schools using a curriculum deemed “exemplary” by the NSF (Borasi& Fonzi, 2002) tended to have higher potential and implementation scores Honors/Advanced classes tended to have higher IQA potential and implementation scores

33. Task Potential & Task Implementation

34. Discussion & Accountable Talk

36. Impact of supports on change in instructional quality No statistically significant relationship Hours spent on math content with mentor Hours of math PD Participation in Professional Community Marginal statistical significance Instructional leadership p<.10). a) lets staff members know what is expected of them, (b) is supportive and encouraging, (c) enforces school rules for student conduct, (d) recognizes staff members for a job well done, (e) provides time for teachers to meet and share ideas with one another, (f) deals effectively with pressures from outside the school, (g) encourages innovative instructional practices, and (h) backs me up when I need it.

37. Interview data Collaboration • Intense collaboration is often not deeply focused on mathematics or the teaching of mathematics. • pacing, classroom management, sharing resources Mentorship • Regular co-planning may be an especially strong form of mentoring that is related to improvement in instructional quality • Improvers tended to have access to the same mentor in the second and third year of teaching • Improvers tended to have mentors who engaged more often in observation and feedback focused on instruction, compared to the mentors of nonimprovers

38. More from interviews Principal support • The positive effect of principal support may operate primarily through the organizational climate established by the principal rather than the amount of direct support or press to improve math instruction • Assignment to mentor • Assignment to a strong team • Observation/feedback not focused on math • Professional Development • Content-focused professional development may be better positioned to foster instructional change when it provides opportunities for ongoing interaction linked to classroom implementation of the content

39. Summary and Implications • Hiring someone who majored in math doesn’t guarantee higher quality instruction • Current supports provided to beginning middle school math teachers insufficient to leverage change in instructional practices (rigor, discussion) • Observation and feedback, co-planning, and teacher collaboration are all potential sources of new teacher development • The “expertise” of who the beginning teacher works with matters. • Instructional context for beginning teachers is likely to get even more challenging when assessments based on Common Core State Standards come online

40. Organization and Effectiveness of Induction Programs for New Teachers, edited by Thomas M. Smith, Laura M. Desimone, and Andrew Porter. • http://nsse-chicago.org/ • Chapters by: Peter Youngs, Ted Britton, Betty Achinstein, Pam Grossman and Susanna Loeb, Sharon Feiman-Nemser, Barnett Berry, Marjorie E. Wechsler, Julie Luft, Steven Glazerman, Richard Ingersoll

41. Contact Information: Tom Smith thomas.smith@vanderbilt.edu Thank you!

43. Implications & Recommendations 1. Mathematics instruction we observed tended to be oriented around the application of mathematical procedures and algorithms and generally offered few opportunities for students to engage in discussions of mathematical ideas or to justify their solutions to the class, even though such opportunities are considered an important practice for mathematics instruction We suggest that districts recognize the role that mentoring and other supports for beginning teachers might have in fostering more rigorous math instruction and consider how mentoring and other supports might be coordinated with a districts math learning goals to enable this goal to be reached.

44. Implications & Recommendations 2. Most teachers experienced mentoring from multiple sources, formal and informal, particularly in the first year. A major difference between formal mentoring and mentoring provided informally by colleagues appears to be the opportunity for teachers to be observed by and receive feedback from a mentor. Informal mentors almost never had the opportunity to observe their colleagues. We suggest that districts consider including an expectation for observation of and feedback on beginning teachers’ instruction in formal mentoring programs, as this is one way in which these programs can support teachers in ways they are not able to be supported by informal colleagues. Mentors vision/instructional practices needs to be considered in selection process

45. Implications & Recommendations 3. Although the quality of mentoring that teachers in this study received varied, having time for teachers and mentors to meet during the school day was one factor associated with higher quality mentoring. We suggest that districts consider the extent to which mentors and beginning teachers have sufficient structured time for interaction, as well as any structures that might be put in place to provide common time for mentoring interactions to occur during the school day.

46. The Motivation of Teachers to Produce Human Capital and Conform to their Social Contexts Addressing the Mathematics-Specific Needs of Beginning Mathematics Teachers Subject-specific Induction Programs: Lessons from Science The State of Teacher Induction in Urban America Mentoring for Diversity and Equity: Focusing on Students of Color and New Teachers of Color Learning to Teach in New York City: How Teachers and Schools Jointly Determine the Implementation of a District-Wide Mentoring Program New Teacher Induction — In and Out of Cyberspace What the Research Tells Us about the Impact of Induction and Mentoring Programs for Beginning Teachers

47. Acknowledgements Laura Desimone & Thomas Smith, Co-Principal Investigators Andrew Porter Nida Arafat Mary Batiwalla Lydia Bentley Monica Bhatt Enakshi Bose Courtney Boswell Marisa Cannata Emily Carter Sebastian Cherng Beth Covay Andrea DiMol Alfred Dunn Tammy Eidson • Maida Finch • Christina Hart • Laura Hawkinson • Erin Henrick • Sara Heyburn • Eric Hochberg • Joy Johnson • Dee Jones • Jun Li • David Long • Dawn Lyken-Segosebe • Caitlin Matyas • Victoria McCardell • Kristin McGraner Jennifer McMaken John Murphy Laura Neergaard David Polett Morgan Polikoff Daniel Robbins Robert Schwartz Daniel Stuckey Joshua Taton Katherine Taylor Haynes Kerri Tobin Judi Vanderhaar Laura Vilines

48. Relationship Between Policy Context and Teachers’ Vision of High-Quality Math Instruction • Little evidence of alignment between visions of high quality instruction of beginning teachers, formal mentors, and principals • Principals emphasized real-world contexts to problems and students playing an active role in task completion and the importance of students being engaged • Mentors stated non-math related teacher actions or student outcomes as their vision • Most of the teachers reported ambiguous, conflicting, or no specific expectations for how to teach

49. Why might this be? • Principals’ insufficient knowledge of high quality math instruction • Observation and evaluation cycle does not give principals enough material to provide meaningful feedback • Selection of mentors not influenced by mentor’s vision of HQMI • High-stakes testing environment validates any instruction that leads to higher test scores

50. Turnover—rates are similar to national data Of the 64 teachers for whom we have turnover data: 49 teachers (77%) remained at the same school after their first year. 8 teachers moved to a different school 7 left teaching 39 teachers (61%) remained at their original school after their second year. 15 moved schools 10 left teaching Of the 50 teachers who began teaching in the 2007-2008 and 2008-2009 school years, just under half (24 teachers) were planning to still be teaching in their original school for a fourth year.