Portability of Teacher Effectiveness across School Settings

1. Bill & Melinda Gates Foundation Evaluation of the Intensive Partnership Sites initiative Portability of Teacher Effectiveness across School Settings Zeyu Xu, Umut Ozek, Matthew Corritore

2. Motivation › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Redistributing effective teachers at the center of several education policy initiatives • Teacher is the most important school input related to student learning • The distribution of effective teachers is uneven (recruiting, who moves, and to where) • Key assumption: Teachers effectiveness is portable • Students face different challenges in learning • School culture, environment and working conditions may affect teacher learning, practices, efforts, burnout, etc. • Literature • Jackson (2010), Jackson & Bruegmann (2009), Goldhaber & Hansen (2010) • Sanders, Wright & Langevin (2009)

3. Research Questions › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Do teachers retain their effectiveness across schools • On average • Across schools with similar settings • Across schools with different settings (by the direction of the change) • Teacher effectiveness measured by • Value-added • Settings defined by • School performance levels • School poverty levels • Conditional on teachers switching schools

4. Preview of Findings › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Among teachers who changed schools, on average their VA was unchanged or slightly improved • The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move • High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years • This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves • Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years

5. Organization › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Data and samples • Methodology • Findings • Summary and discussion

6. Data › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • North Carolina 1998-99 through 2008-09 • Elementary level (4th and 5th grade math and reading teachers, self-contained classrooms) • Secondary level (algebra I and English I teachers, “Algebra I”, “Algebra I-B”, “Integrated Math II”, “English I” classrooms) • Florida 2002-03 through 2008-09 • Elementary level (4th and 5th grade math and reading teachers, “core courses” in a given subject) • Secondary level (9th and 10th grade math and reading teachers, “core courses” in a given subject)

7. Sample restrictions › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Remove charter schools • Remove students and teachers who changed schools during a school year (about 2-4% of obs) • Remove students with missing values on covariates • Keep classrooms with 10~40 students • Remove classrooms with >50% special education students

8. Sample sizes › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Number of Unique Teachers in the Analytic Samples

9. Two-Stage Analysis › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Estimate teacher-year value-added • Difference-in-differences analysis

10. Estimate Teacher VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Test scores standardized by year, grade and subject (mean=0, sd=1) • (X) Covariates include: • 1) grade repetition, 2) FRPL, 3) sex, 4) race/ethnicity, 5) gifted, 6) special education, 7) student school mobility and 8) grade level. • Bias (no school FE) • Noise (EB adjustment) • Alternative model specifications (achievement levels model)

11. DiD › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Three groups: non-movers, movers to a similar school setting, movers to a different school setting • FGLS, se clustered at the teacher level • (Y) Year and (T) teacher FEs • (X) Teacher experience (0-2, 3-5, 6-12, 13 or more years of exp) • (S) School quality (average peer VA) • (C) Classroom characteristics (FRL %, mean pretest score, sd of pretest score) • (Post) Post-move years indicator • (DP, DN) Indicators for school setting differences

12. Define School Settings › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • School performance • NC: % students performing at or above grade level • FL: School performance scores based on both levels and growth • Standardized by year and aggregated across all years • School poverty • % FRPL • Aggregated across all years in which a teacher taught in that school • Change in school setting measures • ∆ = Receiving school – Sending school • Similar setting = within half a SD around the mean of the ∆ distribution • DP = 1 if ∆ > 0.25 (performance) or ∆ > 0.15 (poverty) • DN = 1 if ∆ < -0.25 (performance) or ∆ < -0.15 (poverty)

13. Alternative DiD Specs › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Last pre-move year and first post-move year • Between- vs. within-district moves • Replace the post-move indicator with individual year dummies (It-1, It-2, It-3…; It+1, It+2, It+3)

14. Distribution of Movers › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion By school performance setting change

15. Distribution of Movers › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion By school poverty setting change

16. Mover Characteristics › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion NC elementary school teachers, by mobility status

17. Pre-Post Change in VA (elem) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion

18. Pre-Post Change in VA (sec) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion

19. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Actual year of move “Pseudo” move

20. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Elementary math teachers Elementary math teachers (pseudo move)

21. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Elementary reading teachers Elementary reading teachers (pseudo move)

22. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Secondary math teachers Secondary math teachers (pseudo move)

23. By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Secondary reading teachers Secondary reading teachers (pseudo move)

24. Adjacent Year Correlations › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion

25. Pre-Post Comparisons of VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion North Carolina

26. Pre-Post Comparisons of VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion Florida

27. Summary › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Among teachers who changed schools, on average their VA was unchanged or slightly improved • The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move • High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years • This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves • Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years

28. Discussion › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion • Teacher effectiveness does not appear to be hurt by moving to schools with different settings. • Multiple years of VA estimates can be used with other teacher evaluation data to identify effective teachers, capturing persistent teacher performance better and reducing post-move year shrinkage. • All results take teacher school changes as given.