The Use of Student Assessments in Teacher Evaluations Through the Next Decade

1. The Use of Student Assessments in Teacher Evaluations Through the Next Decade • National Conference on Student Assessment • June 27, 2012 • Minneapolis

2. Agenda • Introduction: Julie Kochanek • Study profile: Julie Riordan • Performance-based teacher evaluation systems in five states • Study profile: Karen Shakman • Changing culture and building capacity: An exploration of district strategies for implementation of teacher evaluation systems • District profile: David Heistad • The use of value-added in Minneapolis public schools • Q & A discussion: Julie Kochanek

3. An Examination of Performance-based Teacher Evaluation Systems Karen Shakman Julie Riordan Maria-Teresa Sanchez Kyle DeMeo Cook Richard Fournier Jessica Brett

4. Rationale for Study • Regional need • Recent research • Race to the Top • State policy context

5. Selection Criteria • Required for all practicing general educators • Operational statewide in 2010–11 school year • Multiple rating categories • Multiple measures of teacher effectiveness

6. Research Questions • What are the key characteristics of state-level performance-based teacher evaluation systems in the study states? • How do state teacher evaluation measures, the teaching standards the evaluations are designed to measure, and rating categories differ across states that have implemented statewide systems?

7. Approach • Examined 50 SEA websites • Conducted additional online search • Reviewed evaluation guides/manuals, rubrics, training materials, regulations, program reports • Contacted state directors

8. States Included in Study • Delaware (RTT Phase 1) • Georgia (RTT Phase 2) • North Carolina (RTT Phase 2) • Tennessee (RTT Phase 1) • Texas (did not apply)

9. Evaluation Measures • Classroom observations • Principal evaluations • Analysis of classroom artifacts • Analysis of teaching portfolios • Teacher self-reports of practice • Student ratings of teacher performance • Value-added (student growth) strategies (Goe, Bell, and Little, 2008)

10. Findings: RQ1 – Key Characteristics

11. Findings: RQ 2 – Teaching Standards

12. Findings: RQ 2- Rating Categories

13. Future Directions • Further investigation about specific nature of the measures in place, including how student data are used in the overall assessment of teachers • Identify the types of knowledge and skills that are commonly observed and evaluated in performance-based systems • Investigate the fidelity of implementation of these systems

14. Changing Culture and Building Capacity: An Exploration of District Strategies for Implementation of Teacher Evaluation Systems Karen Shakman Nicole Breslow Julie Kochanek Julie Riordan Tom Haeford

15. Context • National attention on teacher evaluation • Focus on districts • Systems change as framework

16. Research Questions • What challenges are districts facing as they design and implement new teacher evaluation systems? • What strategies have districts developed to address these challenges?

17. Approach • Conducted semi-structured interviews with 16 district representatives • Employed an emergent code strategy to transcripts and interview notes to identify key themes

18. Findings: 2 Major Challenges • Changing district and school level culture • Building district and school level capacity

19. Findings: Strategies for Changing Culture • Focus on effective teaching • Engage stakeholders in the design process • Involve the teachers union • Align professional development • Establish differentiated and direct communication strategies

20. Findings: Strategies for Building Capacity • Invest in training evaluators and teachers • Establish district-level, cross-functional teams • Identify new roles and fill gaps in expertise • Sequence implementation strategies

21. Remaining Challenges • Redefined role for principals • Use of student achievement measures • Lack of alignment between local and state level reform efforts

22. Future Directions • Research • Implementation studies • Measurement studies • Impact studies • Building a community of practice

24. History of Value-Added Use in Minneapolis Public Schools • Evaluation of the Public School Academy (1992-94) Longitudinal Achievement Effects – Rob Meyer and U of W • Teachers who Beat the Odds in 2nd Grade Reading (1997-99) • Quality Performance Awards (1995–2005) Multiple Measures with School Awards • Teacher Advancement Program (TAP) School, Grade level and individual teacher bonuses (2006-2010) • Teachers who Beat the Odds in Kindergarten Literacy (2006-2010) • Grade level value-added to all K-8 Principals (2010-11) – Bush Foundation and VARC • Classroom level value-added to all K-8 Teachers (2012-13) – VARC and Dr. Chris Moore

25. Predictive Validity of the Minneapolis Public Schools Beginning of Kindergarten Assessment (BKA)

26. Value-added Kindergarten Teacher Effects “beat the odds teachers” • Post-test reading score= • Pretest literacy score • + Free or reduced price lunch • + Racial/ethnic code (4 dummy codes) • + Gender • + English Language Learner status • + Special Education status • + Lives with single parent • + Teacher effects

27. Follow-up with teachers who beat the odds • Surveys, observations, video tape • These teachers can be leads in staff development, mentors, etc. • What about inviting these teachers to work in schools with the high concentrations of poverty and students of color? • Videos at: http://rea.mpls.k12.mn.us/uploads/teachers_who_beat_the_odds.pdf

28. Current Work with the Bush Foundation and Value-added Research Center (VARC) • Teacher level value-added rolled out K-8 in 2012-13 (Rubric Scores and Value-added) • Graduates from 14 institutions tracked into MPS system for feedback to first year teachers and the Institutions of Higher Education (IHEs) • Studies of the human capital teacher pipeline Presentation Title Appears Here

29. For more information…. • jkochanek@edc.org • jriordan@edc.org • kshakman@edc.org • dheistad@mpls.k12.mn.us Presentation Title Appears Here