Jill Lindsey, PhD Suzanne Franco, EdD Ted Zigler , EdD Wright State University, Dayton, OH

1. Jill Lindsey, PhD Suzanne Franco, EdD Ted Zigler, EdD Wright State University, Dayton, OH Research funded by the OERC Investigating Implementation of Ohio's Teacher and Principal Evaluation Systems Making Research Work for Education www.oerc.osu.edu

2. LEA Implementation of OTES &OPES • Sample LEAs • Implementation variations • SGM weights • Positive Feedback • Unintended Consequences • Next Step Questions 2

3. Sample of 12 LEAs Representing: • All RttT regions • RttT LEAs and one non-RttT LEA • Range of LEA typology (revised in 2013)

4. Key Findings • LEAs not fully implementing yet • Positive feedback & “In this together” approach • Time demands are unrealistic • Unintended negative consequences • Many misunderstandings and questions • Unfairness concerns impacting culture • Data experts, resources, and examples are needed

5. OTES Implementation • One LEA implementing OTES without SGM for all teachers in 2012-13 for performance pay • One LEA piloting OTES with SGM for all teachers in 2012-13 • Seven LEAs piloting OTES without SGM for a few teachers in 2012-13 • Three LEAs not piloting or implementing OTES in 2012-13

6. OPES Implementation • Five LEAs implementing the OPES performance rubric with all principals in 2012-13. • One LEAs implementing selected standards from the OPES performance rubric with all principals in 2012-13. • Six LEAs were not piloting or implementing OPES in 2012-2013.

7. Brief Summary of LEAs 7

8. Positive Feedback from LEAs • Goal setting process is useful. • Opportunities for discussions about growth are helpful. • OTES/OPES will help to focus professional development on target areas. • The rubrics well received; represents the work teachers and administrators do. • Inclusion of SGM component for both teachers and principals creates “we are in this together” approach. • Shared attribution reflects LEA philosophy

9. Unintended Consequences …. • Implementation takes away time with students • Not enough school days to observe every teacher • Low VAM can be demoralizing • Requires overemphasis on self and documentation • Culture of collaboration is undermined • Change negatively impacts successful schools

10. Misunderstandings & Questions • Value - added • Vendor assessments • SLOs - Student Learning Objectives • Shared Attribution (building VAM, district VAM, building performance index, and district performance index) 10

11. Variations in Weighting Decisions • Five of the twelve LEAs had SGM weights under consideration • Three of the five LEAs were using Shared Attribution • Six of the twelve LEAs were developing SLOs

12. STUDENT GROWTH MEASURES PLANS

13. Unfairness Concerns Negatively Impacting Culture • Validity of SGM questioned • Differences in required documentation • Problematic that SLO – VAM – Vendor Assessments are used as interchangeable growth measures • ODE look up table for final ratings weights SGM >50% • Labels and color-codes send negative messages • Tremendous stress; need time “to get things right”

14. Summary of Findings • LEAs not fully implementing but piloting • Positive feedback & “In this together” approach • Time demands are unrealistic • Unintended negative consequences • Many misunderstandings and questions • Unfairness concerns impacting culture • Need Data experts, resources, and examples

15. NEXT STEP QUESTIONS • What is the impact of linkage percentages on final ratings? • What is the impact of SGM weighting on final ratings? • Do OTES/OPES scales meaningfully distinguish between levels of performance? • Given budget constraints and length of school year, are the legislated evaluation requirements sustainable? • Do required teacher/principal evaluations improve student performance?

16. Contact information • Jill.Lindsey@wright.edu • Suzanne.Franco@wright.edu • Ted.Zigler@wright.edu