Teacher Evaluation: National Landscape & Learning To Date

Teacher Evaluation: National Landscape & Learning To Date TEACH NM – December 1, 2012 Amanda Kocon, Vice President (akocon@tntp.org) Berrick Abramson, Partner (babramson@tntp.org)

Agenda TNTP: An Introduction Evaluations: Background & National Landscape Evaluations: Best Practices & Components Appendix: Evaluation System Details by State & District

Even one year with a highly effective teacher has a lifelong impact on students. • Great teaching changes lives.

TNTP takes a comprehensive approach to ensuring that all students have access to great teachers.

TNTP focuses on four key areas to improve the effectiveness of the teacher workforce – the key to improving education for students. 4 Goals for Improving Teacher Effectiveness 2 PROFESSIONAL GROWTH Retained teachers improve over time Potential Teacher Performance Current TeacherPerformance EQUITABLE DISTRIBUTION More high-poverty students have effective teachers 3 NEW TEACHER EFFECTIVENESS More newly recruited teachers are effective, as measured by student growth 1 Less Effective More Effective 4 Persistently less effective teachers leave… … and most effective teachers stay SMART RETENTION

For 15 years, TNTP has worked alongside our partners in school districts and states nationwide, helping them reimagine teaching. “TNTP approached the work with an understanding of both the big picture… and of districts’ unique needs and challenges.” Louisiana Department of Education, 2012

At TNTP, we believe that teacher evaluation, observation and feedback is the cornerstone of teacher support and good human capital practice.

“ TNTP’s research made teacher evaluations a national priority, and is guiding district, states and charters as they build better evaluation systems. “[Our] industrial factory model of education treats all teachers like interchangeable widgets. [The Widget Effect] found that almost all teachers are rated the same. Who in their right mind really believes that?” - Secretary of Education Arne Duncan

When it comes to raising student achievement, nothing at school matters more than the quality of the teacher at the front of the class. 1 extra year of learning “The students of an ineffective teacher learn an average of half a year’s worth of material in one school year, while the students of a very good teacher learn 1.5 year’s worth—a difference of a year’s worth of learning in a single year.” (Hanushek, 2010) Very good teacher Ineffective teacher Gap-closing growth in 4 years “Having a top-quartile teacher rather than a bottom-quartile teacher four years in a row could be enough to close the black-white test score gap.” (Gordon, Kane and Staiger, 2006) White students Black students 10-student class size reduction Changing teacher performance from mediocre to very good (25th to 75th percentile) has an effect equivalent to reducing class size by 10+ students in 4th grade, 13+ students in 5th grade, or an “implausible” number in 6th grade. (Rivkin et al., 2005)

However, today’s evaluation systems do little to differentiate teachers. *Ranging from two to five years.

Most evaluation systems fail to provide teachers with the meaningful feedback they deserve as professionals. POOR EVALUATION SYSTEMS TNTP’S EVALUATION APPROACH Infrequent.Teachers can go years between evaluations. Unfocused. Student academic progress is rarely a factor. Undifferentiated.Nearly all teachers are rated good or great. Unhelpful.Teachers say evaluations don’t give them useful feedback. Inconsequential.Ratings rarely factor into employment decisions. The result: We treat teachers like interchangeable parts. Multiple Measures. Create robust picture of teacher performance. Learning Counts.Prioritize student academic growth. Meaningful Data. Best practices for value-add data from rigorous research. Student Surveys. Students know teacher effectiveness when they see it. Productive. Districts have actionable information for employment decisions. The result: Teachers and districts are provided with meaningful feedback.

More than 20 states have passed legislation to improve teacher evaluations.

Though measures and instruments vary, all statewide systems involve observations and multiple measures of student learning. *Indiana details represent the RISE evaluation system created by the Indiana Department of Education. **Not currently being piloted for the 2011-12 or 2012-13 school years but expected to be included in final system. ***New York details reflect state legislation. District systems are to be bargained locally.

When designing and implementing an evaluation system, these guiding principles can increase long-term success and improvement: • Observation and evaluation are related, but different: All teachers should be evaluated annually, which includes both development-oriented feedback (including student learning outcomes) and summative performance ratings. However, teachers will be observed with varying degrees of frequency during the year, depending on their individual needs. • Regular check-ins: The evaluation cycle should be grounded in regular, substantive conversations between teacher and evaluator that act as the cornerstone of the evaluation and development processes. However, processes should not be so regulated that they are cumbersome or prevent necessary action. • Regular feedback: Educators need to receive actionable feedback and follow-up observation(s) conducted to assess progress in the identified development areas. Educators also need an opportunity to receive feedback on their planning practices. • Evaluation and development go hand in hand: The evaluation process should enable individualized development for teachers that is aligned to student learning goals and tied to evaluation results. The observation process needs to feel supportive, not punitive. • Clear expectations: Teacher evaluation and development must be built around a clear framework of expectations, focused on an evidence-based assessment of student learning and teacher competencies.

The length and frequency of observations vary among statewide systems, with most involving multiple classroom visits.

Measures of Student Learning encompass multiple measures. Student growth measures: A measurement of a teacher’s or school’s impact on student learning. • Measured using value-added or growth model • Can include individual teacher growth data and school-wide growth data Measures for Non-tested grades and subjects: Goals of student achievement used in the absence of, or in addition to, student growth measures. This measure, commonly referred to as Student Learning Objectives (SLOs), involves: • Identifying or creating assessments for measurement • Accounting for student starting points when possible • Setting a student learning goal around the identified assessment

Value-added modeling is a method to examine and isolate individual teacher impact on student learning outcomes. • How does it do it? • For student growth measures, it compares students to others with similar past academic performance, native language, disability, poverty, and more in order to control for factors outside of the teachers’ control. • There are various types of value-added models used which differ by the number of years of data they use, how they adjust for missing data, the things they control for, etc.

Student surveys can provide valuable information because they capture aspects of teaching that other measures cannot. While well-trained observers can identify meaningful information about instruction, students have more first-hand experience with a teacher in a particular classroom. Surveys provide teachers with an opportunity to get direct, experiential feedback from the students they teach every day. A teacher’s ability to create a positive, safe, supportive climate is just as important as the teacher’s ability to generate academic growth. Student surveys measure the social and emotional dimensions of teaching. Our definition of a teacher’s job is incomplete if we don’t ask what it feels like to be in that classroom Student surveys measure what it feels like to be a student in a given classroom, plain and simple.

SLOs are used in many sites as a student growth measure for non-tested grades & subjects. Systems can, and should be, designed mindfully for improvement over time. What is a Student Learning Objective (SLO) Process? Teachers and leaders set goals for their students' learning based on where the students enter. Throughout the year, evaluators assess students’ progress toward goals, and assign ratings at the end of course. • Where possible, keep weighting moderate (10 to 35%) in initial years and increase as skills, tools and systems improve. • This can be accomplished through direct system weights, or by adding in school-wide or other measures in a “50% student learning” system • Begin with more discretion in scoring • e.g. “ OR …… exceptional growth, typically representing more than 1.5 year of learning .…little growth, typically representing less than a years’ worth of learning ” • Focus on building skills among district academic managers (Assistant Superintendents, Chief Area Officers) and school administrators

A number of states and districts are moving from piloting SLOs to attaching stakes. In addition to the weighting of SLOs, their use in tested vs. non-tested grades and subjects vary.

We have identified 4 key steps to developing and selecting SLOs and challenges associated with each step that require a thoughtful approach to their use. • Selection of what learning will be measured Often from among a large and sometimes confusing array of state learning standards, and often in the midst of a transition to Common Core standards • Understanding student starting points Often where no good data exists • Setting a goal for where all students should end the year Often without good guidance on where this goal should be • Assessing whether the goal is attained Often where an assessment must be created

As stakes increase, so does the need for quality control. Weight : Influence on final rating Consequences: Results for teachers Accuracy: Accuracy and precision in measurement Fairness: Comparability across classrooms

As the weighting of SLOs and attached stakes increase, it is important to have an established continuum of quality control measures.

Our confidence in SLOs and MOSL increases or decreases based on the level at which the measure is created. State Exam with Growth Model (GM) The assessment “triangle” State Exam (state-created or off-the-shelf) without GM National Exams (PARCC – 2014/2015) Confidence 1 District Off-the-shelf Exam (i.e. NWEA, Scantron) District-Created Exam 2 School/Department-Created Exam 3 Teacher-Created Exam

Below is a list of promising evaluation implementation practices that lead to an effective teacher evaluation system.

Lessons learned from implementation of an evaluation system in a large, urban school district. • Develop specific, common goals for everyone involved along with clearly articulated systems for accountability and support. • Example: evaluators, and their managers, have a goal for the accuracy of their ratings (based on student learning data), which has been clearly articulated to them, along with how progress will be monitored and what supports they can expect. • Communicate, communicate, communicate! • Ensure that stakeholders (principals, teachers, parents, community, etc.) are informed along the way, that they know what the goals are and what to expect. • Prioritize face-to-face support and trainings. • Give teachers and principals as many options as possible to talk to a human, not read a document. • Rely on data to make decisions. • Implementation will not always go as planned, but it is important to focus on available data rather than reacting to isolated issues. • When you plan to scale, a good technology solution is critical and should be planned, designed and thoroughly tested well in advance of implementation. • Teachers and principals deem the system a success or failure based on how they most often interact with it.

Agenda TNTP: An Introduction Evaluations: The National Landscape Evaluations: An Overview of Components Appendix: Evaluation System Details by State & District

*System details from the state–created RISE evaluation system. Districts can adopt RISE, modify RISE, or create their own system.

*Not currently being piloted for the 2011-12 or 2012-13 school years but expected to be included in final system.

*State legislation. Final system must be locally bargained with the union.

A few resources: TNTP: Teacher Talent Toolbox http://tntp.org/teacher-talent-toolbox/view/evaluation Indiana: www.riseindiana.org Washington DC: www.dc.gov/DCPS/impact Tennessee: http://team-tn.org/ Rhode Island: http://www.ride.ri.gov/educatorquality/educatorevaluation/ Delaware: http://www.doe.k12.de.us/csa/dpasii/default.shtml New York: http://usny.nysed.gov/rttt/teachers-leaders/#appr

Teacher Evaluation: National Landscape & Learning To Date