Preparing, Evaluating, Developing and Retaining Teachers and Principals: An Overview of Research and Policy

1. Preparing, Evaluating, Developing and Retaining Teachers and Principals: An Overview of Research and Policy Laura Goe, Ph.D. Research Scientist, ETS, and Principal Investigator for the National Comprehensive Center for Teacher Quality Kansas/Missouri Superintendents Forum Thursday, March 1, 2012  Kansas City, MO

2. Laura Goe, Ph.D. • Former teacher in rural & urban schools • Special education (7th & 8th grade, Tunica, MS) • Language arts (7th grade, Memphis, TN) • Graduate of UC Berkeley’s Policy, Organizations, Measurement & Evaluation doctoral program • Principal Investigator for the National Comprehensive Center for Teacher Quality • Research Scientist in the Performance Research Group at ETS

3. The National Comprehensive Center for Teacher Quality • A federally-funded partnership whose mission is to help states carry out the teacher quality mandates of ESEA • Vanderbilt University • Learning Point Associates, an affiliate of American Institutes for Research • Educational Testing Service

4. ETS’ role in the TQ Center • ETS’ role in the National Comprehensive Center for Teacher Quality is to • Find, synthesize, and disseminate the best scholarly research on teacher quality, teacher effectiveness, and equitable distribution • Provide technical assistance to states on teacher quality issues, meeting the highly qualified teacher requirements, and equitable distribution 4

5. Today’s presentation available online • To download a copy of this presentation or look at it on your iPad, smart phone or laptop, go to www.lauragoe.com • Go to Publications and Presentations page • Today’s presentation is at the bottom of the page

6. The goal of teacher evaluation • The ultimate goal of all teacher evaluation should be… • TO IMPROVE TEACHING AND LEARNING

7. Topics to be addressed • Evaluating teachers, principals, and superintendents • Evaluating teachers using multiple measures • Models for determining of teachers’ contributions to student learning • Linking evaluation results to teacher professional growth plans • Teacher compensation and pay for performance • The equitable distribution of teachers • Preparing teachers for success

8. An aligned teacher evaluation system: Part I

9. An aligned teacher evaluation system: Part II

10. A simple definition of teacher effectiveness • Anderson (1991) stated that “… an effective teacher is one who quite consistently achieves goals which either directly or indirectly focus on the learning of • their students” (p. 18).

11. Race to the Top definition of effective & highly effective teacher Effective teacher: students achieve acceptable rates (e.g., at least one grade level in an academic year) of student growth (as defined in this notice). States, LEAs, or schools must include multiple measures, provided that teacher effectiveness is evaluated, in significant part, by student growth (as defined in this notice). Supplemental measures may include, for example, multiple observation-based assessments of teacher performance. (pg 7) Highly effective teacher students achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth (as defined in this notice).

12. Race to the Top definition of student growth • Student growth means the change in student achievement (as defined in this notice) for an individual student between two or more points in time. A State may also include other measures that are rigorous and comparable across classrooms. (pg 11) 12

13. Goe, Bell, & Little (2008) definition of teacher effectiveness • Have high expectations for all students and help students learn, as measured by value-added or alternative measures. • Contribute to positive academic, attitudinal, and social outcomes for students, such as regular attendance, on-time promotion to the next grade, on-time graduation, self-efficacy, and cooperative behavior. • Use diverse resources to plan and structure engaging learning opportunities; monitor student progress formatively, adapting instruction as needed; and evaluate learning using multiple sources of evidence. • Contribute to the development of classrooms and schools that value diversity and civic-mindedness. • Collaborate with other teachers, administrators, parents, and education professionals to ensure student success, particularly the success of students with special needs and those at high risk for failure.

14. Principal Effectiveness: New Leaders for New Schools Definition • “New Leaders for New Schools advocates for an evidence-based, three-pronged approach to defining principal effectiveness: 1) gains in student achievement, 2) increasing teacher effectiveness, and 3) taking effective leadership actions to reach these outcomes.” • http://www.newleaders.org/wp-content/uploads/2011/08/principal_effectiveness_nlns_overview.pdf

15. Principal Effectiveness: Center for American Progress on Principal Evaluation • Student achievement measures including schoolwide academic growth, attainment measures of achievement, and cohort graduation rates • Recruiting, developing, and retaining effective teachers and effectively implementing teacher evaluations to improve teacher effectiveness and/or retain effective teachers at higher rates while reducing the number of ineffective performers • Research-based rubrics that assess principals against performance standards • Measures of school culture and climate, such as teacher and student attendance, indicators of school discipline, and parent, student, and staff perceptions Summarized from http://www.americanprogress.org/issues/2011/03/pdf/principalproposal-memo.pdf

16. Principal Evaluation: Interstate School Leaders Licensure Consortium (ISLLC) Standards Standard 1: A school administrator is an educational leader who promotes the success of all students by facilitating the development, articulation, implementation, and stewardship of a vision of learning that is shared and supported by the school community. Standards 2: A school administrator is an educational leader who promotes the success of all students by advocating, nurturing, and sustaining a school culture and instructional program conducive to student learning and staff professional growth. Standard 3: A school administrator is an educational leader who promotes the success of all students by ensuring management of the organization, operations, and resources for a safe, efficient, and effective learning environment.

17. Principal Evaluation: Interstate School Leaders Licensure Consortium (ISSLC) Standards (cont’d) Standard 4: A school administrator is an educational leader who promotes the success of all students by collaborating with families and community members, responding to diverse community interests and needs, and mobilizing community resources. Standard 5: A school administrator is an educational leader who promotes the success of all students by acting with integrity, fairness, and in an ethical manner. Standard 6: A school administrator is an educational leader who promotes the success of all students by understanding, responding to, and influencing the larger political, social, economic, legal, and cultural context.

18. Vanderbilt Assessment of Leadership in Education (VAL-Ed)

19. Measures and models: Definitions • Measures are the instruments, assessments, protocols, rubrics, and tools that are used in determining teacher effectiveness • Models are the state or district systems of teacher evaluation including all of the inputs and decision points (measures, instruments, processes, training, and scoring, etc.) that result in determinations about individual teachers’ effectiveness

20. Multiple measures of teacher effectiveness • Evidence of growth in student learning and competency • Standardized tests, pre/post tests in untested subjects • Student performance (art, music, etc.) • Curriculum-based tests given in a standardized manner • Classroom-based tests such as DIBELS • Evidence of instructional quality • Classroom observations • Lesson plans, assignments, and student work • Student surveys such as Harvard’s Tripod • Electronic portfolios/evidence binders • Evidence of professional responsibility • Administrator/supervisor reports, parent surveys • Teacher reflection and self-reports, records of contributions

21. Value-added models Many variations on value-added models TVAAS (Sander’s original model) typically uses 3+ years of prior test scores to predict the next score for a student Used since the 1990’s for teachers in Tennessee, but not for high-stakes evaluation purposes Most states and districts that currently use VAMs use the Sanders’ model, also called EVAAS There are other models that use less student data to make predictions Considerable variation in “controls” used 21

22. Achievement Proficient Teacher A: “Success” on Ach. Levels Teacher B: “Failure” on Ach. Levels Start of School Year End of Year Growth vs. Proficiency Models In terms of growth, Teachers A and B areperforming equally Slide courtesy of Doug Harris, Ph.D, University of Wisconsin-Madison

23. Achievement Proficient Teacher A Teacher B Start of School Year End of Year Growth vs. Proficiency Models (2) A teacher with low-proficiency students can still be high in terms of GROWTH (and vice versa) Slide courtesy of Doug Harris, Ph.D, University of Wisconsin-Madison

24. Most popular growth models: Colorado Growth Model • Colorado Growth model • Focuses on “growth to proficiency” • Measures students against “academic peers” • Also called criterion‐referenced growth‐to‐standard models • The student growth percentile is “descriptive” whereas value-added seeks to determine the contribution of a school or teacher to student achievement (Betebenner 2008)

25. Colorado Growth Model Slide courtesy of Damian Betebenner at www.nciea.org

26. What value-added and growth models cannot tell you • Value-added and growth models are really measuring classroom, not teacher, effects • Value-added models can’t tell you why a particular teacher’s students are scoring higher than expected • Maybe the teacher is focusing instruction narrowly on test content • Or maybe the teacher is offering a rich, engaging curriculum that fosters deep student learning. • How the teacher is achieving results matters!

27. Measuring teachers’ contributions to student learning growth: A summary of current models

28. Recommendation from NBPTS Task Force (Linn et al., 2011) • Recommendation 2: Employ measures of student learning explicitly aligned with the elements of curriculum for which the teachers are responsible. This recommendation emphasizes the importance of ensuring that teachers are evaluated for what they are teaching.

29. School-wide VAM illustration

30. Differentiating among teachers • “It is nearly impossible to discover and act on performance differences among teachers when documented records show them all to be the same.” (Glazerman et al., 2011, pg 1)

31. What assessments are teachers and schools going to use? • Existing measures • Curriculum-based assessments (come with packaged curriculum) • Classroom-based individual testing (DRA, DIBELS) • Formative assessments such as NWEA • Progress monitoring tools (for Response to Intervention) • National tests, certifications tests • Rigorous new measures (may be teacher created) • The 4 Ps: Portfolios/products/performance/projects • School-wide or team-based growth • Pro-rated scores in co-teaching situations • Student learning objectives • Any measure that demonstrates students’ growth towards proficiency in appropriate standards

32. The 4 Ps (Projects, Performances, Products, Portfolios) • Yes, they can be used to demonstrate teachers’ contributions to student learning growth • Here’s the basic approach • Use a high-quality rubric to judge initial knowledge and skills required for mastery of the standard(s) • Use the same rubric to judge knowledge and skills at the end of a specific time period (unit, grading period, semester, year, etc.)

33. Assessing Musical Behaviors: The type of assessment must match the knowledge or skill 4 types of musical behaviors: Types of assessment Slide used with permission of authors Carla Maltas, Ph.D. and Steve Williams, M.Ed. See reference list for details. Responding Creating Performing Listening Rubrics Playing tests Written tests Practice sheets Teacher Observation Portfolios Peer and Self-Assessment

34. Tennessee approved assessments for non-tested subjects & grades

35. Rhode Island’s SLO language • “Student Learning Objectives are not set by educators in isolation; rather, they are developed by teams of administrators, grade-level teams or groups of content-alike teachers and, are aligned to district and school priorities, wherever possible.” (pg 12) From Rhode Island’s “Guide to Measures of Student Learning for Administrators and Teachers 2011-2012” http://www.ride.ri.gov/educatorquality/educatorevaluation/Docs/GuideSLO.pdf

36. Collect evidence in a standardized way (to the extent possible) • Evidence of student learning growth • Locate or develop rubrics with explicit instructions and clear indicators of proficiency for each level of the rubric • Establish time for teachers to collectively examine student work and come to a consensus on performance at each level • Identify “anchor” papers or examples • Provide training for teachers to determine how and when assessments should be given, and how to record results in specific formats

37. Tripod Survey (1) • Harvard’s Tripod Survey – the 7 C’s • Caring about students (nurturing productive relationships); • Controlling behavior (promoting cooperation and peer support); • Clarifying ideas and lessons (making success seem feasible); • Challenging students to work hard and think hard (pressing for effort and rigor); • Captivating students (making learning interesting and relevant); • Conferring (eliciting students’ feedback and respecting their ideas); • Consolidating (connecting and integrating ideas to support learning)

38. Tripod Survey (2) • Improved student performance depends on strengthening three legs of teaching practice: content, pedagogy, and relationships • There are multiple versions: k-2, 3-5, 6-12 • Measures: • student engagement • school climate • home learning conditions • teaching effectiveness • youth culture • family demographics • Takes 20-30 min • There are English and Spanish versions • Comes in paper form or in online version

39. Tripod Survey (3) • Control is the strongest correlate of value added gains • However, it is important to keep in mind that a good teacher achieves control by being good on the other dimensions

40. Tripod Survey (4) • Different combinations of the 7 C's predict different outcomes (student learning is one outcome) • Using the data, you can determine what a teacher needs to focus on to improve important outcomes • Besides student learning, other important outcomes include: • happiness • good behavior • healthy responses to social pressures • self-consciousness • engagement/effort • satisfaction

41. Teacher and principal evaluation in isolated and/or low-capacity districts • External evaluators may need to be brought in for very small, isolated districts • For example, a district where the superintendent is also principal, history teacher, and bus driver • May also be needed when evaluators’ objectivity is impacted by factors such as fear of losing teachers or damaging long-term relationships in the community • Evaluators could be “exchanged” across districts within a specific region (“you evaluate mine, and I’ll evaluate yours”)or regional evaluators could serve a set of districts

42. Measures that help teachers grow • Measures which include protocols and processes that teachers can examine and comprehend • Measures that are directly and explicitly aligned with teaching standards • Measures that motivate teachers to examine their own practice against specific standards • Measures that allow teachers to participate in or co-construct the evaluation (such as portfolios) • Measures that give teachers opportunities to discuss the results for formative purposes with evaluators, administrators, teacher learning communities, mentors, coaches, etc. • Measures that are aligned with and used to inform professional growth and development offerings

43. Interpreting results for alignment with teacher professional learning options • Different approach; not looking at “absolute gains” • Requires ability to determine and/or link student outcomes to what likely happened instructionally • Requires ability to “diagnose” instruction and recommend/and or provide appropriate professional growth opportunities • Individual coaching/feedback on instruction • Observing “master teachers” • Group professional development (when several teachers have similar needs)

44. Memphis professional development system: An aligned system • Teaching and Learning Academy began April ‘96 • Nationally commended program intended to • “…provide a collegial place for teachers, teacher leaders and administrators to meet, study, and discuss application and implementation of learning…to impact student growth and development” • Practitioners propose and develop courses • Responsive to school/district evaluation results • Offerings must be aligned with NSDC standards • ~300+ On-line and in-person courses, many topics

45. Considerations for implementing measurement system • Consider whether human resources and capacity are sufficient to ensure fidelity of implementation • Poor implementation threatens validity of results • Establish a plan to evaluate measures to determine if they can effectively differentiate among teacher performance • Need to identify potential “widget effects” in measures • If measure is not differentiating among teachers, may be faulty training or poor implementation, not the measure itself • Examine correlations among results from measures • Evaluate processes and data each year and make needed adjustments

46. Before you implement teacher and principal evaluation systems, ask yourself… • How will this component of the teacher and principal evaluation system impact teaching and learning in classrooms and schools? • How will this component look different in low-capacity vs. high-capacity schools? • How will reporting on this component be done (to provide actionable information to teachers, principals, schools, districts, teacher preparation programs, and the state)? • How will we know if this component is working as we intended?

47. Evaluating Teacher Preparation Programs (TPPs)

48. Meeting the “standards” • It’s possible to be meeting accreditation standards (NCATE, TEAC) but still not be preparing fully effective teachers • If TPPs are not adequately preparing teachers for the contexts and communities which they serve, their effectiveness may be hampered

49. VAMs and Teacher Prep Program evaluation/assistance • VAMs may be useful in identifying teacher preparation programs (TPPs) whose graduates are not performing at acceptable levels in terms of student gains • However, VAMs cannot be used to diagnose why the TPP’s graduates are failing to meet student progress goals • Additional information should be gathered from the TPP in order to properly diagnose problems • TPPs can then be provided with guidance and support to address specific needs

50. Questions about measuring TPPs through student achievement • How much of a student’s achievement growth is attributable to what a teacher does? • How much of what a teacher does is attributable to what he or she learned in a teacher prep program? • How do you sort out other influences on teacher practice and performance, such as mentoring, school culture, professional development, independent study, and peer support?