Fairfax County Public Schools Teacher Effectiveness Overview September 12, 2011

1. Fairfax County Public Schools Teacher Effectiveness Overview September 12, 2011

2. Agenda Introduction What is Teacher Effectiveness? Evaluation at the Core Rethinking the System

3. Today’s Briefing – Why are we here? • To become more knowledgeable about issues relating to teacher effectiveness • To receive an overview of the National trends • Virginia Board of Education Guidelines for Uniform Performance Standards and Evaluation Criteria for Teachers, effective July 1, 2012. • To receive an overview of the process FCPS will employ to revise Teacher Performance Evaluation

4. Our Mission The District Management Council (DMC) is a membership network that provides superior strategic insights and practical solutions to the most pressing challenges facing school system leaders. DMC delivers high quality, in-depth techniques, tools and training required to successfully raise student achievement while improving operations and lowering costs. Student Achievement Financial Savings Operational Efficiency DMC Focus

5. DMC Services Overview DMC’s deep and growing library of district case studies, best practices, and rigorous analytical frameworks is designed to help districts put best practices into action and achieve results. District Management Journal District Case Studies District Management Briefs Ten Mistakes to Avoid DMC offers a variety of conferences and institutes aimed at superintendents and cabinet-level executives to stay current on an area of expertise - or learn strategies in order to implement new ideas and strengthen leadership skills. Professional Development Meetings Strategic Briefing Meetings Webinars This customized management consulting services is designed to support and guide school districts address the academic mission while improving district management. DMC offers a variety of: Strategic Briefing and Facilitation District Support Services Custom Management Consulting Management Institute In order to help school districts save time and money while putting all the resources of DMC at your fingertips, DMC has developed an integrated and complementary suite of membership options. DMC provides the following membership options: Individual Affiliate Comprehensive Leadership Development Technology Solutions Strategic Advisory Publications Membership • DMC offers school districts a growing array of software-based management tools to assist in strategic alignment, execution and resource allocation. • DMC provides the following software solutions: • Scheduler • Dashboard • Evaluation Manager

6. Agenda Introduction What is Teacher Effectiveness? Evaluation at the Core Rethinking the System

7. The Puzzle of Teacher Quality: Two Paradigms A Nation at Risk (1983) Credentialing Highly qualified • From the seminal Nation at Risk study conducted during the Reagan administration to today, significant attention has been placed on improving the education system through improvements in teacher quality. • As the No Child Left Behind act underscored, the notion of a “Highly Qualified Teacher” has been paramount in improving the quality of education nationwide. • However, a second paradigm has existed in parallel, and is currently in the forefront of national attention. • “Teacher effectiveness”, by comparison, focuses more on student achievement results than on teacher qualifications. Teacher Quality Teacher Effectiveness National Commission on Excellence in Education Task Force on Teaching as a Profession • A Nation Prepared (1986) • Professionalism • Highly effective The roots of this conversation span more than 25 years.

8. From Teacher Quality… A Historical Focus on “Qualifications”: • Over the last fifty years, education research as an industry has focused significant resources and attention on understanding which teacher characteristics and qualifications drive classroom success. • Unfortunately, research has not been able to paint a picture where certain qualifications lead to student success. • Rather than continuing to search for evidence of what does work, a growing emphasis in the national dialogue is on policy reform using evidence of practices that don’t. • For instance, should districts continue to invest in qualifications that have no demonstrable effect on student outcomes? • Broad categories of qualifications, like those at left, have been sliced in many ways to find nuggets of predictive information, mostly with little success. Research Evidence Teacher Experience • Positive relationship Teacher Preparation • Mixed evidence Teacher Certification • Mixed evidence Teacher Coursework • Mixed evidence Teacher Test Scores • Positive relationship

9. …to Teacher Effectiveness Toward a discussion on student performance outcomes: Teachers differ enormously in their effect on achievement: Differences from top to bottom quartile are roughly twice the class size impacts in Tennessee STAR Differences from top to bottom quartile are one-third to one-fourth of achievement gap Difficult to predict success on front end based on degrees, qualifications, aptitude. Best way to predict future performance is past performance Nationally, nearly all teachers are “highly qualified”: where does the conversation go from here? • Teacher effectiveness forces us to look at student performance outcomes instead of teacher characteristics. • Evidence from teacher effectiveness research continues to underscore the importance of great teaching: an effective teacher can quickly change the academic trajectory of a student and an ineffective teacher risks derailing progress.

10. So, What Should Our Focus Be? Does our system need to improve how it develops and supports the moderately effective and less effective teachers? How should we differentiate our interventions? High Does our system need to improve how it exits the most ineffective teachers? Does our system need to improve how it retains and leverages the most effective teachers? Frequency Low Low High Teacher Performance Level • If we consider a generic performance distribution, at least three broad categories with corresponding strategic questions emerge: 1) the most effective teachers, 2) the broad middle, and 3) the least effective teachers. • Once identified, what should the district’s role be in managing these distinct groups? • While most districts are likely to answer “We need to do all three”, the more difficult questions arise when thinking about prioritization and sequencing. • What should the district tackle first? • When resources are limited, what should our priority be? Source: DMC

11. What is “The Widget Effect”? All teachers are rated good or great Excellence goes unrecognized Inadequate professional development No special attention to novices Poor performance goes unaddressed The Widget Effect is characterized by institutional indifference to variations in teacher performance. Teacher evaluation systems reflect and reinforce this indifference in several ways. • A widely-read report titled “The Widget Effect” from the New Teacher Project has been an additional national catalyst in helping define concrete issues to improve teacher effectiveness. • The Widget Effect has helped a dialogue to emerge around five key findings, listed at left. • Each of these findings helps to inform an opportunity for district leadership to rethink how teacher effectiveness is managed. • How good is our teacher evaluation system? • How well is it implemented? • Does the performance data from our evaluation system inform the delivery of professional development? • How do we allocate our development resources to those most in need, including novice teachers or those struggling? • How do we reward our most effective talent? Findings Source: The New Teacher Project Report available at: http://widgeteffect.org/

12. Teacher Effectiveness Scope Defining & Redefining Success Teacher Preparation & Background Recruiting & Hiring Processes Induction Ongoing Performance Management • Education & Training (Course-based & Clinical) • Personality & Ambition • Certification • Geographic reach • Interview processes • Timeline management • Messaging • Subjects covered • Duration • Intensity • Mentoring structure • Evaluation approaches • Career tracking • Staffing models • Professional development Structuring the Discussion: Teacher Effectiveness Scope • The challenge of managing teacher effectiveness as a system is paramount for efficient resource utilization and improved. • The full scope of factors influencing teacher effectiveness is large. • From early teacher preparation to ongoing performance management throughout the teacher’s career, decisions should be informed wherever possible by data to maximize the potential for: • Student achievement growth • Efficient and equitable resource utilization • Operational efficiency Source: DMC

13. Teacher Effectiveness Scope Defining & Redefining Success Recruiting & Hiring Processes Induction Ongoing Performance Management Teacher Preparation & Background • Education & Training (Course-based & Clinical) • Personality & Ambition • Certification • Geographic reach • Interview processes • Timeline management • Messaging • Subjects covered • Duration • Intensity • Mentoring structure • See below Career Tracking & Succession Planning Staffing & Allocation Models Compensation & Rewards/ Recognition Professional Dev’t & Leadership Dev’t Access & Delivery Evaluation: Tool & Process • Qualitative: • Skill/will or Perf./Potential • Observation/360s • Evaluation rubrics • Quantitative: • Growth/Value-added data • Achievement data • Formative assessments Data Tools & Support Teacher Effectiveness: Performance Management “Engine” • Managing teacher effectiveness as a system requires districts to make informed connections between teacher and student performance and related human capital practices across the district. • The “engine” for creating that performance data must lie first within the district’s teacher evaluation system. • Insights from evaluation data should inform other key factors in ongoing performance management, such as professional development, staffing, compensation and career tracking. • Further, the data tools and support should also inform other broad human capital practices. • Can the system get “smarter” for what works locally by analyzing data and tailoring approaches for recruiting, retention, and induction? Ongoing Performance Management Source: DMC

14. Confronting Perceptions: Views on Performance Measures • How does this compare with your perception? • With your district’s evaluation protocol? Teacher Views on Accuracy of Performance Measures • If performance needs to take a central role in the broad pursuit of teacher effectiveness, then it is important to consider the perceptions of the single most important stakeholder group: the teachers themselves. • The Gates Foundation, in conjunction with Scholastic conducted a large national survey of teachers, with some interesting findings. • First, the most common measurement approach for teacher evaluation, principal classroom observations, was found to be only in the middle for perception of accuracy. • Regarding quantitative data, teachers were reluctant to embrace achievement status as a measure, but appear favorable to using student achievement growth as a metric. Source: BMGF

15. Agenda Introduction What is Teacher Effectiveness? Evaluation at the Core Rethinking the System

16. Measuring & Growing Effectiveness: We Need Tools and a Process Should support key decision parameters for other performance management functions (PD/skillbuilding etc.) Promote simplicity Evaluative Developmental Evaluation • The challenges of improving teacher evaluation systems can be structured along two main dimensions. • The tools used in the process, including the rubrics and scoring mechanisms, must address the right questions and support the right dialogue to help teachers be successful. • The process itself is also critical, in that the best tools will be ineffectual if not implemented with fidelity. • It is possible for districts to have one but not the other! Tools Process • Compliance orientation • Other motivation: • Team/peer pressure • Training/intrinsic motivation • Financial Goal: Create Individualized Development Plans that are both evaluative and developmental Goal: Create process that is actually used, and feeds other functions with timely information

17. Delivering Great Evaluations: Best Practices Six Design Principles Annual Process Clear, Rigorous Expectations Multiple Measures Multiple Ratings Regular Feedback Significance • A recent report from the New Teacher Project highlights six design principles for evaluations, shown at left. • DMC’s emphasis for this discussion is on the design of the tool itself, which should combine both qualitative measures of teacher practice, as well as quantitative measures of student achievement. Great evaluation tools use Quantitative measures and Qualitative measures Source: TNTP, DMC analysis

18. Race to the Top: National Catalyst for Teacher Effectiveness Each of the 41 RTTT applications mentioned a focus on Teacher Effectiveness • As applications for Race to the Top grants were developed, only about half of the states had quantitative growth models in place or in progress. • Growth models are data models to track student achievement growth over time, allowing deeper analysis of teacher effectiveness trends, such as that highlighted in the research from Gordon, Kane and Staiger. • However, this data also demonstrates that nationally, the country is still in early stages of incorporating quantitative student achievement outcomes in myriad applications for improved district management. Per the Application, Does the State Have a Student Growth Model? 22% 9 States 29% 12 States 49% 20 States Round 1 of Race to the Top highlights efforts to improve the quantitative efforts around using student achievement to improve teacher effectiveness. Source: Learning Point Associates, May 2010

19. Weighting of Quantitative v. Qualitative Qualitative Quantitative • Also a matter of national discussion with no dominant approach yet is the weighting of qualitative versus quantitative measures in summative evaluations. • Sample splits in core teacher evaluations: • Tennessee: 50% Student achievement (35% Value-added Modeling results) • NYC DOE: 40% Student achievement-based • DCPS: 55% Student achievement (incl. 5% from overall school performance) • Rhode Island: Minimum 50% student achievement • Florida: Minimum 50% student achievement • Massachusetts: split not prescribed by state • Illinois: undefined, but must be “significant” percentage • Virginia: 40% of summative evaluation based on student academic progress

20. Recent Insights from Brookings & Gates • “Two types of evidence – student achievement gains and student feedback – do seem to point in the same direction, with teachers performing better on one measure tending to perform better on other measures.” • “In other words, it is possible to combine in a coherent package a teacher’s student achievement results with feedback on specific strengths and weaknesses in their practice.”” • Relevant to the national teacher effectiveness dialogue, two significant new research reports have been released in the last year: • Learning about Teaching: Initial Findings from the Measure of Effective Teaching Project (Bill & Melinda Gates Foundation) • Evaluating Teachers: The Important Role of Value-Added (Brookings Institution) • Both reports give relatively positive indications about the use of quantitative measures to assist in improving teacher effectiveness. • “Rather than asking value-added to measure up to an arbitrary standard of perfection, it would be productive to ask how it performs compared to classification based on other forms of available information of teachers: • Conclusion: Value-added has an important role to play

21. Measuring Status, Improvement and Growth Status Improvement Growth Change Count or Mean Change • Annual assessment of representative group • Annual assessment of representative group • Comparable across years • Consistent performance standards • Two years of data at minimum • Annual assessment of successive grades • Capability to track individuals across time and buildings • Consistent performance standards/interpretation • Vertically aligned performance standards • 2-3 years of data minimum, depending on statistical methodology Data Needs • Easy! • Accounting for all students by subgroup • Moderate! • Index versus school targets • Accounting for all students by subgroup by year • Range from Easy to Difficult! • Specialized skills and software required • Baseline calculations necessary for most approaches • Significantly more expansive data requirements • Significantly more difficult to execute with poor or missing data Analysis Source: National Center for the Improvement of Educational Assessment, CCSSO, DMC analysis

22. Matching Qualitative and Quantitative Measurement The Risk The Goal Qualitative Scores (e.g. Summative evaluation) High High Qualitative Scores (e.g. Summative evaluation) Quantitative Scores (e.g. VAM) Quantitative Scores (e.g. VAM) Frequency Frequency Low Low Low High Low High Teacher Performance Level Teacher Performance Level • Regardless of which specific approaches are chosen for teacher practice (qualitative) and student outcomes measures (quantitative), a compelling research opportunity arises for all districts. • If, as in the figure on the left, a mismatch occurs between what the qualitative and quantitative data are telling us about teacher effectiveness, how will we reconcile the two observations? • A skewed distribution might be expected due to research that shows current teacher evaluations to “grade inflate” overall summative ratings. • Our qualitative perceptions and measurement of effective teaching should over time correlate with the outcomes that these practices are demonstrating.

23. New Haven Example: Summative Assessment • The challenge of matching qualitative and quantitative evaluation data is highlighted in this example from New Haven (CT). • The ratings for evaluation components will be synthesized into a final summative rating at the end of each year. • Student growth outcomes will play a preponderant role in the synthesis. • While the end-of-year summative rating is the official rating on record, all teachers should be aware of what that summative rating will be, based on ongoing situational feedback, as well as feedback received at each evaluation and development conference throughout the year. • Ratings with this degree of mismatch should be the subject of focused policy review, outside the context of the specific teacher’s evaluation, to determine why such a mismatch is occurring and what, if anything, needs to be corrected. • The individual ratings themselves will also be reviewed to ensure that the given rating in these situations is fair and accurate based on the preponderance of evidence shared by the instructional manager and teacher. Individual ratings may be adjusted for unfairness or inconsistency. Note: Ratings are given on a 1-to-5 scale. Instructional Practices will make up 80 percent of the combined Instructional Practices and Professional Values rating. Professional Value will account for 20 percent. 23 Source: New Haven Public Schools

24. Agenda Introduction What is Teacher Effectiveness? Evaluation at the Core Rethinking the System

25. 10 Steps for a New Teacher Effectiveness System DEVELOP 1. Change the Discussion & Involve Many Stakeholders 2. Articulate Vision of Effective Teaching 3. Develop a Multi-Measure Evaluation System, Customized by Position 4. Incorporate into Systemic Human Capital practices (e.g. PD) 5. Assign Clear Roles and Responsibilities in Evaluation Process Ongoing Management Feedback loops IMPLEMENT 6. Train Evaluators & Communicate Expectations Broadly 7. Conduct Observations, Evaluations & Individualized Responses 8. Move from Pilot to full Implementation, if applicable 9. Integrate Data to Track Evaluation Insights 10. Refine Human Capital practices based on Emerging Insights • DMC has designed this 10-step process to assist districts in structuring a sequence of key events to drive improved teacher effectiveness.

26. The District Management Council If you have any comments or questions about the content of this document, please contact Nick Morgan, DMC Managing Director nmorgan@dmcouncil.org Tel. 877.DMC.3500 x201 (Toll Free) Fax. 617.491.5266 70 Franklin Street, Boston MA 02110 Web: www.dmcouncil.org