The DPS/DCTA Pay for Performance Pilot An Introduction and Update

1. The DPS/DCTA Pay for Performance PilotAn Introduction and Update Presented to the 2002 CPRE National Conference on Teacher Compensation and Evaluation November 22, 2002 Brad Jupp, Design Team Shirley Scott, Design Team

2. Agenda • The Pay for Performance Pilot -- an Introduction and Update • Some Recent Findings • Eight Building Blocks for Positive School District Change Based on Labor/Management Partnership

3. The Pay for Performance Pilot -- An Introduction and Update • The Basics • The District and Association • The Schools • The Teachers and Principals • The Objectives • Student Learning • District Capacity • How the Pilot Will Conclude

4. An Introduction and Update -- The Basics The Premise: Teacher compensation should be based, in part, on the academic growth of the students that they teach.

5. An Introduction and Update -- The Basics The Pilot in a Paragraph The DPS/DCTA Pay for Performance Pilot investigates setting measurable student achievement objectives. It investigates offering teachers bonuses for meeting those objectives. It will conclude in March 2004, when the Design Team will make a recommendation to the teachers and the Board of Education. The final recommendation will include a radically redesigned salary structure, not just a system of bonuses.

6. An Introduction and Update -- The District and Association • August 1999 -- “A landmark agreement” • April 2000 -- a four year pilot • April 2001 -- the Joint Task Force on Teacher Salary • A climate of tactical thinking about teacher compensation • May 2002 -- “Partnership Agreement”

7. An Introduction and Update -- The District and Association Collective Bargaining spring and summer 1999 • The district position • teacher experience steps should be earned based on student achievement • an attractive salary schedule should be a part of the trade-off • entry pay tops the market • “squared range” with a fixed but diminished ratio between cells • MA60 becomes the top earning education column • The association position • The salary system is attractive, but the objective setting process is untested, so run a pilot on the objective setting process. • National research (anchored by the National Commission for Teaching and America’s Future) points to the importance of skills and knowledge pay

8. An Introduction and Update -- The District and Association • The Memorandum of Agreement on Pay for Performance, August 1999 • Objective setting for bonuses • Schools enter by election • Comparative model -- three approaches • Design Team • Outside expert to evaluate the pilot

9. An Introduction and Update -- The Schools • Sixteen of Denver’s 135 schools participate in the pilot. Schools joined the pilot by successful faculty election • Twelve elementary schools, including eleven that have been in the been in the pilot since October 1999 • Two middle schools • One high school and one “high school education complex”

10. An Introduction and Update -- The Schools

11. An Introduction and Update -- Teachers and Principals • There were 18 principals and 4 assistant principals in the pilot in the 01-02 school year. • Two principals have been in the pilot since its beginning, a turnover rate similar to the overall turnover rate of principals in DPS. • There were 633 teachers in the pilot in the 01-02 school year. • 124 of the original 307 pilot teachers remain in their same schools, a turnover rate similar to the overall turnover rate in DPS.

12. An Introduction and Update -- The Objectives • Teacher objectives are the practical cornerstone of the pay for performance pilot. • Teachers set two objectives and receive a bonus of $750 if they meet the objective. • There was no clear compensation theory underlying the model -- in other words, there was not agreement that the bonus was to motivate, reward, punish or create an incentive. • During the pilot, the work of the Design Team evolved into the work of implementing this objective setting system.

13. An Introduction and Update -- The Objectives • Objectives contain two bodies of information, measurement content and learning content. • Measurement content includes the technical information -- assessment, population, interval and expected gain -- that is used to set and measure progress toward expectations • Learning content includes the major elements of the teacher’s strategic thinking about student expectations -- rationale, student learning priorities and teaching strategies.

14. An Introduction and Update -- The Objectives

15. An Introduction and Update -- Student Learning • Evidence that student achievement or student learning has improved in pilot schools, or is better than student learning in schools outside the pilot, is mixed. • Design Team research shows there is no obvious correlation between percentage of objectives met and student achievement levels or improvement in student achievement levels at the school level. • CTAC research shows that there is correlation between the quality of objectives and student performance on independent measures of student achievement. • CTAC research shows that pilot schools outperformed comparison schools on some tests but not on others. • CTAC research has not correlated those differences in performance to objective setting or pay for performance.

16. An Introduction and Update -- District Capacity • District capacity to support the objective setting process has grown over the course of the pilot • The district has provided teachers and administrators on-line access to a wide range of student achievement information • In September 2002, OASIS had an average of 50 hits a day. • The district has placed the objective setting in a web-bsed environment • In the 01-02 school year 632 teachers (100% of participants) completed their objectives on line. • The district is aligning local, state and federal accountability initiatives through a common set of performance indeicators.

17. An Introduction and Update -- District Capacity • The district and association have engaged in at least 6 different efforts at alternative compensation since 1994 • Market incentives for ELA S teachers • Salary freezes for teachers with unsatisfactory principal evaluations • Tuition supplements and extra pay for teachers with National Board Certificates • Differentiated pay for Teachers in Residence • Market incentives to attract and retain hard to recruit positions • Extra pay for instructional coaches in the literacy program

18. An Introduction and Update -- How the Pilot will Conclude • August 2002 -- the final year that teachers and principals set objectives in 16 schools • April 2003 -- The Joint Task Force on Teacher Compensation presents its first draft recommendation of a teacher compensation system • October 2003 -- The Joint Task Force presentts its revised draft of a recommendation. • December 2003 -- The Community Training and Assistance Center presents its research study on the pilot • January 2004 -- DPS and DCTA will develop a collective bargaining agreement based on the recommendations of the Joint Task Force and the CTAC research study • March 2004 -- the teachers and the Board of Education vote on the collective bargaining agreement

19. Some Recent Findings • Sources • Evidence from teachers gathered by the DCTA • Research by the Community Training and Assistance Center (CTAC) • Research by Ciruli and Associates • Evidence from teachers, principals, schools and DPS gathered by the Design Team • The experience of the Design Team

20. Findings to Date -- Three Areas • It is a sound practice to set measurable objectives based on student growth and based in the teacher’s discipline • Educators are clearly resistant to the concept of pay for performance, but remain open-minded about teacher compensation in general and the results of the Pay for Performance Pilot in specific, and have mixed feelings about the compensation system used in the pilot. • Educators are unclear about the relation between teacher objectives and other accountability initiatives, like accreditation, school improvement planning and, especially teacher evaluation

21. 1. It is a sound practice to set objectives • CTAC Research shows that when teacher objectives are clear about learning content, complete and cohesive, and express high and measurable expectations, they correlate with higher student achievement on independent assessments. • Educators and CTAC research report that the greatest potential of the objective setting practice is the professional conversation between principal and teacher to focus on student growth expectations • Teachers and principals in the pilot report that the objective setting process has worked best where there is effective principal leadership • Teachers and principals report that they rely on training, clear procedures, and information about how to quantify student growth expectations in order to set high quality objectives..

22. 1. It is a sound practice to set objectives • Teachers and principals report that they need flexibility to adjust teacher objectives to meet the unique needs of the students they teach. • Teachers and principals report that they need timely and reliable access to meaningful student growth data if they are to set rigorous objectives. • Teachers and principals report that it is easier for teachers and principals to set high quality objectives when there are performance standards for students and well-aligned assessments that measures student growth. • The Design Team and CTAC have reported that there has been little supervision of the objective setting process by anyone other than the Design Team, whether by district education staff (like area or level superintendents) or by central departments (like Planning or Curriculum).

23. 1. It is a sound practice to set objectives • While the Pay for Performance Memorandum of Understanding between DPS and DCTA provides for extraordinary dispute resolution procedures for teachers and principals, they have not been used during the life of the pilot.

24. 2. Educators have mixed feelings about the compensation system used in the pilot • Research by CTAC, Gonder, Baird and Ciruli Associates all shows that teachers are resistant to the concept of “pay for performance” • In spite of this obvious resistance, research by CTAC shows not only that the majority of educators are open-minded about the results of the pilot, but also that a greater portion of pilot educators describes themselves as open-minded • Research by CTAC shows that many educators believe that pay for performance is inevitable. • Research by DCTA and CTAC indicates that teachers do believe that some portion of their pay or some portion of their evaluation should be accurately measured student growth.

25. 2. Educators have mixed feelings about the compensation system used in the pilot • Research by CTAC mines a much richer body of teacher perceptions that shows a common basis of educator expectations based on what the Design Team Calls “worst fear elements” and “perceived elements of sound practice.” • Research by CTAC also points to places where pilot teachers than non-pilot teachers are more optimistic when they anticipate a future pay for performance system. • The Design Team has explored teacher attitudes in systematic faculty meetings held to gather teacher perceptual data and found that teacher attitudes about receiving bonuses for meeting objectives are ambivalent.

26. 3. Educators are unclear about how teacher objectives fit with other accountability measures • The Design Team has found that teachers and administrators are confused about the relation between objective setting and teacher evaluation. • The Design Team and CTAC research have found that the difference between administrator pay for performance and teacher pay for performance has confused principals and teachers. • The Design Team and CTAC research have found that the current school improvement planning process has little consistent impact the setting of teacher objectives. • The Design Team and CTAC research have found that there is little or no connection between school accreditation and other school based accountability initiatives, such as school improvement planning and teacher objective setting. • The Design Team and CTAC research have found that there is little or no connection between Colorado School Accountability Reporting and other school based accountability initiatives, such as school improvement planning and teacher objective setting.

27. Eight Building Blocks for Positive School District Change Based on Labor/Management Partnership • Saying yes was more powerful than saying no. • Neither the district nor the association will be perfectly ready to begin. • Changing teacher compensation will not necessarily lead to urban school reform. • Do not be afraid to work with student achievement. • There must be shared commitment to complete all reform efforts. • Current district and union capacity is not sufficient. • Be prepared to pay. • Evaluate everything.

28. Thank You. DPS/DCTA Pay for Performance Design Team http://denverpfp.org Brad_Jupp@dpsk12.org or Shirley_Scott@dpsk12.org 303.764.3629