Update of State Council on Educator Effectiveness Recommendations

1. Update of State Council on Educator Effectiveness Recommendations Linda K. Barker Director of Teaching and Learning Colorado Education Association

2. Guiding Principles • Data should inform decisions, but human judgment will always be an essential component of evaluations. • The implementation and evaluation of the system must embody continuous improvement. • The purpose of the system is to provide meaningful and credible feedback that improves performance. • The development and implementation of educator evaluation systems must continue to involve all stakeholders in a collaborative process. • Educator evaluations must take place within a larger system that is aligned and supportive.

3. State Requirements • Conduct performance evaluations for all teachers and principals at least once each school year. • Base at least half of each teacher’s and principal’s evaluation on multiple measures of students’ academic growth (CSAP, as appropriate, plusother growth data). • Requires all teachers and principals in one of at least three performance standards, “highly effective”, “effective”, “partially effective”, or “ineffective”. • Award non-probationary status to teachers with three consecutive years of “effective” performance and remove it for those who are not “effective” for two consecutive years. • Consider factors such as student mobility and the numbers of students with disabilities or at risk of failing school. • Require mutual consent of teachers and principals to teacher assignments. • Factor in teacher effectiveness before seniority when considering district-level layoffs.

4. Requirements Guided by state law, the Council’s work focused on: • Defining teacher effectiveness and principal effectiveness. • Defining Quality Standards and elements for teachers and principals. • Establishing performance standards and guidelines for assigning educators to them. • Developing guidelines to districts for a fair, rigorous and transparent system to evaluate teachers and principals. • Recommending state policy changes to prepare, evaluate and support teachers and principals.

5. Eight Key Components

6. Teacher Evaluation Framework

8. Weighting of Quality Standards Common Statewide • Quality Standard VI equal to at least 50% • Quality Standard I-V equal to no more than 50% • Each of Quality Standards I-V are at least 15% of total sub-score evaluation of professional practice performance District Flexibility • Vary weight afforded to each of Quality Standards I-V by up to 25% of total sub-score evaluation of professional practice performance

10. Measuring Student Growth-QS VI #11-21 • Measures of Teacher Student Growth • Common Statewide • Technical guidelines for valid and reliable measures • Multiple measures of student growth • Include statewide summative assessment data where available • District Flexibility • In collaboration with representatives of local teacher association, districts select measures • Increase frequency and/or variety of measures collected for novice, ineffective or other categories of teachers • Assign teachers to categories based upon quality of available measures • Encouraged to attribute student growth to teams of educators

11. Measures of Growth Categories-#13 • Category A (state criterion-referenced/standards-based) data: CDE-certified student-level assessment data (e.g. CSAP) that is of a technical quality (standardized, external and objective) that allows student growth to be calculated for personnel in specific grades and subjects using the Colorado Growth Model, and justifies its use as a major portion of the educator’s student growth score effectiveness evaluation. • Category B: Student-level assessment data collected from district-created or vendor-created assessment tools that are comparable across classrooms with demonstrated rigor which meet CDE guidelines for technical quality. This category may also include assessments such as the National Assessment of Educational Progress (NAEP), the SAT and ACT, and other norm-referenced tests. These measures may or may not allow for the calculation of student “growth.” • Category C : Student-level assessmentdata using unique teacher or school-based measures collected at the school or individual classroom level, which do not meet the higher technical requirements of Category A and Category B data but which do comply with minimal technical guidelines developed by CDE. These measures may be highly valid as measures of student progress/learning against standards, but will not technically allow for the calculation of student “growth.”

12. Development of New Student Growth Measures - #45 • Academic standards for 10 content areas, statewide assessment for 4 in some grades. In order to better measure student growth for the 70% of teachers who teach an untested subject or grade: • State shall develop statewide summative assessments in Science and Social Studies. • State shall develop or facilitate a state consortium to develop measures of academic outcomes for early childhood. • State shall support districts to develop new measures of student growth for other subjects and grades that are aligned with district scope and sequence of curriculum. • Pilots will also pilot student growth objectives (SGOs).

18. Principal Evaluation Framework

23. Pilots and Integration Districts

25. Pilots Districts PILOT SITES:  These districts will pilot the state model evaluation system starting with the Principal Evaluation protocols during the 2011-12 school year. Pilot site 1: Moffat South Routt Pilot site 2: Jefferson County Pilot site 3 Wray Pilot site 4: Kiowa Crowley Miami-Yoder Custer Pilot site 5 Valley RE-1 Pilot site 6 St. Vrain Pilot site 7 Platte Canyon Pilot site 8 Salida Del Norte Mountain Valley Center

26. Pilot requirements: • Execute a Memorandum of Understanding (MOU) between your district and CDE regarding your participation in the pilot and field tests.  • Include, at a minimum, schools within the district from each level (elementary, middle and secondary) in the pilot.  District-wide participation is encouraged, but we acknowledge that this may be difficult for larger districts. • Participate in training for the principal evaluation provided by CDE during September 2011 (approximately 1.5 days) and for teacher evaluation provided during late spring of 2012 (approximately 1.5 days) as well as some interim training during the initial pilot years. • Evaluate principals during the 2011-2012 academic year using the state model system. • Provide feedback on the teacher evaluation instruments and system during the 2011-2012 academic year. • Provide copies of all evaluation materials and other data identified in the MOU to CDE.

27. Pilot requirements continued: • Participate in interviews and focus groups designed to determine needed changes and to gather ideas for improvement of the system from you and your staff members. • Implement both the principal and teacher evaluation processes in your district during the 2012-13 and 2013-14 school years, and provide information and feedback requested by CDE. • Attend and actively participate in meetings conducted by CDE related to the development process. • Complete a district review including surveys of teachers, principals and district leaders in August 2011, August 2012, August 2013, August 2014, August 2015 and August 2016. • Collect and report data to CDE about the pilot process and selected outcomes for a 5-year period from 2011-2016.  Please note that much of the information collected during the 2013-14, 2014-15, and 2015-16 academic years will be data that districts will be required to report to CDE as part of the new educator effectiveness law. • Provide student achievement data that is linked to teachers beginning in the 2012/13 school year. 

28. Integration Districts: $9.7 Million Colorado Legacy Foundation will partner with the Colorado Department of Education and four local school districts to accelerate and integrate implementation of new educator performance evaluation systems and the Colorado Academic Standards using instructional tools aligned to those standards. The school districts will be determined later this summer and will serve as demonstration sites.   The tools developed and lessons learned from those sites will be shared statewide.  • Denver Public Schools • Eagle County • Centennial • Thompson SD

29. Measures

30. Federal priorities (August 2010) • From “Race to the Top” and reiterated in the August 5, 2010 Federal Register (Vol. 75, No. 150) “Secretary’s Priorities for Discretionary Grant Programs” • Teachers should be evaluated using state standardized tests where possible • For non-tested subjects, other measures (including pre- and post-tests) can be used but must be “rigorous and comparable across classrooms” and must be “between two points in time” • Multiple measures should be used, such as multiple classroom evaluations

31. Multiple measures of teacher effectiveness • Evidence of growth in student learning and competency • Standardized tests, pre/post tests in untested subjects • Student performance (art, music, etc.) • Curriculum-based tests given in a standardized manner • Classroom-based tests such as DIBELS • Evidence of instructional quality • Classroom observations • Lesson plans, assignments, and student work • Student surveys such as Harvard’s Tripod • Evidence binder (next generation of portfolio) • Evidence of professional responsibility • Administrator/supervisor reports, parent surveys • Teacher reflection and self-reports, records of contributions

32. Measuring teachers’ contributions to student learning growth: A summary of current models

33. Non CSAP tested subjects and grades • Measuring effectiveness for the “other 69 percent” is probably the most challenging aspect of including student achievement growth as a component of teacher evaluation. • Non CSAP tested subjects and grades, there are few state models that demonstrate how contributions to student learning growth can be systematically measured and analyzed in ways that allow for differentiation among teachers. • Research has not yet been conducted on how such evidence is being used within evaluation systems.

34. Assessing Musical Behaviors: The type of assessment must match the knowledge or skill 4 types of musical behaviors: Types of assessment Responding Creating Performing Listening Rubrics Playing tests Written tests Practice sheets Teacher Observation Portfolios Peer and Self-Assessment

35. What assessments are teachers and schools going to use? • Existing measures • Curriculum-based assessments (come with packaged curriculum) • Classroom-based individual testing (DRA, DIBELS) • Formative assessments such as NWEA • Progress monitoring tools (for Response to Intervention) • National tests, certifications tests • Rigorous new measures (may be teacher created) • The 4 Ps: Portfolios/products/performance/projects • School-wide or team-based growth • Pro-rated scores in co-teaching situations • Student learning objectives • Any measure that demonstrates students’ growth towards proficiency in appropriate standards

36. Factors for Consideration: • Is there a consensus on the competencies students should achieve in this content area? • What assessments/measurements can be used to reliably measure these competencies with validity? • Should the use of schoolwidevalue-added models be considered as a means to measure student progress in non-tested subjects and grades? • How will growth in performance subjects (e.g., music, art, physical education) be determined? • How will related personnel (“caseload” educators) be factored into the system? • Do these measurements meet all of the federal requirements (i.e., rigorous, between two points in time, and comparable across classrooms)? Are measurements aligned with federal priorities? • Can these measurements be applied to all grades and student populations?

37. Options for Measuring Student Growth: • Existing tests-end of course • Create new assessments • Use for 4 P’s—portfolio, products, performances, or projects. • Student learning objectives • Classroom based

38. Rubric for student learning objectives

39. Rubric for student learning objectives (cont’d)

40. SLO Model Strengths/Weaknesses Strengths • Teachers take an active role in determining student learning goals • Good professional growth opportunity for teachers • If objectives are of high-quality and teachers plan instruction to meet them, students should benefit Weaknesses • Heavily dependent on administrator understanding and time commitment to supervision • Not clear how or if “rigor” could be determined • Not “comparable across classrooms” because teachers set the objectives and they will vary widely • Not clear how students’ beginning point is determined

41. ACCURATE ATTRIBUTION OF STUDENT GROWTH • Do our data systems allow for the accurate and timely linking of teachers with the students they teach? • What are innovative ways to capture the contribution of a teaching team or an entire school to individual student learning? • What characteristics does our linking system need to have in order for teachers to agree that the system is accurate and fair for purposes of evaluation? • What support do teachers, principals, and districts need in developing and supporting linking systems and in analyzing and using the information they produce?

42. DEVELOPMENT OF NEW ASSESSMENTS AND STUDENT LEARNING OBJECTIVES (SLOS) • What, if any, additional statewide assessments should the state develop? • What training and support do teachers and principals need in order to be capable of creating meaningful student learning objectives, and how can that support be most efficiently provided? • What support do districts need in developing new assessments? How should the state think about “quality control” for district-developed assessments?

43. TEACHER QUALITY FRAMEWORK • Given the comprehensiveness of the Teacher Quality Standards, under what circumstances and for what teachers does it make sense to prioritize “must-haves” vs. “nice to- haves?” • How can teachers be involved in designing classroom observation rubrics that are dynamic and flexible enough to capture a wide range of effective practices? • What kinds of trainings do observers and evaluators need in order to ensure consistency across raters within districts and across districts, and how can this training best be delivered? • What are the best ways to link professional development offerings with individual teacher needs so that teachers are able to access timely and relevant professional development on an as-needed basis?

46. Questions/Resources www.coloradoea.org : Teaching and Learning—Educator Effectiveness www.cde.state.co.us/EducatorEffectiveness