Lisa Foehr Rhode Island Department of Education Thanos Patelis Center for Assessment

1. The Design and Implementation of Educator Evaluation Systems, Variability of Systems and the Role of a Theory of ActionRhode Island Lisa Foehr Rhode Island Department of Education Thanos Patelis Center for Assessment Presentation at the National Conference on Student Assessment New Orleans, June 25, 2014

2. Overview Overview of the educator evaluation system Some results Variability that exists Theory of Action Recap Foehr & Patelis – NCSA - 06/25/2014

3. Overview of the System In 2012-2013, full implementation of educator evaluations. Multiple measures are utilized to develop final effectiveness ratings for each educator. Foehr & Patelis – NCSA - 06/25/2014

4. Overview of the System The evaluation system is embedded within a year-long process representing goal setting, observation, data collection, and reflection. Foehr & Patelis – NCSA - 06/25/2014

5. Overview of the System Professional Practice Rating involves observation using a rubric on eight components adapted from Domains 2 and 3 of the 2013 version of Danielson’s Framework for Teaching. Foehr & Patelis – NCSA - 06/25/2014

6. Practice Adapted from Domains 2 and 3 of the 2013 version of Charlotte Danielson’s Framework for Teaching Foehr & Patelis – NCSA - 06/25/2014

7. Foundations Foehr & Patelis – NCSA - 06/25/2014

8. Overview of the System + Foehr & Patelis – NCSA - 06/25/2014

9. Results from 2012-2013 - Teachers Highly Effective = 47.2% Effective = 48.4% Developing = 3.7% Ineffective = 0.5% Foehr & Patelis – NCSA - 06/25/2014

10. Results - Teachers Foehr & Patelis – NCSA - 06/25/2014

11. Results – Building Administrators Foehr & Patelis – NCSA - 06/25/2014

12. Variability • Rhode Island has four approved systems for teachers and two approved systems for building administrators: • Teacher Evaluation Systems • Coventry Teacher Evaluation System • Used by one LEA • Learning Community Teacher Evaluation System • Used by a charter chool • RI Innovation Consortium Teacher Evaluation and Support System • Used by 7 LEA’s • RI Model Teacher Evaluation and Support System • Used by 46 LEAs • Building Administrator Evaluation Systems • Coventry Building Administrator Evaluation System • Used by one LEA • RI Model Building Administrator Evaluation System • Used by 53 LEAs • A new evaluation system for support professionals is being developed and gradually implemented in the 2013-14 school year in preparation for full implementation during the 2014-15 school year. Foehr & Patelis – NCSA - 06/25/2014

13. Variability (cont’d) • LEAs have the opportunity to establish local policies and structures that will support the effective implementation of their educator evaluations. • Local variability can be found on the professional practice and foundations components. • There are state-wide guidelines for the development that includes training. But there is local variability with the number (above the minimum of two), the type of assessments used, how targets are set, etc. • Variability permissible if meet RI educator standards. • To assist LEAs with this essential work, RIDE is producing several guidance documents to help LEAs align local policies and long-term plans both with their own visions for great teaching and school leadership, and with Rhode Island’s Educator Evaluation System Standards. Foehr & Patelis – NCSA - 06/25/2014

14. Variability (cont’d) • In the 2013-14 school year a Differentiated Evaluation Process for Teachers was introduced. • The Differentiated Evaluation Process for Teachers provides schools and districts with greater flexibility for classroom observations, evaluation conferences and Professional Growth Plans. • The Differentiated Evaluation Process for Teachers establishes new minimum requirements for classroom observations and evaluation conferences, but schools and districts may choose to conduct more than the minimum of either. Foehr & Patelis – NCSA - 06/25/2014

15. Role of Theory of Action • Summarizes the goals and expectations of the system. • Assists in the evolution and development of the systems. • Offersa means to evaluate local policies and structures. • Permits the development of a validation agenda. Foehr & Patelis – NCSA - 06/25/2014

16. Overarching Goals of RI’s Theory of Action Source: Hall, 2013. Obtain measurable improvements in student learning and achievement. Obtain measurable improvement in overall educator effectiveness. Establish a fair, reliable, legally defensible educator effectiveness measure to support human capital decisions. Incrementally increase the number of new educators who reflect the range of knowledge, skills and competencies necessary to be effective. Improve perceptions of educator quality and accountability across various stakeholder groups. Increase support provided to educators to improve effectiveness. Foehr & Patelis – NCSA - 06/25/2014

17. Evaluation Criteria & Claims Source: Hall, 2013. • Three criteria: • Professional Practice – a measure of effective instruction and classroom environment • Professional Foundations – a measure of instructional planning and the contributions teacher make as members of their learning community • Student Learning – a measure of a teachers’ impact on student learning. • For each of the criteria produced statements for the following: • Description • Data source • Scoring methodology • Performance categories • Rational for inclusion in the system • Associated claims Foehr & Patelis – NCSA - 06/25/2014

18. Evaluation Criteria & Claims • Sample for one criterion… • Criterion: • Professional Practice (PP) • Claim: • Teachers who score high on the PP display the instructional skills and classroom environment expected from teachers. • Teachers who score low on the PP can benefit fro actionable feedback and targeted professional development. Source: Hall, 2013. Foehr & Patelis – NCSA - 06/25/2014

19. Recap Educator evaluations systems in Rhode Island (now two years old) represent an effort to capture multiple measures involving professional practice, professional foundations and student learning. Variability within Rhode Island is possible, as long as systems conform to the RI Professional Teaching Standards, RI Educational Leadership Standards and the RI Code of Professional Responsibility and as expressed in the Rhode Island Theory of Action. Ongoing feedback is captured and incorporated into revisions. Results show comparability. Efforts and processes represent major undertaking for the department. Theory of Action assists in decision making, particularly around understanding and accommodating variability. Theory of Action permits the development of a validation agenda to substantiate claims. Foehr & Patelis – NCSA - 06/25/2014

20. References Bell, C. A., Gitomer, D. H., McCaffrey, D., Hamre, B., Pianta, R., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17, 1-26. Danielson, C. (2013). The Framework for Teaching Evaluation Instrument. Princeton, NJ: The Danielson Group. Hall, E. (2013). Validation Framework for the Rhode Island Model Evaluation and Support System. Unpublished report produced for the Rhode Island Department of Education. Foehr & Patelis – NCSA - 06/25/2014

21. Questions and Comments? Center for Assessment www.nciea.org ThanosPatelis tpatelis@nciea.org Foehr & Patelis – NCSA - 06/25/2014