Observation models

1. Observation models OEA Leadership Academy 2011 Michele Winship, Ph.D. winshipm@ohea.org

2. Multiple measures of teacher effectiveness • Evidence of growth in student learning and competency • Standardized tests, pre/post tests in untested subjects • Student performance (art, music, etc.) • Curriculum-based tests given in a standardized manner • Classroom-based tests such as DIBELS • Evidence of instructional quality • Classroom observations • Lesson plans, assignments, and student work • Student surveys such as Harvard’s Tripod • Evidence binder (next generation of portfolio) • Evidence of professional responsibility • Administrator/supervisor reports, parent surveys • Teacher reflection and self-reports, records of contributions

3. Evidence of Instructional Quality: Observations • Observations have been the primary source of evidence in most traditional teacher evaluation systems.

4. Teacher observations: strengths and weaknesses • Strengths • Great for teacher formative evaluation (if observation is followed by opportunity to discuss) • Helps evaluator (principals or others) understand teachers’ needs across school or across district • Weaknesses • Only as good as the instruments and the observers • Considered “less objective” • Expensive to conduct (personnel time, training, calibrating) • Validity of observation results may vary with who is doing them, depending on how well trained and calibrated they are

7. Charlotte Danielson’s framework

10. The class observation tool • The Classroom Assessment Scoring System™ (CLASS™) is an observational instrument developed at the Curry School Center for Advanced Study of Teaching and Learning to assess classroom quality in PK-12 classrooms. It describes multiple dimensions of teaching that are linked to student achievement and development and has been validated in over 2,000 classrooms. The CLASS™ can be used to reliably assess classroom quality for research and program evaluation and also provides a tool to help new and experienced teachers become more effective. • The CLASS™ system began as part of a national study in early childhood development. Eventually, that research grew in significance as education policies shifted focus to teacher accountability. • Teachers needed some way to see how their methods were working in the classroom. The CLASS™ instrument became a much-needed tool that could both effectively measure teacher-student interactions in a classroom setting and offer resources for strengthening those interactions across any subject area or age group

11. Example: University of Virginia’s CLASS observation tool

12. Kim Marshall rubric Rationale and suggestions for implementation 1. These rubrics are organized around six domains covering all aspects of a teacher’s job performance: A. Planning and Preparation for Learning B. Classroom Management C. Delivery of Instruction D. Monitoring, Assessment, and Follow-Up E. Family and Community Outreach F. Professional Responsibilities The rubrics use a four-level rating scale with the following labels: 4 – Expert 3 – Proficient 2 – Needs Improvement 1 – Does Not Meet Standards

13. Kim Marshall rubric 2. The rubrics are designed to give teachers an end-of-the-year assessment of where they stand in all performance areas and detailed guidance on how to improve. They are not checklists for classroom visits. To knowledgeably fill out the rubrics, principals need to have been in classrooms frequently throughout the year; it is irresponsible to fill out the rubrics based on one classroom observation. Regular, unannounced mini-observations followed by face-to-face conversations are the best way for principals to have an accurate sense of teachers’ performance, give formative praise and suggestions, and listen to push-back. For a detailed account of the development of these rubrics – and the rationale for not including student results – download Kim’s September/October 2006 Kappan EDge article at http://www.marshallmemo.com (click on Kim Marshall Bio/Publications and scroll down).

14. Kim Marshall rubric

15. GATES MET Project • Classroom observations and teacher reflections: Researchers will videotape four lessons each year in each selected class and subject. Teachers will provide written commentary and any relevant supporting materials to provide context about the videotaped lesson and the videotapes will be reviewed by trained experts using several different sets of nationally-recognized teaching standards. • For detailed information about the video capture process used in the study, see the paper Classroom Observations and the MET Project. • For details about the rubrics used to evaluate the videotaped lessons, see: • The Classroom Assessment Scoring System (CLASS) measure developed at the University of Virginia • The Framework for Teaching (FFT) developed by Charlotte Danielson • The Mathematical Quality of Instruction (MQI) developed at the University of Michigan and Harvard University • The Protocol for Language Arts Teaching Observation (PLATO) developed at Stanford University

16. Teachscape 360-camera Costs $4,800 plus annual software licensing fees that range from $65 to $140 per teacher.

18. OTES Observation Rubric • Being modified to meet new legislation. • Developed from the Ohio Continuum of Teacher Development which is based on and an expansion of the Ohio Standards for the Teaching Profession. • RTTT districts must have evaluations that distinguish levels of teaching performance and effectiveness. • HB 153 requires the categories to be Accomplished, Proficient, Developing and Ineffective.

19. OTES Observation Tool

20. OTES Observation narrative

21. OTES Observation Rubric

22. Addressing weaknesses of Teacher observation • Weakness • Only as good as the instruments and the observers • Solutions • Create or select quality instruments that • Clearly identify standards of performance • Delineate multiple levels of performance for each standard • Include enough description for each level of performance to provide substantive feedback to educators • Require documented evidence (anecdotal or physical) to support evaluator ratings • Allow for teacher input/clarification in pre- and post-conferences and/or written reflections • Carefully select and train ALL evaluators

23. Addressing weaknesses of Teacher observation • Weakness • Considered “less objective” • Validity of observation results may vary with who is doing them, depending on how well trained and calibrated they are • Solutions • Create a standardized observation protocol • Train all staff in the observation protocol and its language • Use an instrument that provides clear descriptions in its standards and levels of performance • Train and calibrate ALL evaluators regularly • Periodically review evaluators’ ratings for reliability • Solicit feedback from teachers about their experiences using the observation protocol

24. Addressing weaknesses of Teacher observation • Weakness • Expensive to conduct (personnel time, training, calibrating) • Solutions • Earmark RTTT or other grant funds for evaluation development • Redirect professional development time and funding for all staff training • Use extended contract time for training and calibrating administrator evaluators • Build a cadre of peer observers whose role is to provide formative feedback

25. The most important aspect of teacher observation • Training • Training • Training • Training all staff in the observation protocol • Training and calibrating ALL evaluators (use video teaching segments to develop a training program) • Training new evaluators and recalibrating current evaluators on a regular basis

26. Observation instruments Charlotte Danielson’s Framework for Teaching http://www.danielsongroup.org/theframeteach.htm CLASS http://www.teachstone.org/ Kim Marshall Rubric http://www.marshallmemo.com/articles/%20KM%20Teacher%20Eval%20Rubrics%20Sept%202010.pdf Gates MET Project http://www.metproject.org/project

27. References • Laura Goe, Presentation to Learning First Alliance Annual Leadership Council, May 12, 201. Available: http://www.lauragoe.com/LauraGoe/Publications.html • Ohio Teacher Evaluation System draft 4/5/11

28. Contact information • Michele Winship • 614-227-3001 • winshipm@ohea.org