Providing Effective, Supportive Feedback on Teacher Observations

1. Virginia Department of Education Office of School Improvement Providing Effective, Supportive Feedback on Teacher Observations 2013-2014

2. Agenda

3. Why is Lesson Observation Important?

4. Why do we observe teachers? • “…[T]he purpose of supervision should be the enhancement of teachers’ pedagogical skills, with the ultimate goal of enhancing student achievement.” Marzano, R.J., Fronteri, T., & Livingston, D. (2011). Effective supervision: Supporting the art and science of teaching. Alexandria, Virginia: ASCD..

5. Does this relate to teacher evaluation? “My ideal evaluator would care deeply about the work I do and would have a vision for how that work, done artfully, contributes to the betterment of people. Tomlinson, C.A. (2012). “The evaluation of my dreams.” Educational Leadership, 70(3). ASCD.

6. Why Evaluation Only Is Not Enough Davis, D. R., Ellett, C. D., & Annunziata, J. (2002). Teacher evaluation, leadership and learning organizations. Journal of Personnel Evaluation in Education, 16(4), 287-301. p. 288.

7. Making Permanent Changes: The Importance of Leadership

8. Question 2 What is evidence, and how do we collect it objectively?

9. What is Evidence? “The available body of facts or information indicating whether a belief or proposition is true or valid.” -Dictionary.com The teacher is Proficient in the teaching standard.

10. Focus of Evidence STANDARDS & INDICATORS Teacher Practice Student Learning

11. Evidence in Observations… • Is standards-based • Can include both examples of meeting or not meeting the expectation • Does not include judgment • 3-4 observations a year = less than 1% of overall teaching time • Observations = snapshots

12. Main Areas for Evidence Collection during Observations • Standard 1: Professional Knowledge • Standard 2: Instructional Planning • Standard 3: Instructional Delivery • Standard 4: Assessment of/for Learning (formative assessment) • Standard 5: Learning Environment • Standard 6: Professionalism • Standard 7: Student Academic Progress

13. How do we collect evidence objectively?

14. Documenting Evidence Tips • Avoid terms that express judgment (“neat classroom,” “fun activity,” “caring attitude”) • Avoid words that imply, but do not specify, quantity (“most,” “few,” “several”) • Stick to the five senses • Remember Who, What, When, Where, How

15. How can this evidence be improved? (1) 3.3 Differentiates instruction to meet students’ needs. Spent the majority of the time on whole class instruction for math though several of the students demonstrated proficiency on the long division process in the first few minutes of class. How much? How many? How?

16. 3.3 Differentiates instruction to meet students’ needs. The teacher spent 45 minutes reviewing the steps in the long division process with the whole class though four students correctly completed all steps in the process on the problem introduced during the warm-up activity. Better Example

17. 5.1 Arranges the classroom to maximize learning while providing a safe environment. Classroom neatly arranged so students can work in small groups. How can this evidence be improved? How? Desks in groups? Center areas? How is this defined? When during the lesson do students work in small groups?

18. Improve this Evidence: One Example 5.1 Arranges the classroom to maximize learning while providing a safe environment. All desks are arranged in clusters of 5-6. During Explore activity, teacher distributed task cards to each cluster and had them work together to solve the Explore problem.

19. Using an Standards-Based Observation Form

20. Standard 1: Professional Knowledge • 1.1 Effectively addresses appropriate curriculum standards. • 1.2 Integrates key content elements and facilitates students’ use of higher level thinking skills in instruction. • 1.3 Demonstrates an ability to link present content with past and future learning experiences, other subject areas, and real world experiences and applications. • 1.4 Demonstrates an accurate knowledge of the subject matter. • 1.5 Demonstrates skills relevant to the subject area(s) taught. • 1.6 Bases instruction on goals that reflect high expectations and an understanding of the subject. • 1.7 Demonstrates an understanding of the intellectual, social, emotional, and physical development of the age group. • 1.8 Communicates clearly and checks for understanding.

21. Example Observation

22. Sample Indicators • Sample Performance Indicators: • Examples may include, but are not limited to: • The teacher: • 3.1 Engages and maintains students in active learning. • 3.2 Builds upon students’ existing knowledge and skills. • 3.3 Differentiates instruction to meet the students’ needs. • 3.4 Reinforces learning goals consistently throughout the lesson. • 3.5 Uses a variety of effective instructional strategies and resources. • 3.6 Uses instructional technology to enhance student learning. • 3.7 Communicates clearly and checks for understanding.

23. Example Observation (3)

24. Sample Indicators (1) • Sample Performance Indicators: • Examples may include, but are not limited to: • The teacher: • 4.1 Uses pre-assessment data to develop expectations for students, to differentiate instruction, and to document learning. • 4.2 Involves students in setting learning goals and monitoring their own progress. • 4.3 Uses a variety of assessment strategies and instruments that are valid and appropriate for the content and for the student population. • 4.4 Aligns student assessment with established curriculum standards and benchmarks. • 4.5 Uses assessment tools for both formative and summative purposes, and uses grading practices that report final mastery in relationship to content goals and objectives. • 4.6 Uses assessment tools for both formative and summative purposes to inform, guide, and adjust students’ learning. • 4.7 Gives constructive and frequent feedback to students on their learning.

25. Example Observation (4)

26. Sample Indicators (5.1) • Sample Performance Indicators: • Examples may include, but are not limited to: • The teacher: • 5.1 Arranges the classroom to maximize learning while providing a safe environment. • 5.2 Establishes clear expectations, with student input, for classroom rules and procedures early in the school year, and enforces them consistently and fairly. • 5.3 Maximizes instructional time and minimizes disruptions. • 5.4 Establishes a climate of trust and teamwork by being fair, caring, respectful, and enthusiastic. • 5.5 Promotes cultural sensitivity. • 5.6 Respects students’ diversity, including language, culture, race, gender, and special needs. • 5.7 Actively listens and pays attention to students’ needs and responses. • 5.8 Maximizes instructional learning time by working with students individually as well as in small groups or whole groups.

27. Example Observation (5)

28. Increasing Reliability, Decreasing Bias in Observations

29. What are some of the most common reliability issues in observations?

30. How consistent are the evaluator’s observations with those of an expert observer? Problem:Lack of Criterion Reliability The MET Project, 2013

31. Solution:Establishing Criterion Reliability • Training with an expert observer • Practice scoring using videos, documentation, etc. • Instruction on how to interpret evidence • Information about common sources of systemic observer error The MET Project, 2013

32. How consistent is the evaluator in his/her own observations? Problem:Establishing Intra-Evaluator Reliability The Met Project, 2013

33. Watch the same lesson multiple times with time between Give ratings based on multiple pieces of evidence rather than single observations Solution:Establishing Intra-Evaluator Reliability The Met Project, 2013

34. How consistent is evidence collected between two or more evaluators? Problem:Lack of Inter-Evaluator Reliability The Met Project, 2013

35. Solution:Establishing Inter-Evaluator Reliability • Calibrate evidence-collection during initial training • Conduct tandem observations and performance reviews with multiple evaluators The Met Project, 2013

36. How do we make observations more reliable overall? • Acknowledging the existence of bias • Understanding the different types of bias and how to counteract them The Met Project, 2013

37. How do we decrease bias?

38. Problem: Observer Personal Bias Observers collect idiosyncratic or irrelevant evidence during lesson observations Definition

39. Example: Personal Bias Well, you used higher level questioning, the students seemed engaged, and your formative assessment was on-point. Unfortunately, I noticed that your Unifix cubes weren’t organized by color, so you lost a few points there.

40. Train evaluators on objective ways to collect evidence on uniform, research-based criteria. Solution

41. Problem: Halo and Pitchfork Effect Early impressions of the teacher influence all further observations and evidence-collection Definition

42. Halo Effect Don’t worry, Mr. Williams. You can receive three more poor lesson observations before you use up all the points you earned at your interview for being well-spoken and using that heavy-bond resume paper.

43. Pitchfork Effect That was a great lesson you just taught, Ms. Johnson. It’s really too bad you came late to that meeting back in September and I can only mark you negatively now.

44. Solutions • Train evaluators on objective ways to collect evidence on uniform, research-based criteria • Multiple observers provide various perspectives

45. Problem: Observer Drift Tendency of observers who originally had a desired level of agreement and accuracy to gradually and unconsciously redefine criteria Definition

46. Student learning is measured by the number of higher level questions students ask during the lesson. No, it’s measured by the number of students who are awake at any given moment. Observer Drift I think it’s more to do with quality of the products student create during a lesson.

47. Overall Ways to Increase Reliability and Decrease Bias

48. Same evaluator observing multiple lessons by the same teachers Overall Ways to Increase Reliability

49. Adding more evaluators within the school to the same observation Overall Ways to Increase Reliability (1)

50. Add observers or evaluators from outside the school to either observe or review observations Evaluators’ performance increases when they know they are being monitored (Center of Educational Compensation Reform, 2009) Can be a small sample of teachers’ observations Overall Ways to Increase Reliability (2)