LEAP Observation Calibration

1. LEAP Observation Calibration

2. Content/Language Objective At the end of the training, participants will be able to explain how their thinking has changed on specific indicators,using language from the Framework Evidence Guide, with their group: • using the “Guidelines for Observation” handout • justifying the individual original score for each indicator with evidence • reaching consensus on a score for each indicator • Using the stem, “My thinking on indicator ______ has changed to ___________ because _______."

3. Agenda • Review the Framework • Set Guidelines • Watch Video and Capture Evidence • Categorize and Discuss Evidence

4. Most Common Sources of Rater Bias • Beliefs about effective instructional practice: Inter-rater agreement is more likely when raters accept evaluation system’s goals and definition of good teaching • Easy/Hard Rater: a way of perceiving your own rating style or projecting your beliefs about instruction • Wishing something had happened: lowering the score based on your beliefs about how the teacher should have taught the lesson

5. Addressing Rater Bias • Develop shared understanding of the definition of effective practices embodied by the rubric • Acknowledge and be aware of your biases • Separate your personal instructional beliefs from the observation • Stick to the rubric and its definition of professional practices

6. Framework for Effective Teaching • 15min. to read the Framework • Read Effective in each indicator • Read LE.1, I.1, I.2, all categories

7. Observation Rubric • What are the general characteristics each category?

8. Observation Rubric

9. Collecting Evidence • What tools will you use? • Pen/paper? • Computer • Word document?

10. View the Video • Collect evidence during the video • Don’t worry about writing every single thing you hear or see, just try to collect teacher and student evidence that may be relevant to the Framework indicators.

11. Best Practices for Scoring As an observer you should be able to provide the following for every indicator: • Teacher and/or student evidence from the observed lesson (Each indicator must include specific information about what the teacher and/or students said/did) • Evidence Guide language • Clarifying Statements (Rationale for the scores, if not visible from the evidence) Take 30 min. to independently score each indicator.

12. Mental Processes for Scoring • Compare performance observed to the rubric, not to other teachers • Respect the rubric over your gut feeling (Don’t rely on “I know good teaching when I see it”) • Evaluate based only on the evidence collected; if no evidence, make no inference. • Base judgment on the predominance of evidence; select practice with the majority of evidence, not the level below. • Rate performance on each element separately. • Do not compensate for a near miss on one element with a generous rating on another.

13. Calibration Process • Groups of 3-4 • Everyone provide individual scores for each indicator first • Discuss indicators, prioritizing those with the biggest area of discrepancy • Using group member as a timekeeper, aim for about 7 minutes per indicator • Assign group spokesperson for each indicator who will share evidence, alignment and score

14. Guidelines for Calibration Discussion • Try to avoid “I think” or “I feel” statements • Consensus vs. Compromise • Use the following sentence stems: • The evidence I cited was______________. • The evidence aligned to ____________ bullet in the framework. • The score I aligned the evidence and bullet(s) to is ______________.

15. Normed Scores • Share the normed report • Before each indicator normed score is revealed, each group should share the consensus score.

16. Closing • My thinking about the indicator ______ has changed because ______. • As a teacher, I will think of ___ practice differently. Contact LEAP@dpsk12.org with questions or feedback!