SUMMATIVE TEACHER EVALUATION: THE GNARLY NEW GAME IN OUR TOWN

1. SUMMATIVE TEACHER EVALUATION: THE GNARLY NEW GAME IN OUR TOWN W. James Popham University of California, Los Angeles CREATE National Evaluation Institute Atlanta, Georgia October 10-12, 2013

2. PILLARS OF A PUBLIC ADDRESS • Convey import of topic to be treated. • Establish speaker’s credentials. • Preview presentation’s structure. • Present intended outcomes.

3. JIM QUOTES JIM “One of the most elusive targets in the history of educational research is a valid index of teacher effectiveness.”* *Performance Tests of Teaching Proficiency: Rationale, Development, and Validation, American Educational Research Journal, Vol. 8, No. 1, January, 1971. (This is the initial sentence of the article—followed by many equally enchanting sentences.)

4. TWO FEDERAL CATALYSTS SERIOUSLY SHAKE UP U.S. TEACHER EVALUATION • THE RACE TO THE TOP PROGRAM • THE ESEA FLEXIBILITY PROGRAM These two programs reveal that federal authorities definitely know how to employ both the carrot and the stick.

5. DOMINANT TEACHER-EVALUATION THEMES OF THE FEDERAL INITIATIVES • Instructional improvement sought • Three performance levels needed • Multiple evidence sources with student growth being a “significant factor” • Regularly performed evaluations • Clear, timely, and useful feedback • Contributory to personnel decisions

6. FOUR WAYS TO UNDERMINE THIS WELL-CONCEIVED FEDERAL VISION • Using inappropriate evidence of a teacher’s quality • Improperly weighting appropriate evidence of a teacher’s quality • Failing to adjust evidence weights for a given teacher’s instructional setting • Confounding the functions of formative and summative teacher evaluation

7. A MYTH-BUSTING MOMENT The Myth: If a teacher is fired because of a flawed teacher-evaluation system, the unfairly fired teacher can readily get the dismissal overturned by the courts. The Reality: Both federal and state courts have categorically declined to weigh in on the quality of evaluative procedures measuring the performance of teachers.

8. A TURN-AND-TALK TASK Please turn to one or two persons nearby, preferably choosing individuals who appear to be reasonably bright. Then decide, as a group of two or three, whether seriously flawed teacher-evaluation programs will be most harmful to (A) our nation’s teachers, (B) our nation’s students, or (C) our nation.

9. A MYTH-BUSTING MOMENT: The Myth: There is, in truth, such a thing as “the effective teacher.” The Reality: Because of the staggering number of influential variables mucking up the diverse instructional settings in which a given teacher functions, a teacher’s quality turns out to be another instance of “different strokes for different folks.”

10. EVALUATION BASICS: KEY LINGO • Evaluation: Determining the worth of a person, process, or performance • Evaluative Criterion: A factor to be used when determining worth • Evidence: Data or documentation that operationalizes an evaluative criterion • Evidence Sources: All criterion-relevant, potential evaluative evidence

11. DEFENSIBLE TEACHER EVALUATION’S SINE QUA NON If multiple evidence sources must be individually weighted, and those weights may sometimes need to be adjusted when applied to a particular teacher’s distinctive instructional setting, what will always be required for defensible teacher evaluation is: HUMAN JUDGMENT

12. WHAT KIND OF TEACHER-EVALUATION STRATEGY CAN EVER BE DEFENSIBLE? (Mental Drum-Roll, Please!) WEIGHTED-EVIDENCE JUDGMENT

13. WHO DOES THE JUDGING IN WEIGHTED-EVIDENCE JUDGMENT? • Teacher evaluators can range from a solo principal to a review panel consisting of multiple educators. • Ideally, teacher evaluators will be trained, certified, and monitored to increase the likelihood that their evaluative judgments about teachers will be as accurate as possible.

14. WEIGHTED-EVIDENCE JUDGMENT: FIVE STEPS TO EVALUATIVE SPENDOR • Select evaluative criterion (or criteria). • Choose evidence source(s) for each evaluative criterion. • Weight each evidence source chosen. • Decide if any weights need adjustment when applied to a particular teacher. • Coalesce the weighted evidence sources into an overall judgment.

15. WILL THIS APPROACH TO TEACHER EVALUATION EVER INACCURATELY EVALUATE CERTAIN TEACHERS? Of course it will. No teacher-evaluation process will ever be flawless. However, a weighted-evidence judgmental approach to teacher evaluation, because it employs human judgment at key points in the process, will make fewer mistakes than unbending, “cookie-cutter” procedures.

16. A MYTH-BUSTING MOMENT: The Myth: We can accurately evaluate a teacher by observing the teacher teach. The Reality: Although classroom observers can usually spot the super-strong and the super-weak teachers, it is often impossible to differentiate accurately among the very substantial “mysterious middle” group of teachers who have less extreme levels of skill.

17. ANOTHER MYTH-BUSTING MOMENT The Myth: In most instances, ratings of teachers by administrators provide fairly accurate evaluative evidence. The Reality: Because years of experience indicate that most school-site administrators make far too many generosity errors when they evaluate teachers, such ratings are suspect.

18. Typical Administrators’ Ratings of Teachers LOW RATINGS HIGH RATINGS

19. DesiredRatings of Teachers LOW RATINGS HIGH RATINGS

20. Typical Administrators’ Ratings of Teachers Jim’s Methodological “Breakthrough” LOW RATINGS HIGH RATINGS

21. WHAT CAN YOU DO ABOUT THIS? • Learn about key teacher-evaluation issues. (Read Jim’s book, preferably aloud, to friends or strangers.) • Learn about the teacher-evaluation program to which you are closest. • Don’t be deferent to the work of supposed teacher-evaluation experts. • Bust a gut to get a flawed teacher-evaluation program fixed.

22. A TURN-AND-TALK TASK Again, turning to one or two neighbors, the same or different ones—depending on how the previous turn-and-talk task turned out—then imagine that your state’s or a local school district’s teacher-evaluation program is seriously flawed. If you decided you were going to try to help fix it, how would you realistically attempt to do so?

23. TO LEARN ALL THAT ANY SANE PERSON EVER NEED KNOW ABOUT THIS TOPIC Simply consult an entrancing new book published by Corwin Press: Popham, W. James (2013) EVALUATING AMERICA’S TEACHERS: MISSION POSSIBLE?

24. Genial Jim’s e-mail address: wpopham@ucla.edu