“Evaluating Teaching: Beyond the Bubble-In Student Evaluation Form”

1. “Evaluating Teaching: Beyond the Bubble-In Student Evaluation Form” Bernard W. Bopp Director, Center for Teaching and Learning

2. Many ideas and quotes from Raoul Arreola’s book, which I gratefully acknowledge.

3. Where We’re (Hopefully) Headed Today • Review/summarize information on student ratings of teaching • Grapple with a “definition” of teaching • Explore additional approaches for teaching evaluation • Peer evaluations • Student Observers • Teaching portfolios • Recognize obstacles that impede efforts to improve teaching evaluation

4. Summary Advice on Student Evaluation • Summative vs. formative evaluation (don’t mix the two!). • Use student evaluations as one component of teaching evaluation. • Don’t re-invent the wheel! • Consider number of raters and don’t over-interpret! • Consider course level and demographics.

5. Summary Advice on Student Evaluation, cont’d • Use written comments for formative evaluation only. • Defend student anonymity. • Give thought to the process of the evaluation. • Let the students know what changes.

6. “Good Teaching Involves _______” • Content expertise • Instructional delivery skills and characteristics • Instructional design skills • Course management skills

7. Wrestling with Definitions • Content – “The body of skills, competencies, and knowledge…in which the faculty member has received advanced experience, education, or training.” • Instructional delivery – “…human interactive skills…which make for clear communication of information/concepts and promote or facilitate learning…” • Clarity in exposition • Enthusiasm • Ability to motivate, interest, and capture the attention of students

8. Wrestling with Definitions, cont’d • Instructional design skills – “...technical skills in designing, sequencing, and presenting experiences which induce student learning. Designing, developing, implementing learning assessment tools.” • Designing tests and learning outcomes • Developing syllabi, handouts, supporting materials • Properly using instructional technology • Course management skills – skills in operating and managing a course • Timely grading of exams, assignments • Maintaining office hours • Arranging for needed facilities and resources

9. Student Evaluations and Our Four “Defined” Dimensions of Teaching • Content expertise – NO • Instructional delivery – YES • Instructional design – YES, to some degree • Course management - YES

10. Peer Evaluation of Teaching • Appealing idea to many faculty, but with complications and pitfalls • Substantial observation time commitment • Strong positive bias in most peer evaluation systems (esp. summative) • Why Classroom Observations Should Be Avoided • Single visits insufficient • Need multiple visits by two or more peers • Can be strongly affected by interpersonal dynamics and dept. politics

11. What Should Peers Evaluate? • Course materials • Coverage, currency, accreditation standards • Syllabi • Instructional methods • Assessment strategies BUT: Peers need to understand “best practice” issues and not evaluate on opinions, preferences

12. Peer Evaluation and Our Four “Defined” Dimensions of Teaching • Content expertise – YES • Instructional delivery – Perhaps… • Instructional design – YES • Course management - YES, to limited degree

13. Student Observer Eleanor Mays The CTL Student Observer Program • Voluntary and confidential • Formative evaluation (may become part of T&P dossier) • Student Observers are carefully trained • Observers are impartial • Observer reports highlight positives and negatives “[The Observer] listened to my concerns and evaluated my class and offered thoughtful suggestions. Her efforts are greatly appreciated.” - anonymous faculty comment, spring 2006

14. Student Observer Reports and Our Four “Defined” Dimensions of Teaching • Content expertise – NO • Instructional delivery – YES • Instructional design – YES • Course management - YES, to some degree

15. What is a “Teaching Portfolio”? • A factual description of a professor’s teaching strengths and accomplishments • Includes documents and materials which collectively suggest the scope and quality of teaching performance • It is to teaching what lists of publications, grants, and honors are to research and scholarship. - from Seldin, “The Teaching Portfolio”, 2nd ed.

16. Sample Portfolio Items: From Oneself • Sample course syllabi • Reflective statement on teaching philosophy, teaching strategies, objectives, and goals • Instructional innovations, new course projects, new materials (and assess effectiveness) • Future plans/ideas for innovation and change

17. Sample Portfolio Items: From Others • Student teaching evaluation data • Statements from colleagues who have observed teaching or reviewed materials • Honors or other recognition from colleagues • Statements by alumni on teaching quality

18. Sample Portfolio Items: Products of Teaching/Student Learning • Student scores on pre- and post-course exams • Examples of graded student essays with comments • Student publications or conference presentations on course-related work • Successive drafts of student papers along with instructor comments on how each draft could be improved

19. Do We Really Want to Change Our Teaching Evaluation System? • Obstacles • Administrative apathy • Costs, both real and political • Other higher priority issues • Faculty resistance • Nobody likes to be evaluated! • Evaluation by incompetent people? • Being held accountable for performance in an area in which they have little or no training or interest.

20. “College teaching is probably the only profession in the world for which no specific training is required. The profession of scholarship is rich in prerequisites for entry, but not that of instruction.”- Milton, O., and Shoben Jr., E. J. (1968) in Learning and the Professor (Athens Ohio: Ohio University Press)

21. An Observation from Lee Shulman, Carnegie Foundation President: “With regard to the quality of research, we tend to evaluate faculty the way the Michelin guide evaluates restaurants. We ask, 'How high is the quality of this cuisine relative to the genre of food? How excellent is it?' With regard to teaching, the evaluation is done more in the style of the Board of Health. The question is, ‘Is it safe to eat here?’”