What do you know about IDEA?

1. What do you know about IDEA? Student Learning Model

2. Welcome The IDEA SystemOverview and Coaching Tool Agenda and Purpose Introductions

3. IDEA as a Diagnostic to Guide Improvement Benefits: A Tool to Improve Student Learning and Teaching Effectiveness

4. Purpose of Presentation • To provide a practical model for colleges • Get us on the “same page” • Describe The IDEA Center’s underlying assumptions about the use of student ratings • Review IDEA - Student Ratings of Instruction system • FIF • Selecting objectives • Diagnostic Survey • Diagnostic Form Reports • Administration • Resources • To share Best Practices

5. Improvement of Student Learning • Student Ratings can have a positive impact if... • The instrument • Is “learning focused” • Provides a diagnostic • The emphasis for “summative” faculty evaluation is appropriate • 30%-50% of the overall evaluation of teaching • Results are not over-interpreted • Faculty trust the process • 6-8 classes-more if small <10

6. What does IDEA stand for? Individual Development and Educational Assessment

7. Improvement of Student Learning • Campus and college culture has great impact • Is teaching excellence a high priority? • Are student ratings given an appropriate weight in the overall evaluation of teaching?

8. Improvement of Student Learning • Does the institution provide resources to aid those seeking to improve their instructional skills? • Teaching and Learning Centre • Department Chair/Faculty Development Director • Mentor Faculty • Brown bag sessions • IDEA Resources • Is credible support for improvement available?

9. Underlying Assumptions • Students are not qualified to assess: • Faculty expertise • Appropriateness of goals, content, and organization of course • Materials used in delivery • How student work is evaluated, including grading practices

10. Underlying Assumptions • Nor are they qualified to assess “indirect” contributions to instruction • Support for departmental efforts • Assistance to colleagues • Contributing to a positive atmosphere

11. Other Sources of Evidence for Evaluation • Self-evaluation/reflective statement • Course materials • Graded appraisal tools • tests, essays, papers, etc. • Feedback from mentors/colleagues • Peer observations • Classroom assessment/research efforts • Teaching portfolio • Faculty Development Plan

12. IDEA Student Ratings of Instruction The Student Learning Model

13. Student Learning Model • Types of learning must reflect instructor’s purpose • Effectiveness is determined by student progress on objectives stressed by instructor • Specific teaching behaviors influence certain types of student progress under certain circumstances.

14. IDEA Student Ratings of Instruction Overview Faculty Information Form Student Survey Diagnostic Form Short Form

15. IDEA: FIF Faculty Information Form Handout #1

16. Faculty Information Form FIF

17. 12 Learning Objectives

18. FIF: Selecting Objectives • 3-5 as “Essential” or “Important” • Is it a significant part of the course? • Do you do something specific to help students accomplish the objective? • Does the student’s progress on the objective influence his or her grade?

19. Relevant Objectives • Basic Cognitive Background • Items 1, 2 • Applications of Learning • Items 3, 4 • Expressiveness • Items 6, 8

20. Relevant Objectives • Intellectual Development • 7, 10, 11 • Lifelong Learning • 9, 12 • Team Skills • 5

21. Course Description Items • Optional • Best answered toward end of term • Do NOT influence your results Bottom of Page 1 Top of page 2

22. Student Survey Diagnostic Form Handout # 2

23. Diagnostic Survey Form

24. Diagnostic Student Survey: • Specific teaching behaviors… • Teaching Methods: Items 1-20 • ...influence certain types of student progress… • Learning Objectives: Items 21-32 • …under certain circumstances. • Student and Course • Student Characteristics: Items 36-39, 43 • Course Management/Content: Items 33-35 • Others • Summary Measures: Items 40-42 • Experimental Questions: Items 44-47 • 20 Additional Questions

25. Student Survey • Using extra questions • Comments • Constructive • How to become a better teacher • Administration’s role

26. False Assumptions • Effective instructors effectively employ all 20 teaching methods. • The 20 teaching methods items are used to make an overall judgment about teaching effectiveness. • Students should make significant progress on all 12 learning objectives

27. Best Practices Faculty • Know the objectives before the course starts • Day 1 • Discuss meaning of objectives with students • Inform them they will be asked to rate their own progress on objectives • Reflect on their understanding of course purpose • Discuss differences in perception of objectives’ meaning • Throughout the term - reinforce

28. Best Practices • Department • Multi section courses • Coaching, feedback and follow-up • Department • Institution • The scorecard • Excellence of teaching • Progress on learning • Recognition

29. Best Practices for Online • Know the objectives prior to the start of the class • Incorporate into your announcements, discussions, email, chat • Follow-up in the discussion thread and use “Kim’s” post to illustrate the objective

30. Let’s Review • How many learning objectives should you select? • What is one way to communicate your expectations for learning with students? • Were there any new key points in this section?

31. Report Overview-Handout #3

32. Interpreting the IDEA Report Administrative Decisions

33. The IDEA Report • Diagnostic Form Report • Administrative Decisions • What were students’ perceptions of the course and their learning? • Pages 1 & 2 • Professional Improvement • What might I do to improve my teaching? • Page 3

34. Page 1

35. Page 1: Global Assessment • Weighted average of Progress on instructor selected (“relevant”) objectives • Overall, I rate this instructor an excellent teacher • Overall, I rate this course as excellent

36. Summary Evaluation of Teaching Effectiveness

37. Adjusted Scores • Control for factors beyond instructor’s control • Regression equations

38. Adjusted Scores • Diagnostic Form • Student Work Habits (#43) • Student Motivation (#39) • Class Size (Enrollment, FIF) • Student Effort (multiple items) • Course Difficulty (multiple items) • Short Form • Adjusted for 3 factors

39. Summary Evaluation of Teaching Effectiveness: Adjusted Scores

40. Understanding Adjusted Scores

41. Page 2: Student Progress • Report of progress on “Essential” or “Important” objectives • Student perceptions of course demands • Student self-assessment of motivation, academic effort, and work habits

42. Page 2: Progress on Specific Objectives 4.5+4.5 4.4+4.4 4.2+4.2 +3.2 7

43. Page 2: Description of Course and Students

44. Page 3: Diagnostic – Typically Used Only Formatively • The diagnostic component • Teaching methods to consider employing more frequently to support learning • Not intended for making summative judgments

45. Page 3: Improving Teaching

46. Page 4: Statistical Detail • Statistical Detail • Additional Questions

47. Using IDEA Formative Use

48. The Big Picture Page 1

49. Progress on Learning Page 2

50. Teaching Opportunities Page 3