1 / 22

Interpreting IDEA reports

Interpreting IDEA reports. Diagnostic Form Short Form www.lvc.edu/idea. Page 1 of report. Page 1 of report. Guide says: “Report Responds to These Questions”. 1. Overall, how effectively was this class taught? 2. How does this compare with the ratings of other teachers ?

peggy
Download Presentation

Interpreting IDEA reports

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interpreting IDEA reports Diagnostic Form Short Form www.lvc.edu/idea

  2. Page 1 of report

  3. Page 1 of report

  4. Guide says: “Report Responds toThese Questions” 1. Overall, how effectively was this class taught? 2. How does this compare with the ratings of other teachers? 3. Were you more successful in facilitating progress on some class objectives than on others? 4. How can instruction be made more effective?* 5. Do some salient characteristics of this class and its students have implications for instruction? * Diagnostic Form Report only.

  5. Our Modest Ambitions Today • Understand summative scores • Raw averages • Converted scores • Adjusted scores • How used at LVC • Formative use of p.2 – relevant objectives • POD papers on objectives • Formative use of p.3 for Diagnostic only • POD papers on strategies

  6. Summative scores ….

  7. How “Summary Evaluation” Computed • 41. Overall, I rate this instructor an excellent teacher. (25%) • 42. Overall, I rate this course as excellent. (25%) • Objectives 1-12 (21.-32) that are ranked as Important or Essential (50%)

  8. “Converted averages” • Transform group’s mean to 50, s.d. to 10. • Examples: • Score 57 means .7 group s.d.’s above average • That’s at 75.8 %-tile of Normal Distribution • … but these groups of IDEA scores not Normal

  9. Adjusted Scores:Effects that are removed P. 2 of Diagnostic Form report:

  10. Adjusted Scores:Effects that are removed P. 2 of Diagnostic Form report: Only these on Short Form

  11. Is unadjusted Score on Progress toward Objectives high? (≥4.2) • Are • adjusted • scores lower or • higher than • unadjusted • scores? Lower Yes Higher Use adjusted scores Use unadjusted (raw) scores When to Use Adjusted Scores for Administrative Decisions: Where is there evidence of effective teaching? (old) No *Expectations defined by your unit.

  12. Do unadjusted (raw) scores meet or exceed expectations?* • Are • adjusted • scores lower or • higher than • unadjusted • scores? Lower Yes Higher Use adjusted scores Use unadjusted (raw) scores When to Use Adjusted Scores for Administrative Decisions: Where is there evidence of effective teaching? (new) No *Expectations defined by your unit.

  13. College Use for Evaluating Faculty • Current: • Summative: Pay attention to extremes • Assess formative use of IDEA by instructor • IDEA recommendation • Summative: Establish standards • Faculty set standards • Department level or College level • Assess progress in meeting them

  14. Page 2: Objectives

  15. P. 3 of Diagnostic Form Report:Directions…. Follow the links… www.theideacenter.org/podidea

  16. Teaching Methods Relevant toTarget Objectives

  17. Teaching Methods Relevant toTarget Objectives

  18. Try it 1. Identify objective… 2. Find related teaching method… 3. Follow link: www.theideacenter.org/podidea

  19. Last page: Raw response frequencies… especially additional questions We can retrieve the additional questions so you know which are which.

  20. Effects of Evals being On-Line? • Study of on-line vs paper by IDEA published in 2010: • http://www.theideacenter.org/sites/default/files/Technical Report 16.pdf • Online avg response rate 57% • LVC response rates 70-75% • No significant change in means or s.d.’s of responses • If loss of middle group, s.d. would go up. • On 12 objectives, s.d.’s slightly lower for on-line!

  21. Key questions for LVC (from IDEA) • How much emphasis will you give student ratings in the evaluation of teaching? • How many classes need to be rated? [LVC: all] • Performance categories: How many should you use? • Do you plan to authenticate objectives? • Will you weight the importance of items or groups of items in the decision-making process? If so, how? • Will you use averages on a five-point scale or converted averages? • Will you use adjusted or unadjusted scores?

  22. References • www.lvc.edu/idea – Campus IDEA page • POD notes: • The 20 methods: www.theideacenter.org/podidea • The 12 objectives: www.theideacenter.org/podidea/PODNotesLearning.html • IDEA: www.theideacenter.org • Technical: http://www.theideacenter.org/sites/default/files/techreport-12.pdf - Data about adjusting, etc.

More Related