1 / 29

Understanding and Using Your IDEA Evaluation Results

Understanding and Using Your IDEA Evaluation Results. Nina Campanicki & Ernie Linsay Faculty Development Day March 20, 2010. What is IDEA?. INDIVIDUAL DEVELOPMENT AND EDUCATIONAL ASSESSMENT. IDEA Evaluation Components:. Faculty Information Form Student Survey Diagnostic Form

luna
Download Presentation

Understanding and Using Your IDEA Evaluation Results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Understanding and Using Your IDEA Evaluation Results Nina Campanicki & Ernie Linsay Faculty Development Day March 20, 2010

  2. What is IDEA? INDIVIDUAL DEVELOPMENT AND EDUCATIONAL ASSESSMENT

  3. IDEA Evaluation Components: • Faculty Information Form • Student Survey • Diagnostic Form • Individualized Diagnostic Report

  4. Faculty Information Form(FIF)

  5. Faculty Information Form (FIF)IDEA Objectives

  6. Faculty Information Form (FIF)IDEA Objectives: • 3-5 “Essential” or “Important” should reflect most significant aspects of course • WU has specific IDEA objectives set in advance for each course

  7. Faculty Information Form (FIF)Contextual Questions There are 6 Contextual Questions at the bottom of front page and the top of back page: Please take a moment to fill in this information. The IDEA Center uses this information to improve the interpretation of student ratings.

  8. Diagnostic Student SurveyLong Form

  9. Diagnostic Student Survey • Specific teaching behaviors… • Teaching Methods are Items: 1-20 • …influence certain types of student progress… • IDEA Learning Objectives are Items: 21-32 • …under certain circumstances. • Course Management/Content are Items: 33-35 • Student Characteristics are Items: 36-39, 43 • Other Data • Summary Measures are Items: 40-42 • Experimental Questions are Items: 44-47 • Extra Questions are Items: 48-67

  10. Diagnostic Student SurveyFalse Assumptions • Effective Instructors employ all 20 teaching methods • The 20 Teaching Methods Items are used to make an overall judgment about teaching effectiveness • Students should make significant progress on all 12 learning IDEA objectives

  11. Diagnostic Student SurveyBest Practices • Overall: Students need to be informed about the survey and the IDEA objectives should be discussed throughout the course.

  12. What Does Your IDEA Report Mean? What were my students’ perceptions of the course and their learning? Overall, how effectively was this class taught? How do I compare with other teachers? Was I more successful in facilitating progress on some IDEA objectives than on others? What might I do to improve my teaching?

  13. Info on Page 1 • Number of students responding • Average scores – raw & adjusted • Converted scores • Comparison to data bases Sample DF

  14. Impact of Response Rates • Number responding <10 Unreliable 10-14 Marginal reliability 15-24 Fairly reliable 25-39 Reliable >40 Highly reliable • Representativeness of Results <50% inadequate to assure representativeness 50-64% may not be representative 65-74% probably representative 75% + representative Sample DF

  15. Adjusted Scores Level Playing Field • Fr/Soph/Gen Ed classes • Students with lower motivation • Adjusted scores > raw scores • Jr/Sr/Grad classes • Students are more motivated & better work habits • Adjusted scores < raw scores Sample DF

  16. Adjusted Scores • Student Work Habits (#43) • Student Motivation (#39) • Class Size (Enrollment, FIF) • Student Effort (multiple items) • Course Difficulty (multiple items) Sample DF

  17. Work Habits (Item 43) Student Motivation (Item 39) Low Low Avg. Avg. High Avg. High Low 3.51 3.66 3.80 3.95 4.08 Low Avg. 3.60 3.76 3.91 4.05 4.07 Average 3.73 3.87 4.02 4.12 4.21 High Avg. 3.88 3.97 4.13 4.23 4.33 High 4.01 4.12 4.25 4.33 4.48 Impact of Extraneous Factors • Gaining Factual Knowledge – Average Progress Ratings Technical Report 12, page 40

  18. Converted Scores • 5-point ratings converted to T-scores with “50” being the middle • Not percentiles • Gray area (45-55T) represents 40% of faculty Sample DF

  19. Comparative Information • All classes in IDEA database • 128 institutions • Updated only periodically • IDEA discipline database • Most recent 5 yrs • Updated annually on Sept 1 • Minimum of 400 classes • Institution (WU) database – all classes • Most recent 5 yrs • Updated annually on Sept 1 • Minimum of 400 classes Sample DF

  20. Example

  21. Info on Page 2 • Student ratings of learning on Essential and Important IDEA objectives • Description of course and students Sample DF

  22. Calculating “Progress on Relevant IDEA Objectives” Essential objectives have double weight of Important objectives Gaining factual… 4.5 X 2 = 9.0 Learning fundamental… 4.4 X 2 = 8.8 Learning to apply… 4.2 X 2 = 8.4 Learning how to find… 3.2 X 1 = 3.2 29.4/7 = 4.2 Sample DF

  23. Info on Pages 3 & 4 Page 3 • Improving teaching effectiveness • What might you do differently Page 4 • Statistical details Sample DF

  24. Personal Comments • IDEA results can spike up or down • Grades do not influence IDEA results • Knowledge Base in IDEA Center website has great info about teaching & learning and understanding the report • www.theideacenter.org • Item #3 in Web Resources for Teaching in Faculty Forum under Faculty in WU home page Sample DF

  25. IDEA Evaluation Process • How and When do you Receive the IDEA Evaluations? • You will receive a packet in your mailbox or via email approximately 2 weeks before your course ends.

  26. IDEA Evaluation Process • What is the Hard Copy Procedure? • Select a student to proctor the evaluations. • They will hand out and collect the Diagnostic and Comment forms. • Take your FIF out of the packet and… • Write Your Name, Course Number and Time/Day information on the Board for the students • Tell the Student/Proctor where you will be and… • Leave the room

  27. IDEA Evaluation Process • When the Student comes out of the classroom • Ask them to place your FIF into the envelope • Direct the Student to the Nearest attended Workstation or Drop-off Box • You are Not Allowed to Handle the Packet after the Students have Completed the Evaluations. • Do not let the Student leave your Evaluation Packet on an unattended desk

  28. IDEA Evaluation Process • What’s Next: • All IDEA Evaluations are sent to New Castle for processing • IDEA Evaluations are grouped: • By College • By Instructor (alpha-order)

  29. IDEA Evaluation Process • Then: • They are packaged and sent to IDEA Center. • Results returned from IDEA Center within 14 days • Distribute results to the Deans within 3 days • Deans and Program Coordinators review and send you the Individual IDEA Reports • You can discuss your IDEA Report with your Program Coordinator or Faculty Development & Support

More Related