180 likes | 263 Views
From Creating the Form to Analyzing the Results:. Anatomy of a Campus-Wide Instructor Evaluation. Julie E. Thomas, Ph.D. Central Texas College. Jump!. Background. Items drafted by a faculty committee Draft reviewed by IE office Form approved by faculty senate. Go to 7. Preliminaries.
E N D
From Creating the Form to Analyzing the Results: Anatomy of a Campus-Wide Instructor Evaluation Julie E. Thomas, Ph.D. Central Texas College Jump!
Background • Items drafted by a faculty committee • Draft reviewed by IE office • Form approved by faculty senate Go to 7
Preliminaries • Download course listing • Delete classes with <10 enrolled • Exclude labs (except nursing, EMT) • Sort by department, then instructor • Send course list to department chairs to verify accuracy
Sample definition • Classes are sampled units • Random sample with restrictions • Instructors may switch with permission • Overall sample is slightly more than half of courses meeting criteria
Sample Selection • First stage: randomly select one class for every instructor • Second stage: randomly select a group from the remaining classes • Third stage: delete classes if more than three for an instructor; substitute courses where necessary
Issues & Accommodations • Course listings don’t tell the whole story! • Team-taught courses • Last-minute instructor changes • Special requests
Customizing the form • Teleform edit menu: Copy page to clipboard • Paste into MS Word as a picture • Overlay with text boxes • Insert merge fields in text boxes • Course listing file is used as data file for mail merge Go to 11
Other details • Mailing labels for the evaluation packets • Keep the same sort order! • Copies of forms will work, but try to avoid second-generation copies • Packets distributed by campus mail
Receiving Completed Surveys • Completed forms collected by department offices • Returned to IE office when all forms have been received by department • BOCTAE!
Scanning Forms • Excel file produced by Teleform • May also be set to generate other types of data files • Class ID is included on each form: used to match results with course listing file
Compiling & Analyzing Results • Statistical software such as JMP is very helpful • Data summary • One-to-many and many-to-one sorting • Can be done entirely in Excel! (not the most efficient way) • Trimmed means used for class summary data
Data Reported • A page of results in generated for each class, including for each item: • Class means (20% trimmed) • Frequency distributions (per class) • Comparison data
Comparison Data • Instructor means • Course means • Discipline means • Department means • Campus Means • All based on weighted averages of class trimmed means
Reporting • Mail merge again! • Automatically produces a one-page report for each class evaluated • To fix formatting, global find-and-replace commands are used
Individualized Reports • Each faculty member receives summary pages of his/her own classes • Department chairs receive the summary pages for classes in their departments • Complete report distributed to a very limited audience • “No-names” edition is more widely available
Custom Reports • Special requests can be accommodated as time permits • Standard deviations • Rank ordering • Graphical reports
QUESTIONS? • Julie.Thomas@ctcd.edu • Central Texas College • PO Box 1800 • Killeen, TX 76540 • 254 526 1484 • To view the latest “no-names” edition: http://www.ctcd.edu/ie/seci_spring2005.pdf