eSET : Combined Versus Uncombined Reports

1. Materials linked from the October 11, 2012 Faculty Senate agenda. eSET: Combined Versus Uncombined Reports Stefani Dawn, PhD Assistant Director of AssessmentAcademic Programs, Assessment and Accreditation

2. If Oregon State University solicits or accepts student survey evaluations of the classroom or laboratory performance of a faculty member, the survey evaluations shall be conducted anonymously… (OAR 576-003-0070) Interim Decision Courses with fewer than 6 students will not be evaluated. Vote to occur in November faculty Senate meeting Minimum Class Size for Evaluations

3. Advantages and disadvantages Combining reports preserves student responses Example: 2 section slash course – one has 5 students, one has 4 students – combining would allow the faculty members to receive the evaluations. The numbers: Fall 2011, Winter 2012, Spring 2012 Reporting: Combined or Uncombined

4. Some faculty and departments prefer the combined data others do not Differences in P & T forms/data per department Graduate versus undergraduate Having separate reports and combining data after the fact is not feasible Median calculation is not a standard approach and cannot be done easily in an Excel spreadsheet by the department or individual instructor The Office of APAA does not have the resources to combine reports that have been separated Reporting: Combined or Uncombined

5. Graduate Course eSET Questions • If reports are combined: • All graduate sections of 400/500 courses could have additional questions that can be automatically added for consistency • Allows us to place disclosure language • Would not interfere with ability to have instructor-added questions

6. Graduate Course eSET Questions • These questions would be preceded by a disclosure statement such as: The following questions provide additional information about the course objectives.  In some instances, responses to these questions may make it easier to infer a particular participant’s identity, depending on the number of graduate students in the course. Accordingly, while OSU would like to receive this information, OSU emphasizes responding to the following questions is voluntary.

7. Graduate Course eSET Questions • The course content met my needs as a graduate student. • The course expectations were appropriate for graduate students. • The assessments were at a graduate level. • The assessments matched the course outcomes. • I believe I attained the graduate-level outcomes for this course. These would use the existing likert scale: very poor, poor, fair, good, very good, excellent, unable to rate

8. eSET Timing Pilot • Three options for closing eSET • Before finals • After finals, before grades are released • At the end of grading week with grades hidden until evaluations are complete (incentive for completion) • Have two terms of data for option 3, one term of data for option 2

9. eSET Timing Pilot • Premliminary data show a statistically significant change in response rate (67% to 58%) (p=0.0001) • Would like to obtain another term of data for option 2 (before grading week, after finals)

10. eSET Timing Pilot • Up to two terms of data for closing before finals • Data • Response rates • Scoring trends • Data presented to FS in 2013 to inform vote on timing