eSET

1. eSET Stefani Dawn Assistant Director of Assessment Office of Academic Program, Assessment and Accreditation

2. Decisions for Faculty Senate HIGHEST PRIORITY DECISIONS Minimum class size for evaluations Combined or uncombined reporting eSET-RELATED ITEMS and PROCESSES Demographic data collection Courses to include or exclude in eSET Financial, resource and technological limitations of eSET to be configured for other evaluation approaches (formative evaluations or department-centric evaluations)

3. Minimum Class Size for Evaluations OAR 576-003-0070 If Oregon State University solicits or accepts student survey evaluations of the classroom or laboratory performance of a faculty member, the survey evaluations shall be conducted anonymously… History Requests were made to have separate reports for slash courses. That was implemented last term and quickly put on hold when it was discovered that some of the cross listed sections had only a few students in it, even if all of the sections combined was a large number. Took the issue to the Faculty Senate EC to make some interim decisions to ensure that the OARs are upheld.

4. Minimum Class Size for Evaluations Interim Decision Courses with fewer than 6 students will not be evaluated. OSU guidelines University of Oregon does not conduct evaluations in courses with 5 or fewer students. Question – What should the number be to protect student anonymity? Factors to Consider Student input

5. Reporting: Combined or Uncombined There are advantages and disadvantages to combining reports. Combining reports preserves data Example: 2 section slash course – one has 5 students, one has 4 students – combining would allow the faculty members to receive the evaluations. Some faculty and departments prefer the combined data others do not Differences in P & T forms/data per department Graduate versus undergraduate Having separate reports and combining data after the fact is not straight forward Median calculation is not a standard approach and cannot be done easily in an Excel spreadsheet by the department or individual instructor The Office of APAA and Enterprise Computing does not have the resources to combine reports that have been separated.

6. The Median Calculation Selected by the Faculty Senate in 2004 Based on "Measurement and Evaluation in Psychology and Education", Fourth Edition, by Robert L. Thorndike and Elizabeth P. Hagen. This calculation assumes that multiple values within a range are distributed evenly over the range.  Say you have the following distribution: Score     Freq       Cumulative Frequency 6              29           106 5              29           77 4              38           48 3              10           10 The total responses is 106, so half of that is 53. 53 falls between the cumulative frequency of 48 and 77 corresponding to the scores 4 and 5. 53-48 = 5 more cases in the “5” range are needed.  Since there are 29 responses for “5”, we need 5/29 of them or .20 (rounded from .17). We look at the range 4.5 – 5.5 and add .2 to 4.5 to get 4.7 as the median. 

7. Demographic Data Previous demographic data collected: Students are still potentially identifiable by: Gender class status (e.g. the only graduate student in an undergraduate course) Grade: Audit

8. Demographic Data Factors to Consider Do faculty use the data? Survey fatigue by the students

9. Everything is An “If-Then” If you decide to combine reports: then how will that impact departmental processes for P & T? If you decide to have separate reports: Then what data will be lost? (Which is more important the data or the separate reports?) Saving data and accumulating it over time is not an option for two reasons OAR 576-003-0070 - Survey instruments from which evaluation data are obtained shall be delivered to the faculty member. There are over 4000 courses and >1500 faculty evaluated per term. Tracking this types of data is not feasible with the current FTE arrangements. Then how would reports be recombined (e.g. a person has 7 different slash versions of a course)? Could change the median calculation to a standard calculation If you decide to keep the demographic data Then how might it compromise anonymity? Is it information really used and needed? Are there potential issues in the way questions are asked and student response to those questions? Then how might it impact survey fatigue?