200 likes | 295 Views
Student Satisfaction with Online Courses. Amber Settle, CTI, DePaul University joint work with Chad Settle, University of Tulsa CCSC: Southeast Region November 11, 2005. Satisfaction with distance learning. Distance learning is popular with CTI students
E N D
Student Satisfaction with Online Courses Amber Settle, CTI, DePaul University joint work with Chad Settle, University of Tulsa CCSC: Southeast Region November 11, 2005
Satisfaction with distance learning • Distance learning is popular with CTI students • There are 8 M.S. and 1 M.A. degree online (out of 10 M.S., 1 M.A., and multiple joint degrees) • Distance learning students are 21% of the student population • It has been asserted that while outcomes are similar in DL and traditional classes, DL classes are less satisfying to students (Carr 2000) • Is DL less satisfying for CTI students? If so, how?
The test cases • The courses • Foundations of Computer Science (CSC 415) • Graduate discrete mathematics • 9 sections between Fall 2001 and Fall 2003 • Programming in Java I and II (CSC 211 and CSC 212) • Java I: 8 sections between Fall 2003 and Winter 2005 • Java II: 7 sections between Spring 2003 and Fall 2004 • The format • Traditional • Sibling DL: Runs parallel to a traditional class; entire classroom interaction is recorded automatically • Pre-recorded DL: High production quality independent of any live class; broken into five modules (CSC 415 only)
Course evaluations • Conducted every quarter for every CTI course • Mandatory for all students • Online using secure login; anonymous • Completed during the 8th and 9th week of 10 week quarter • Results are withheld from instructor until grades are submitted; results are then published on the CTI web site • Consists of 22 multiple choice questions • 10 questions about course-related factors; 12 questions about instructor-related factors • Ratings on a scale from 0 to 10; a higher number indicates greater satisfaction; 0 indicates the question is not applicable
Summary of CSC 415 results • Similarities in evaluations: Overall scores • None of the coefficients for instructor-related or course-related questions were significantly different from 0 for DL sections • Differences in evaluations: Not applicable response rate • Q-IR5 (Encourage participation) • DL: 60% • Traditional: 6% • Q-IR12 (Teaching effectiveness) • DL: 15% • Traditional: 2.5%
Potential explanations • Pre-recorded DL is better organized which compensates for the lack of interaction (Swan 2001) • DL students are not watching the recordings • Small DL sample size
Summary of Java results • Course-related questions • Two out of 10 questions (Q-CR1: Course organization, Q-CR2: Achieving course objectives) were statistically different from 0 for DL students • Both coefficients were negative, indicating less satisfaction • Similar results were not seen in an analysis of only the live sections • Instructor-related questions • Nine out of 12 questions (all but Q-IR4, Q-IR7, Q-IR10) were statistically different from 0 for DL students • All coefficients on the 9 questions were negative (from -0.3 to -0.8) • For live students only two questions were significant over time; one was positive (Q-IR 5: student participation), the other negative (Q-IR 6: availability) • Sibling DL students are not as satisfied with the instructor as traditional students or students in live sibling sections • Q-IR 12 (comparison to other instructors) is used for T/P and merit • Not applicable rate was fairly consistent between sections; only differences on Q-IR5 (7% for DL, < 1% for live) and Q-IR7 (6% for DL, < 1% for live)
Potential explanations • Larger data set • DL students are more sensitive to course organization because of the lack of interaction • A more interactive sibling course will highlight the shortcomings of the class for DL students • Watching the students in the sibling section participate • Listening to instructor questions with no chance of responding • The course evaluations do not allow evaluation of the course delivery technology, causing the instructor ratings to suffer
Appendix • Course evaluation questions • Course-related questions • Instructor-related questions • Statistical analysis • Least squares regression • Data for CSC 415 • Data for Java I and II
Course-related questions • Was this course well organized? • Do you feel the course objectives were accomplished? • The amount of work you performed outside of this course was: • How difficult was this course material? • The textbook for this course was: • Supplementary reading for this course was: • The assignments for this course were: • What is your overall estimate of this course? • How valuable was this course in terms in your technical development? • Would you recommend this course to another student?
Instructor-related questions • How would you characterize the instructor’s knowledge of this subject? • How would you characterize the instructor’s ability to present and explain the material? • Does the instructor motivate student interest in the subject? • How well does the instructor relate the course material to other fields? • Did the instructor encourage participation from the students? • Was the instructor accessible outside of class?
Instructor-related questionscontinued • What was the instructor’s attitude? How did he/she deal with you? • How well did the instructor conduct, plan, and organize classes? • Were the instructor’s teaching methods effective? • How fair was the grading of the homework and exams of this course? • Would you take this instructor for another course? • Rate the teaching effectiveness of this instructor as compared to other faculty in the department.
Statistical analysis • Ordinary least squares regression: • X2i = 0 for traditional, X2i = 1 for DL • If 2 is statistically different from 0, it indicates a difference in how DL students view the course vs. traditional students Qi = 0 + 1 X1i + 2X2i + ui Time DL Error Question i Constant
CSC 415: Course-related results Coefficient estimates are presented with standard errors in parentheses. *Statistically significant at the 10% level of a two-tailed test. **Statistically significant at the 5% level on a two-tailed test. ***Statistically significant at the 1% level of a two-tailed test.
CSC 415: Instructor-related results Coefficient estimates are presented with standard errors in parentheses. *Statistically significant at the 10% level on a two-tailed test. **Statistically significant at the 5% level on a two-tailed test.
Java: Course-related results Coefficient estimates are presented with standard errors in parentheses. *Statistically significant at the 10% level of a two-tailed test. **Statistically significant at the 5% level on a two-tailed test. ***Statistically significant at the 1% level of a two-tailed test.
Java: Instructor-related results Coefficient estimates are presented with standard errors in parentheses. *Statistically significant at the 10% level of a two-tailed test. **Statistically significant at the 5% level on a two-tailed test. ***Statistically significant at the 1% level of a two-tailed test.