1 / 21

Experiments on Education as a Service

Experiments on Education as a Service. Hyoyoung Kim James R. Morrison Department of Industrial and Systems Engineering KAIST. Contents. Motivation Experiments Statistical Analysis Concluding Remarks. Literature Review.

martin-perez
Download Presentation

Experiments on Education as a Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiments on Education as a Service Hyoyoung Kim James R. Morrison Department of Industrial and Systems Engineering KAIST

  2. Contents • Motivation • Experiments • Statistical Analysis • Concluding Remarks

  3. Literature Review [1] Bagchi, U.; , "Delivering student satisfaction in higher education: A QFD approach," Service Systems and Service Management (ICSSSM), 2010 7th International Conference on , vol., no., pp.1-4, 28-30 June 2010, doi: 10.1109/ICSSSM.2010.5530090 [2] Mathew Joseph, Beatriz Joseph, (1997) "Service quality in education: a student perspective", Quality Assurance in Education, Vol. 5 Iss: 1, pp.15 – 21 [3] SangeetaSahney, D.K. Banwet, S. Karunes, (2004) "A SERVQUAL and QFD approach to total quality education: A student perspective", International Journal of Productivity and Performance Management, Vol. 53 Iss: 2, pp.143 – 166 [4] Kay C. Tan and Sei W. Kek, (2004) “Service Quality in Higher Education Using an enhanced SERVQUAL approach,” Quality in Higher Education, Vol. 10, No. 1, pp. 17-24

  4. Motivation Course Experience Better Satisfaction Teaching Better Evaluation Service Service Service Service • Take Home Point: • Good service improves the student experience and increases satisfaction • But what is good service?

  5. Motivation • Identify how to increase customer satisfaction in Education • Experiment 1 • GOAL: Determine Basic, Performance, and Excitement needs • Kano’s model survey • Result: Identified 4 BN, 11 PN, 20 EN • Experiment 2 • GOAL: Determine which services (among excitement needs) make statistically significant difference in student satisfaction • SERVQUAL survey • Result: • Observed drift of service and service memory effect • Numerous services improve/influence student perceptions

  6. Experiment 1 - Spring, 2010 • GOAL: Determine Basic, Performance, and Excitement needs • KAIST IE 200 Introduction to Operations Research class • Class size: 186 students (from all science and engineering majors) • Conducted Kano’s model survey • 76 Functional and Dysfunctional questions • 18 Solution questions • 15 Overall satisfaction questions Functional Dysfunctional

  7. Experiment 1 - Spring, 2010 RESULTS: • Basic Need (Must-be): • Lecture room density is NOT high (large room size with small number of students) • Lecture notes are uploaded at least one day before class • Lecture room is NOT disturbed by outside noise • Class time is well kept Satisfaction Done very poorly Done very well Must-be Dissatisfaction

  8. Experiment 1 - Spring, 2010 RESULTS: • Performance Need (One-dimensional): • Exams and/or homework are directly related to lectures • You feel you learned a lot at the end of the semester • Lecture room is close to dormitory • Professor is confident about lecture contents • Professor enjoys teaching • Professor prepares the lecture well • Professor delivers the lecture clearly • Lecture is easy to understand • Professor respects you • TAs are kind • Lecture notes are easily understandable • Lecture room is clean Satisfaction One-dimensional Done very poorly Done very well Dissatisfaction

  9. Experiment 1 - Spring, 2010 RESULTS: • Excitement Need: • Furniture and classroom equipment are up-to-date • Professor gives candies when answering questions • Professor and TAs provide sufficient and convenient office hours • Professor is humorous • Professor smiles while giving lectures • Professor cares about students • Professor’s appearance is neat • Professor calls students by name • Professor is kind • In-class activities are fun • Course material can be applied in the real world • Lectures make you think Satisfaction Attractive Done very poorly Done very well Dissatisfaction

  10. Experiment 2 - Fall, 2010 • KAIST IE 332 Operations Research IIclass, 60 students (IE majors) • SERVQUAL survey to measure expectation, performance, and importance • GOAL: Determine which excitement needs measurably improve student satisfaction

  11. Experiment 2 - Fall, 2010 • Handling missing data: Regression • Data reliability: Cronbach’s alpha (α) • 0 ≤ α ≤ 1, ↑ α, ↑reliable • Recommended to have a reliability of 0.70 or higher to be used as data • Data of six surveys: 0.89 ≤ α ≤ 0.94 ⇒ can trust data • Correlation • Measure of how much each pair of scores relates • Statistical testing • SAS 9.1 • Wilcoxon signed-rank test (paired sample test) • Kruskal-Wallis test 0.00-0.25: little if any 0.26-0.49: low 0.50-0.69: moderate 0.70-0.89: high 0.90-1.00: very high • non-parametric statistical hypothesis test for the case of two related samples a non-parametric method for testing equality of population medians among groups

  12. Statistical Analysis <Summary> • Perception 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree

  13. Statistical Analysis <Humor> • Input • Jokes and funny/silly examples • Short funny music • Output • Spillover effect – effect of humor lasted longer than other services. • H0: there is no difference or relationship between the variables of interest • If p-value ≥ 0.05 , H0 is accepted → no difference in measurements • Students still remembered humor and rated humor higher on the next survey 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree Accept (p-value: 0.854 ≥ 0.05) Accept(p-value: 0.420 ≥ 0.05) Reject (p-value: 0.013 ≤ 0.05) Reject (p-value: 0.017 ≤ 0.05) Accept (p-value: 0.939 ≥ 0.05) Reject (p-value: 0.003 ≤ 0.05)

  14. Statistical Analysis <Humor> • Outcome 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree Correlation of 0.672

  15. Statistical Analysis <Candy> • Input • Give candy to students when they ask or answer questions • Output • Perceived candy • Drift of needs (Excitement need to basic need): • 2.26/5 → 3.11/5 for the importance of giving candies 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree

  16. Statistical Analysis <In-class Activity> • Input • Skits • Games • Team activity • Active learning activities • Output • Correlation of 0.43324 between in-class activities and confidence Time, effort 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree

  17. Statistical Analysis <Nominal Service> • Outcome Significant change Insignificant change 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree

  18. Statistical Analysis <Full Service> • Input • Candy • Humor • In-class activity • Enthusiasm • Neat appearance • Output 1: Strongly disagree 2: Disagree 3: Neutral 4: Agree 5; Strongly agree

  19. Statistical Analysis < Full Service > • Outcome • Positive • Negative Significant change Insignificant change

  20. Concluding Remarks • Students perceived the offering of such services • We observed needs drift of such services • We also observed the service memory effect • Needs that require further improvement: • Lecture notes are easy to understand • Sufficient and convenient office hours • Feel you learned a lot • Exams and/or homework are directly related to lectures • Future Research: Service increases student learning outcome

  21. Thank You.Any Questions?

More Related