Assessing the Value of e-Learning Systems*

1. Assessing the Value of e-Learning Systems* Yair Levy, Ph.D.Assistant ProfessorGraduate School of Computer and Information Sciences Nova Southeastern University (NSU) * For extensive information about the theory, case study and results please refer to my book: Levy, Y. (2006). Assessing the value of e-learning systems. Hershey, PA: Information Science Publishing. (ISBN#s: 1591407265 HC; 1591407273 SC)

2. Statement of the problem • Currently, most common measures of information system success or IS effectiveness uses user satisfaction only. • However, user satisfaction measures fail as they don’t measure what is valuable for the user. • Most assessment measures are course base rather than program base.

3. Definition of Satisfaction Learner’s Satisfactionis defined as the perceived performance level users find at a post-experience point of time with the e-learning system.

4. Proposed New Approach • The proposed new approach provides extensive empirical evidence that measuring user satisfaction doesn’t give practitioners the accurate picture. • The use of the value construct in conjunction with the commonly used, user satisfaction, provides a true picture of system success or system effectiveness. • The case study was on e-learning systems

5. Definition of Perceived Value Learner’s perceived value is defined as an enduring core belief about the level of importance users attribute to the e-learning system.

6. Theoretical Background • User Satisfaction Theory • IS (Bailey & Pearson, 1983; Ives et al., 1983; Doll & Torkzadeh, 1988;1991; Etezadi-Amoli & Farhoomand, 1991; Sethi & King, 1999) • IS Effectiveness Theory • IS (DeLone & McLean, 1992; Grover et al., 1996; Leidner & Jarvenpaa, 1993; Alavi et al., 1995; Webster & Hackley, 1997) • Value Theory • Psychology (Allport et al., 1951; Kluckhohn, 1951; Feather, 1967; Rokeach, 1969; 1973; 1979) • Marketing (Beatty et al., 1984; Kahle et al., 1986) • IS (Keeney & Raiffa, 1976; 1993; King & Epstein, 1983; Keeney, 1999)

7. Overview of the Proposed Research Model Current Theory

8. The Misleading Measure… SATB SATA Program B Program A

9. The New Measure… VALB VALA SATB SATA Program B Program A Effectiveness = Satisfaction * Value

10. The Misleading Picture… Not so Effective Very Effective

11. The New Picture…

12. Value-Satisfaction Grid for e-Learning Systems

13. Effectiveness Curves of e-Learning Systems

14. Effectiveness Grid of e-Learning Systems (Cont.)

15. Characteristics from Literature & Focus Groups in Webster and Hackley’s (1997) Dimensions

16. Grid Results Dimension A - Technology & Support (n=192) *** Low/High cut-of @ 75% of Scale Maximum (6) ***

17. Grid Results (Cont.) Dimension B - Course (n=192) *** Low/High cut-of @ 75% of Scale Maximum (6) ***

18. Grid Results (Cont.) Dimension C – Professor (n=192) *** Low/High cut-of @ 75% of Scale Maximum (6) ***

19. Grid Results (Cont.) Dimension D – Learner (n=192) *** Low/High cut-of @ 75% of Scale Maximum (6) ***

20. Grid Results (Cont.) - Submit assignments remotely Professor - Learning at anytime -Internet & Comp Skills Freedom of learning - Travel & Learn - Family support E-mail Support ISP & Internet Costs - Phone Support Classmates

21. Overview of Learners’ Value Index of Satisfaction (LeVIS) LeVIS → 0 indicates very low learners’ implied OLS effectiveness LeVIS → 1 indicates very high learners’ implied OLS effectiveness

22. Index Results

23. Index Results (Cont.)

24. Index Results (Cont.)

25. Index Results (Cont.)

26. Index Results (Cont.)

27. Conclusions & Contributions • Knowing the value, or the level of importance, learners find in each characteristic of the system along with the level of satisfaction allows administrators of e-learning programs to find the areas in most need to improvement • The empirical validation of the survey to assess value along side satisfaction of e-learning systems • The validated set of benchmarking tools (the Value-Satisfaction grids and LeVIS index) helps administrators of e-learning programs realize the key eff. characteristics of their program.

28. Conclusions & Contributions (Cont.) • The book provides a set of benchmarking tools such as the Value-Satisfaction grids and LeVIS index to help administrators of e-learning programs realize the key effective characteristics of their program. • The book provides a “cook book” guidelines approach on how to implement the proposed theory and tools in your own e-learning program.

29. Thank you! • Questions?

30. http://www.ijds.org/ Mission: The mission of the International Journal of Doctoral Studies (IJDS) is to provide readers worldwide with high quality peer-reviewed scholarly articles on a wide variety of issues in doctoral studies using the Informing Science (IS) framework. The editorial objective of IJDS is the facilitation of knowledge enhancement related to doctoral studies in areas such as (but not limited to): informing science, information systems, information technology, information science, information security, and IT education. Yair Levy, Editor-in-Chief (editor@IJDS.org) International Journal of Doctoral Studies (IJDS)

31. Contact Information - Levy Yair Levy, Ph.D.Assistant ProfessorNova Southeastern UniversityGraduate School of Computer and Information SciencesThe DeSantis Building - Room 40583301 College AvenueFort Lauderdale, FL 33314Tel.: 954-262-2006        Fax: 954-262-3915E-mail: levyy@nova.edu Site: http://scis.nova.edu/~levyy/