Examining an online self-paced learning modules using Usability Testing: What did learners say?

1. Examining an online self-paced learning modules using Usability Testing: What did learners say? Hui-Hsien Tsai Jiazhen Wang University of Missouri

2. Introduction • Using e-learning to deliver training material becomes a trend in business (Neidj & Wolpers,2004). • Higher dropout rates and users easily felt frustrated. • Most Influential factor: poor usability design of e-learning applications

3. Research Questions • To what extent does the information structure and site functionality meet heuristic standards as evaluated by usability experts? • How efficient and effective are learners in finding and retrieving needed training materials as requested in the usability testing protocol? • How successfully do they complete the task? • How much time does it take to complete the task? • How satisfied are learners with the web site as a training and support site?

4. Content & Demographic Data • Content: self-paced Microsoft Excel 2003 e-learning lessons • Demographic data:

5. Methodology

6. HeuristicEvaluation • Adapted from Jacob Nielsen (2005), Deniese Pierotti (2003), and align with the five perspectives of usability in e-learning from Mehlenbacher et al. in 2005. • 12 categories with 84 criteria.

7. HeuristicEvaluation Results • Overall, the Microsoft Excel e-learning lessons only met 57.3% of the heuristic evaluation criteria. • Categories with lowest percentage: • User Control and Freedom (33.3%); • Error Prevention (33.3%); • Help and Documentation (33.3%); and • Pleasurable and Respectful Interaction with the user (37.5%).

8. User Exploratory Test • Purpose: efficiency, effectiveness, and the satisfaction of the Microsoft Excel e-learning lessons from users’ perspectives • 6 major tasks with 14 subtasks

9. User Exploratory Test Results • General Rating: • Easiest: Setting up a Workbook; • Hardest: Performing Calculations on Data; • Task Completion Rate: • Hardest task: Finding the location of the animated instruction in formatting a chart (25%)

10. User Exploratory Test Results(Cont.) • Time on Tasks: • The most time-consuming task: creating a chart in a new worksheet; • The second time-consuming task: Finding the instruction of errors in calculation • Navigation Patterns: • Different navigation patterns; • Participants’ navigation patterns influence their performance and their navigation patternsarebased on their understanding of the website functionality.

11. Interviews • Overall Impression: • In general, all of these participants did not know this website; • Half participants think this website is helpful; • Experienced users think overall of the MELL website is good; • Suggestions from participations: e.g. Increase font size, especially for headings

12. Recommendations • Based on User Exploratory Test and Interviews: • Avoiding the terms that novices are not familiar with, such as smart tags; • Ordering the sequence of lessons, according to novices’ mental models; • Using correct colors, such as headings; • Reducing the white spaces in the main page; • Increasing the font size