Multimedia Usability Testing

1. Multimedia Usability Testing C E Russell

2. Usability • Making computer systems easy to learn and easy to use, • through a user-centred design process (Preece et al., 1994). • Poorly designed computer-based interactive multimedia systems can be extremely annoying to users. • "…capability in human functional terms to be used easily and effectively by the specified range of users, given specified training and user support, to fulfil the specified range of tasks, with the specified range of environmental scenarios." CER

3. Evaluation • Formative evaluation is used to obtain data to guide instructional material revision or improvement. • Cronbach (1963) defined evaluation as: • "…the collection and use of information to make decisions about an educational program. … Course improvement: deciding what instructional materials and methods are satisfactory and where change is needed…" CER

4. User-Centred Design • an approach which views knowledge about users and their involvement in the design process as a central concern (Preece et al., 1994). • implies that good system design depends upon solving the dynamic interacting needs of: users, tasks, productivity, usability (Dumas & Redish, 1993). • Gould and Lewis (1985) describe three principles for user-centred design: • (a) early focus on users and task; • (b) empirical measurement of product usage; • (c) iterative design in the production process. CER

5. Usability Testing • "a measure of the ease with which a system can be learned or used, its safety, effectiveness and efficiency, and attitude of its users towards it" (Preece et al., 1994) • usability testing can be used to address and solve a system’s usability problems before it goes into production. • provides a means of identifying problem areas • The best way to carry out usability testing is to watch and listen to real users, under real situations interfacing with a multimedia program. CER

6. Maxims of Usability (Reed, 1992) • design for the software end user, not for the designers/clients; • test the multimedia software, not the user; • test usability with real users early and often; • don’t test everything at once; • measure performance of real-world tasks with software, not functionality of the program; • test usability problems that software designers never imagined. CER

7. Learnability • "the ease with which new or occasional users may accomplish certain tasks" (Lindgaard, 1994) • multimedia systems should be easy to remember CER

8. Performance Effectiveness Measured in terms of • Speed • Error • Design to achieve a high level of productivity. CER

9. Flexibility • Variations in task-completion strategies • Provide a range of different commands CER

10. Error Tolerance and System Integrity • When errors do occur, users should easily recover from them. • System Integrity if the prevention of data corruption or loss. CER

11. Multimedia software should be; enjoyable to use, aesthetically pleasing to users. User satisfaction should be within acceptable levels of user cost in terms of; tiredness, discomfort, frustration, and individual effort. Motivational elements can motivate the user and increase satisfaction. They include; typographical cueing, colour, graphical images, animation, sound User Satisfaction CER

12. Table of Dimensions of Usability Testing CER

13. Advantages Easy to learn and use. Inexpensive to implement. To identify problems early in the design process. Limitations Debriefing session is necessary to find the indication of how to fix problems. Methods/Techniques of Usability Testing -Heuristic Evaluation Methods: To use a predefined list of heuristics to find usability problems. CER

14. Advantages Easy to learn and use. To allow iterative testing. To meet the criteria of all parties involved in the test. Limitations Difficult to find a proper context of task-performed for usability testing. Methods/Techniques of Usability Testing - Pluralistic Walkthroughs: To evaluate a product from the perspective of the end-user. CER

15. Advantages To represent different knowledge domains. To get a list of problems and solutions for usability. To evaluate both cognitive processing and behavioural tasks. Limitations Generally, end-users are not involved. Difficult to find a proper testing context of task-performed. Methods/Techniques of Usability Testing - Formal Usability Inspections: To test within the context of specific user profiles and defined goal-oriented scenarios. CER

16. Advantages Effective for finding cause and effect. Effective for addressing a specific question or problem. Limitations Time consuming and expensive to conduct. Need to train a skilled practitioner. Methods/Techniques of Usability Testing - Empirical Methods: An experimental test to prove or disprove a hypothesis. CER

17. Advantages Effective for predicting problems. Effective for capturing cognitive process. Limitations Need to train a skilled evaluator. Focused on one attribute of usability. Methods/Techniques of Usability Testing - Cognitive Walkthroughs: To test the ease of learning to use product by exploration. CER

18. Advantages Adequate for analysing a minimum of problem-solving behaviour. Effective for identifying problems in the early stage. Useful for comparing the different design of usability. Limitations Difficult to learn and use. Only suitable for analysing expert behaviour. Methods/Techniques of Usability Testing - Formal Design Analysis: To test the understanding of the task requirements to be performed. CER

19. Table of Data Collection Methods CER

20. Seven Basic Procedures of Usability Testing Conducting a useful usability testing takes planning and attention to detail. Following are the general procedure of usability testing for effective multimedia development: Planning a usability test • Selecting a representative sample and recruiting participants • Preparing the test materials and actual test environment Conducting the usability test • Debriefing the participant • Analysing the data of the usability test • Reporting the results and making recommendations to improve design and effectiveness CER

21. Guidelines for Conducting Usability Testing • Keep the session neutral • Treat each participant as a completely new case • Assist the participants only as a last resort • Use humour to keep the session relaxed • Use the ‘thinking aloud’ technique • Be aware of the effects of your voice and body language • If you make a mistake, keep going on CER