Evaluation

1. Evaluation IMD07101: Introduction to Human Computer Interaction Brian Davison 2010/11

2. Agenda • Heuristic evaluation (experts) • Usability testing (users)

3. Heuristic evaluation • Establish aims, intended users and context of use • Look back at the user requirements • Select heuristics • Brief evaluators about the technology and intended use • Scenarios are useful here • Evaluators independently make a first pass through the user interface • Look at both typical and critical ‘tasks’ (what a user needs to do) • Review against the heuristics • Produce a consolidated list of prioritised problems • Linked to heuristics and suggested fixes

4. The automated mobile phone service line Please listen carefully to the following six options...

5. Step 1 of 4 (5 minutes) • In your usual groups, discuss • What the service line should be like for the user • Important characteristics • Additional desirable characteristics • Select appropriate heuristics to apply • Select four typical tasks to complete using the service line

6. Step 2 of 4 (15 minutes) • For the test to work, everyone must use the same tasks • After the discussion, complete your results sheet by filling in the task descriptions.

7. Step 3 of 4 (10 minutes) • Divide into groups according to your network • On your own carry out the evaluation using the template provided • For each task, consider each selected heuristic and note down any problems. Suggest any possible solutions.

8. Step 4 of 4 (5 minutes) • Combine your results onto a single sheet • Start with the longest list and add further items • Count the number of reported problems for each heuristic

9. Results

10. Usability testing • Preparation - draw up a test plan • Introduction • Testing • Users carry out tasks • You observe them (and/or record the process, with permission) • Implement any metrics • Debriefing • Interviews, questionnaires if used • Consider open questions and rating scales • Also ask about the testing process • Write up quickly

11. Adapted procedure • Select aspects to evaluate • Design evaluation tasks • Prepare evaluation template • Perform evaluation tasks • Combine results

12. The mobile phone user interface

13. Step 1 of 5 (5 minutes) • In your original groups, discuss the desirable characteristics of a mobile phone interface. Consider • People • Activities • Contexts • Technology

14. Step 2 of 5 (10 minutes) • Design a task that a subject could perform to test • The learnability of the interface • The subjective satisfaction in using the interface • The efficiency of the interface • Select an associated metric for each of these criteria

15. Step 3 of 5 (10 minutes) • Prepare test template • For the test to work, everyone must use the same tasks and metrics • After the discussion, complete the template with a description of each task and metric

16. Step 4 of 5 (15 minutes) • Divide into groups according to your make of phone • Find a partner with a different make of phone • Decide who will be evaluator and who will be subject • Note the scores on your results sheet

17. Step 5 of 5 (10 minutes) • Form groups according to the make of phone you were testing • Combine your results onto a single sheet

18. Final results