M23CDE: Usability evaluation

1. M23CDE: Usability evaluation • The next 4 weeks: • This week >Predictive evaluation methods • Usability testing • Usability testing • Design principles / heuristic inspection / Games usability • We will cover 5 methods. Benyon, Turner and Turner. Designing Interactive Systems. Chapter 12. Dix, Abowd, Finlay, Beale. Human Computer Interaction (3rd edition) Chapter 9

2. Usability ISO 9241 (part 11) defines usability as: “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” …we could add ‘learnability’ to this list.

3. Effectiveness: Can you actually do a specified task? Efficiency: Can you do it quickly, without getting bored or frustrated? Satisfaction: Is it fun, or at least pleasant to use? Learnability: Can you use it without constantly reaching for the manual or asking for help. Implication: When we evaluate any design / device / UI – we need to be specific about which parameter of usability we are testing for.

4. What are we testing for? Which usability feature (effectiveness, efficiency, satisfaction, learnability) Might it be important to test for when evaluating: • A Playstation game • Data entry screen in a high pressure call centre • An ATM machine • Purchasing page in an online shop.

5. Key questions Evaluation should be considered at all stages of the design and implementation of an application. For evaluation to be meaningful, however, the designer should be able to answer some key questions: • Why are we testing? (e.g. to help us make design choices, to fix design mistakes? Or to test whether some users can complete typical supported tasks?) • When can we test: Pre-design, Initial prototypes, finished interface • What are we testing for? (Effectiveness, efficiency, satisfaction, learnability) • What evaluation method is most suitable, given 1, 2 and 3 above.

6. Evaluation: an explanation • tests usability and functionality of a system, application or interface. • occurs in laboratory, field and/or incollaboration with users Goals of Evaluation • assess extent of system functionality • assess effect of interface on users (e.g. is it effective, efficient, satisfying and easy to learn) • identify specific usage problems

7. Predictive evaluation techniques Evaluation techniques deployed before an interface is built (pre-design) or very early in the design process to help guide design choices >Cognitive Walkthrough >Keystroke level model

8. Cognitive/User Modeling • Idea: If we can build a model of how a user works, then we can predict how s/he will interact with the interface • Predictive modeling, predictive evaluation • We do not even need a mock-up or prototype

9. Predictive Models • evaluating products or designs without directly involving users • psychological models of users are used to test designs • less expensive than user testing – Don’t have to build UI prototype • Can compare design alternatives with no implementation whatsoever • usefulness limited to systems with predictable tasks • telephone answering systems, mobile phones, etc. • based on expert behavior • assumes experts or experienced users

10. goal execution evaluation system Norman’s execution/evaluation loop • user establishes the goal • formulates intention • specifies actions at interface • executes action • perceives system state • interprets system state • evaluates system state with respect to goal

11. goal execution evaluation system Norman’s execution/evaluation loop • user establishes the goal • formulates intention • specifies actions at interface • executes action • perceives system state • interprets system state • evaluates system state with respect to goal

12. goal execution evaluation system Norman’s execution/evaluation loop • user establishes the goal • formulates intention • specifies actions at interface • executes action • perceives system state • interprets system state • evaluates system state with respect to goal

13. Cognitive walkthrough • Usability attributes tested: learnability • Employed: on interface prototypes to predict if available actions are visible to the user and if the system state is observable.

14. Cognitive Walkthrough method • Proposed by Polson et al. • evaluates interface design on how well it supportsuser in learning a task • usually performed by expert in cognitivepsychology • expert ‘walks though’ steps in a proposed interaction process to identifypotential problems using psychologicalprinciples (e.g. recognition rather than recall, affordances, feedback c.f. Donald Norman) • Needs a script in the form: User action / system response

15. Cognitive Walkthrough (cont.) For each task the walkthrough considers: 1: Will the user be trying to achieve the right effect? What is users’ goal – will they want to select this action? 2:Will the user know that the correct action is available? Is control (button, menu, switch, triple-click, etc.) for action apparent (visible)? 3: Will the user know that the correct action will achieve the desired effect? (consequence of action) Once users find control, will they recognise that it is the correct control to produce the desired effect? 4: If the correct action is taken, will the user see that things are going ok? After correct action, will users realise progress has been made towards the goal (feedback)?

16. Keystroke Level Model (KLM)Card, Moran and Newell (1980) • quantitative refinement of the GOMS model • allows predictions to be made about how long it takes an expert user to perform a task • identifies basic actions involved • time is measured for each action • overall time is computed • sum of individual actions in simple cases • applied by measuring user interaction activities • keystrokes, mouse movements • mental preparation, hand re-positioning

17. GOMS Goals • what the user wants to achieve Operators • basic actions user performs Methods • decomposition of a goal into subgoals/operators Selection • means of choosing between competing methods

18. Goal • End state that user is trying to achieve • decomposed into sub-goals (like HTA) Select sentence Move sentence Cut sentence Move to new spot Paste sentence Place it

19. Operators • Basic actions available for performing a task (lowest level actions) • Examples: move mouse pointer, drag, press key, read dialog box, …

20. Methods • Sequence of operators (procedures) for accomplishing a goal (may be multiple) • Example: Select sentence • Move mouse pointer to first word • Depress button • Drag to last word • Release

21. Selection Rules • Invoked when there is a choice of a method • GOMS attempts to predict which methods will be used • Example: Could cut sentence either by menu pulldown or by ctrl-x

22. GOMS Procedure • Walk through sequence of steps • Assign each an approximate time duration • Know overall performance time • (Can be tedious)

23. GOMS example • GOAL: CLOSE-WINDOW • . [select GOAL: USE-MENU-METHOD • . MOVE-MOUSE-TO-FILE-MENU • . PULL-DOWN-FILE-MENU • . CLICK-OVER-CLOSE-OPTION • GOAL: USE-CTRL-W-METHOD • . PRESS-CONTROL-W-KEYS] • For a particular user: • Rule 1: Select USE-MENU-METHOD unlessanother • rule applies • Rule 2: If the application is GAME, • select CTRL-W-METHOD

24. Keystroke Level Model (KLM) six execution phase operators • Physical motor: K – keystroking (or B button press) P – Pointing (with mouse) H - Home hands between mouse and keyboard D - Draw straight line with mouse • Mental M - mental preparation • System R – response Texecute = TK + TP + TH + TD + TM + TR

25. Response times for keystroke level operators

26. Where do these figures come from? • Keystroke determined by typing speed • 0.28 s average typist (40 wpm) • 0.08 s best typist (155 wpm) • 1.20 s worst typist • Pointing determined by Fitts’ Law • T = a + b log(d/s + 1) • OR • T ~ 1.1 s for all pointing tasks • Homing estimated by measurement • 0.40 s (between keyboard and mouse) • Mental preparation estimated by measurement • 1.35 s

27. Fitts’ Law • predicts the time to point at an object using a device • function of the distance from the target object and the object’s size • the further away and the smaller the object, the longer the time to locate it and point • time to locate an object is important for some devices and activities • handheld devices like mobile phones • computer games • navigation in multi-screen Web pages formulated by Paul Fitts (Fitts, 1954)

28. Visit Tog’s website and do Tog’s Fitts’ law quiz: http://www.asktog.com/columns/022DesignedToGiveFitts.html

29. How to do KLM • List all the physical and homing operators (KPBH’s) in a given interaction sequence. • Decide where to put the ‘mental preparation’ (M) elements in the sequence • Card, Moran and Newell use 4 rules for deciding on the location of the M’s

30. Rules for adding M’s • Basic idea: M before every chunk in the method that must be recalled from long-term memory • Insert Ms before each K & P • K => MK • P => MP (if P points at a command) • Delete Ms in typed chunks • MK MK MK => M KK .. K if Ks form a command name, single text string, or number • Delete anticipated Ms • x M y => x y if x fully anticipates y • e.g., point-and-click is a chunk, so PMK => PK

31. USE-CTRL-W-METHOD H[to kbd] 0.40 M 1.35 K[L7 key] 0.28 Total 2.03 s USE-CLOSE-METHOD P[to menu] 1.1 B[LEFT down] 0.1 M 1.35 P[to option] 1.1 B[LEFT up] 0.1 Total 3.75 s KLM example GOAL: ICONISE-WINDOW [select GOAL: USE-CLOSE-METHOD . MOVE-MOUSE-TO- FILE-MENU . PULL-DOWN-FILE-MENU . CLICK-OVER-CLOSE-OPTION GOAL: USE-CTRL-W-METHOD PRESS-CONTROL-W-KEY] • compare alternatives: USE-CTRL-W-METHOD vs. USE-CLOSE-METHOD • assume hand starts on mouse

32. Limitations of KLM • Only expert users doing routine (well-learned) tasks. • Only measures efficiency – not learnability, memorability, errors, etc. • Ignores errors (methods must be error-free) • Ignores parallel action (shift-click) • planning & problem solving (how does user select the method?) • Doesn’t account for fatigue