M23CDE: Usability evaluation

1. M23CDE: Usability evaluation Benyon, Turner and Turner. Designing Interactive Systems. Chapter 12. Dix, Abowd, Finlay, Beale. Human Computer Interaction (3rd edition) Chapter 9

2. Usability ISO 9241 (part 11) defines usability as: �The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use� �we could add �learnability� to this list.

3. Effectiveness: Can you actually do a specified task? Efficiency: Can you do it quickly, without getting bored or frustrated? Satisfaction: Is it fun, or at least pleasant to use? Learnability: Can you use it without constantly reaching for the manual or asking for help. Implication: When we evaluate any design / device / UI � we need to be specific about which parameter of usability we are testing for.

4. What are we testing for? Which usability feature (effectiveness, efficiency, satisfaction, learnability) Might it be important to test for when evaluating: A Playstation game Data entry screen in a high pressure call centre An ATM machine Purchasing page in an online shop.

5. Key questions Evaluation should be considered at all stages of the design and implementation of an application. For evaluation to be meaningful, however, the designer should be able to answer some key questions: Why are we testing? (e.g. to help us make design choices, to fix design mistakes? Or to test whether some users can complete typical supported tasks?) When can we test: Pre-design, Initial prototypes, finished interface What are we testing for? (Effectiveness, efficiency, satisfaction, learnability) What evaluation method is most suitable, given 1, 2 and 3 above.

6. Evaluation: an explanation tests usability and functionality of a system, application or interface. occurs in laboratory, field and/or in collaboration with users Goals of Evaluation assess extent of system functionality assess effect of interface on users (e.g. is it effective, efficient, satisfying and easy to learn) identify specific usage problems

7. Predictive evaluation techniques Evaluation techniques deployed before an interface is built (pre-design) or very early in the design process to help guide design choices >Cognitive Walkthrough >Keystroke level model

8. Cognitive/User Modeling Idea: If we can build a model of how a user works, then we can predict how s/he will interact with the interface Predictive modeling, predictive evaluation We do not even need a mock-up or prototype

9. Predictive Models evaluating products or designs without directly involving users psychological models of users are used to test designs less expensive than user testing � Don�t have to build UI prototype Can compare design alternatives with no implementation whatsoever usefulness limited to systems with predictable tasks telephone answering systems, mobile phones, etc. based on expert behavior assumes experts or experienced users

10. Norman�s execution/evaluation loop user establishes the goal formulates intention specifies actions at interface executes action perceives system state interprets system state evaluates system state with respect to goal

11. Norman�s execution/evaluation loop user establishes the goal formulates intention specifies actions at interface executes action perceives system state interprets system state evaluates system state with respect to goal

12. Norman�s execution/evaluation loop user establishes the goal formulates intention specifies actions at interface executes action perceives system state interprets system state evaluates system state with respect to goal

13. Cognitive walkthrough Usability attributes tested: learnability Employed: on interface prototypes to predict if available actions are visible to the user and if the system state is observable.

16. Keystroke Level Model (KLM)Card, Moran and Newell (1980) quantitative refinement of the GOMS model allows predictions to be made about how long it takes an expert user to perform a task identifies basic actions involved time is measured for each action overall time is computed sum of individual actions in simple cases applied by measuring user interaction activities keystrokes, mouse movements mental preparation, hand re-positioning

17. GOMS Goals what the user wants to achieve Operators basic actions user performs Methods decomposition of a goal into subgoals/operators Selection means of choosing between competing methods

18. Goal End state that user is trying to achieve decomposed into sub-goals (like HTA)

19. Operators Basic actions available for performing a task (lowest level actions) Examples: move mouse pointer, drag, press key, read dialog box, �

20. Methods Sequence of operators (procedures) for accomplishing a goal (may be multiple) Example: Select sentence Move mouse pointer to first word Depress button Drag to last word Release

21. Selection Rules Invoked when there is a choice of a method GOMS attempts to predict which methods will be used Example: Could cut sentence either by menu pulldown or by ctrl-x

22. GOMS Procedure Walk through sequence of steps Assign each an approximate time duration Know overall performance time (Can be tedious)

23. GOMS example GOAL: CLOSE-WINDOW . [select GOAL: USE-MENU-METHOD . MOVE-MOUSE-TO-FILE-MENU . PULL-DOWN-FILE-MENU . CLICK-OVER-CLOSE-OPTION GOAL: USE-CTRL-W-METHOD . PRESS-CONTROL-W-KEYS] For a particular user: Rule 1: Select USE-MENU-METHOD unless another rule applies Rule 2: If the application is GAME, select CTRL-W-METHOD

24. Keystroke Level Model (KLM) six execution phase operators Physical motor: K � keystroking (or B button press) P � Pointing (with mouse) H - Home hands between mouse and keyboard D - Draw straight line with mouse Mental M - mental preparation System R � response Texecute = TK + TP + TH + TD + TM + TR

25. Response times for keystroke level operators

26. Where do these figures come from? Keystroke determined by typing speed 0.28 s average typist (40 wpm) 0.08 s best typist (155 wpm) 1.20 s worst typist Pointing determined by Fitts� Law T = a + b log(d/s + 1) OR T ~ 1.1 s for all pointing tasks Homing estimated by measurement 0.40 s (between keyboard and mouse) Mental preparation estimated by measurement 1.35 s

27. Fitts� Law predicts the time to point at an object using a device function of the distance from the target object and the object�s size the further away and the smaller the object, the longer the time to locate it and point time to locate an object is important for some devices and activities handheld devices like mobile phones computer games navigation in multi-screen Web pages formulated by Paul Fitts (Fitts, 1954)

28. Visit Tog�s website and do Tog�s Fitts� law quiz: http://www.asktog.com/columns/022DesignedToGiveFitts.html

29. How to do KLM List all the physical and homing operators (KPBH�s) in a given interaction sequence. Decide where to put the �mental preparation� (M) elements in the sequence Card, Moran and Newell use 4 rules for deciding on the location of the M�s

30. Rules for adding M�s Basic idea: M before every chunk in the method that must be recalled from long-term memory Insert Ms before each K & P K => MK P => MP (if P points at a command) Delete Ms in typed chunks MK MK MK => M KK .. K if Ks form a command name, single text string, or number Delete anticipated Ms x M y => x y if x fully anticipates y e.g., point-and-click is a chunk, so PMK => PK

31. KLM example GOAL: ICONISE-WINDOW [select GOAL: USE-CLOSE-METHOD . MOVE-MOUSE-TO- FILE-MENU . PULL-DOWN-FILE-MENU . CLICK-OVER-CLOSE-OPTION GOAL: USE-CTRL-W-METHOD PRESS-CONTROL-W-KEY] compare alternatives: USE-CTRL-W-METHOD vs. USE-CLOSE-METHOD assume hand starts on mouse

32. Limitations of KLM Only expert users doing routine (well-learned) tasks. Only measures efficiency � not learnability, memorability, errors, etc. Ignores errors (methods must be error-free) Ignores parallel action (shift-click) planning & problem solving (how does user select the method?) Doesn�t account for fatigue