Lecture #3 TESTING WITHOUT USERS II.

1. Lecture #3TESTING WITHOUT USERS II. Y39TUR Winter 2011/2012 Tvorba uživatelského rozhraní

2. Human Computer Interaction TUR 2011 • Three main objectives • Improve the access to the computers • Reduce the complexity of this access • Reduce the probability of errors while using computers • Can be done by: • Direct observation of using computers • Use of cognitive models • Mathematical models of using computers

3. Cognitive Science TUR 2011 • Part of psychology investigating mental processes such as: • Perception, memory, thinking, learning, problem solving, language, … • Extensive applications in other fields • Engineering disciplines • Purpose is to improve the design • Major interest by • Human Computer Interaction • Human Factors (Cognitive Ergonomics)

4. Mental Models TUR 2011 • How do we understand the way how the world works around us? • Memory (past experience) • Expectation of the behavior • When throwing a ball at a friend • How fast should I do that? • How do they know where to stand? • We have certain understanding of the world’s mechanics • How the objects react? • Which is the relationship among them?

5. What is a Mental Model? TUR 2011 • A cognitive structure • “It’s in one’s head.” • Describes how certain aspects of the world work • How objects of certain class mutually interact with objects of different class • How objects change their properties during an interaction • Each person has their own unique mental model of the world

6. Reality …

7. … and a mental model THIS STATIONIS SOMEWHERE AROUND HERE ELECTRICITY COURSES TOURISTS MY BUDDY LIVESSOMEWHERE AROUND HERE Y39TUR FARE INSPECTORS!!

8. Mental Model Example: The Subway • Having a mental model, we can ask: • “How far are we from station X?” • “Where do we need to change when going to Y?” • “What is the next station?” • “Which way to go to my buddy’s?” • Mental model contains information AND knowledge of uncertainty • “I don’t know what’s on the B-line.” TUR 2011

9. Mental Model • How the things are understood • Not how they actually work! TUR 2011

10. How do we use mental models? TUR 2011 • Prediction of the world’s behavior • “Two more stops. We still have time to read three pages of the book” • Mental models are fuzzy • Can lead to incorrect predictions • We change the mental models during their use • We learn and adapt

11. Cognitive Models in HCI TUR 2011 • Understanding cognitive models: • Important for understanding the user’s interaction with the system • Understanding how the computers are understood • Understanding how the data are understood • Usability tests are used to uncover users’ mental models • Mismatch between cognitive models and reality leads to usability issues • User’s mental models can explain their behavior

12. Cognitive Models in HCI (contd.) • Users make use of the cognitive model of the user interface, e.g.: • What to expect when clicking a button • “We should be one click away from that big dialog window that tells me the CD is burning.” • To tell we did the right/wrong thing • “Oops, we should not get to this page. I should have clicked the link below.” • “Oh no, I got on the wrong train.” TUR 2011

13. Example: Mismatch of Cognitive Models • Telephone banking in an imaginary bank • Task: Make a payment using your credit card • Device: Telephone DTMF menu • A menu would say: • “For account balance, press 1” • “For payments, press 2” • “For credit card operations, press 3” Guess again! Send money (from my account, using CC…) Receive money See how much I have USER’S COG. MODEL BANK SERVICES DESIGNER’S COG. MODEL Checking Accounts Department Credit CardsDepartment TUR 2011

14. Cognitive Models in Usability Testing • When preparing a test, we must not make the user accept our mental model • Instructions for user is not a user guide • Suggestive instructions • A real user has no existing mental model of the tested application • Making use of a previous experience • That’s why the recruitment from the target group is so important TUR 2011

15. Cognitive Models in Usability Testing • A use-case • “Making a payment using a credit card.” • Incorrect wording of the task in a usability test • “Choose ‘credit card payments’ from the menu of the telephone service.” • Correct wording of the task in a usability test • “Try using the credit card to make a payment in the menu of the telephone service.” TUR 2011

16. Cognitive Science in UI Testing

17. It is easy to describe a machine … • More-or-less, all diagrams of a computer look like this: • Prediction of function is easy • Can we find such a simple description for a human being? Input Proc. Output Memory TUR 2011

18. Human Information Processor Model • An approach to model how information is handled by the user. • Technicist approach to model the human • First formulations 1980s – 1990s TUR 2011

19. Human Information Processor Model “input” “processing” “output”

20. Human Information Processor Model for Testing TUR 2011 • Basic idea: • Similar to cognitive walkthrough • When stimuli are known, what will be the corresponding human behavior? • No need for implementation or even prototypes • No need for real users • Gives a scientific foundations for a design • Like for other engineering disciplines

21. Cognitive Theories in HCI TUR 2011 • Models based on Human Information Processor Model: • Fitt’s Law • How long it takes to select a target • Evaluation of input devices • KLM (Keystroke-level Model) • Efficiency of the user interfaces assessed through low-level actions • GOMS (Goals, Operators, Methods, Selectors) • Higher level than KLM • Structure and hierarchy of tasks

22. FITT’S LAW

23. Fitt’s Law • Paul Fitt (1954) • Based on ergonomics • How fast would a person reach a target with their hand? • Prediction of time needed to acquire a target, based on: • Distance to the target (D) • Dimension of the target (W) Small Something Other thing Large TUR 2011

24. Fitt’s Law • D, W … distance, width (amplitude) • Device-dependent constants • a … operation cost (e.g. time needed to press a button) • b … inherent speed of the device (how fast can we move around?) • Originally for 1-D problems TUR 2011

25. Fitt’s Law Index of Difficulty Target 1 Target 2 ID equal From: Marti Hearst, User Interface Design & Development TUR 2011

26. Fitt’s Law Index of Difficulty Target 1 Target 2 ID smaller TUR 2011

27. Fitt’s Law Index of Difficulty Target 1 Target 2 ID bigger TUR 2011

28. Fitt’s Law for Testing • Additional design heuristic for Heuristic Analysis • Used as a qualitative suggestion. • Fitts’ law often used for determining best case for new kinds of input methods • Used as a theoretical framework for conducting experiments where two approaches are compared. TUR 2011

29. Design Heuristics based on Fitt’s Law • Things done more often should be assigned a larger button. • Size of the Enter key • Sides of the screen and corners have “infinite size” • Possible problems of consistency • Things done more often should be “closer to each other” • Context menu • Frequency-based order vs. logic-based order TUR 2011

30. Infinitely Large Objects TUR 2011

31. Keystroke-level Model (KLM)

32. KLM != Koninklijke Luchtvaart Maatschappij TUR 2011

33. Keystroke-level Model • Cognitive Walkthrough, Heuristic Evaluation • Good source of qualitative findings of: • Usability issues • No ability to tell the time taken by … • No data on actual performance • To measure time: • Large number of users (statistically valid data) • Expensive (time and people) • Is there a cheap alternative? TUR 2011

34. Keystroke-level Model • Purpose of the KLM • A discount usability method • A method of evaluation of the UI • KLM defines a metric of performance of a UI • Provides an estimate of minimal duration of a UI walkthrough • Will be worse in reality • Based on a model of “virtual user” • On Human Information Processor Model • Formalism of behavior • Focuses on performance only TUR 2011

35. Keystroke-level Model • Experimental basis of the KLM • Low-level model of a “typical user” • A large number of people observed while using generic GUI • Measurement of: • Reaction times • Duration of elementary actions TUR 2011

36. Keystroke-level Model • Input: • Similar to Cognitive Walkthrough • Detailed description of sequence of actions • Output: • Estimate of the lowest time possible TUR 2011

37. Keystroke-level Model • Set of operators • Further indivisible actions • Based on the current application domain • Physical actions • Reach for mouse, drive the cursor somewhere, etc. • Mental actions • Make a decision, select one item of many, etc. TUR 2011

38. Keystroke-level Model • Uses of KLM • Determine what is the minimum time of a UI walkthrough • Compare two different walkthroughs of UI leading to the same result • Compare performance of users of different profiles • Beginner user (no shortcuts, all commands from the menu) • Intermediate user (minimum amount of shortcuts) • Advanced user (ample use of shortcuts + command line) • Calculate the potential volume of savings • Is it useful to invest into any UI optimizations? • Is it useful to train the users? TUR 2011

39. Keystroke-level Model • Theoretical support of a particular design change suggestion • Comparison of the current state and the design • It is possible to formally support a claim on correctness of the designed solution • (More on this in the NUR course) TUR 2011

40. Keystroke-level Model • An ideal walkthrough is considered • We test the correct and minimal solution • No confused users • No mistakes • No roll-backs • Time that we calculate is minimal possible • Reality will probably be worse TUR 2011

41. KLM Operators • K Keystroke • Key hit, pressed, or released. • Also for the mouse button • 0.08 s – 1.20 s, based on the skill level • P Point on a target • 1.10 s, average performance • H Home the input device • 0.40 s • M Mental preparation • 1.35 s • R System reaction time • Whatever time the system takes • Card et al. (1983) TUR 2011

42. KLM Simple Walkthrough Example • Example: Make “The cat” in sentence“The cat sat on the mat” bold. • Note: K = .60 (average typist) • Steps • Select “The cat” • Reach for the mouse (H = .40) • Point to “The” (P = 1.10) • Double-click and hold down the button (K = .60) • Move to “cat” (P = 1.10) • Release the mouse button (K = .60) • Set to bold • Press Ctrl (K = .60) • Press “B” (K = .60) • Release Ctrl (K = .60) • Total = 5.6 seconds TUR 2011

43. KLM Example: Alternative – Using a Menu • Total = 7.2 seconds • Example taken from Newman & Lamming (1995) Interactive System Design • Select “The cat” • (see previous slide) • Set to boldface • Point to “Format” menu (P = 1.10) • Press mouse button (K = 0.60) • Move to bold (P = 1.10) • Release mouse button • (K = 0.60) TUR 2011

44. More on KLM Operators TUR 2011 • K – Keystroke • Determined by typing speed • T = 0.08 s … excellent typist (155 WPM = 775 CPM) • T = 0.28 s … average typist (40 WPM = 200 CPM) • T = 1.20 s … worst typist; unfamiliar keyboard • P – Pointing • Average value T = 1.10 s given • For any action of pointing • More precise value must be determined by an experiment (e.g. using Fitt’s Law)

45. More on KLM Operators • H – Homing • Switch between the keyboard and the mouse • Not applicable if each hand operates one device • T = 0.40 s • M – Mental preparation • T = 1.35 s • How determined: • Numerous tasks observed and analyzed • Total time minus time spent on physical operations (K, P, H) • Divide by number of “mental activities” TUR 2011

46. When to use the “M” operator • “When do people think?” • Place M before each K and P • K  MK • P  MP • “People need to think where to move next, what to type next, etc.” • Remove M between the letters of a typed word • MKMKMK  MKKK • “People don’t think before each letter” • Remove M between compound actions (“point-and-click”) • MPMK  MPK • “People take point and click as a single operation” TUR 2011

47. Delete words using selection M P [start of selection] K [click] M P [end of selection] K [shift] K [click] H [rt hand on Delete] M P [Delete] 7.37 seconds Delete words, letter-by-letter M P [first letter] K [click] H [rt hand on Delete] M K × 14 8.36 seconds KLM Example #2 TUR 2011

48. KLM Verification Card, Moran and Newell, “The Keystroke Level Model for User Performance Time with Interactive Systems” TUR 2011 (48)

49. Limitations of KLM • The task is analyzed as if performed by an expert user • That is, without an error • Only predicts the effectiveness • Not the learnability • Not the memorability • What is ignored • Parallel execution • Task interleaving • Mental (cognitive) load • Planning and problem-solving (“How do I choose what to do?”) TUR 2011

50. GOMS