The Growth of Cognitive Modeling in HCI Since GOMS

1. The Growth of Cognitive Modeling in HCI Since GOMS Anna Kolesnichenko Songmei Han

2. Overview • GOMS as cognitive modeling • Advances in modeling specific serial components • Extensions of the basic framework • What Cognitive Modeling in HCI can and cannot do

3. Cognitive modeling • the progress in modeling the kind of cognition involved in HCI • basic and advanced sets of parameters • account for the time of given activities • formal modeling in grammars and production systems • Error production • Time to learn • Savings from previous learning • critical path analysis • Specification of interacting processes and their durations

4. Cognitive Models • Predict how users will interact with proposed designs • Constrain the design space • Answer specific design decisions • Estimate total time for task performance • Provide base for calculating training time and designing training documentation • Determine stages of activity that take the longest time or produce the most errors

5. Cognitive modeling (cont.) • Method used for design, evaluation and training • Gaps in understanding the process of interacting with computers • Human learning • Design of consistent user interfaces • Error production and management • Interpretation of visual displays for meaning • Concurrent vs. sequential processes

6. Gaps in cognitive theory • Fails to capture • User’s fatigue • Individual differences • Mental workload • Change expected in work life • User’s judgment of the acceptability of the software

7. Analytic models of human performance with computers • 1980-1983 – Card, Moran and Newell - significant advance from modeling in cognitive psychology • modeled together many of the processes contributing to the full cycle of perception to action • described in enough detail the knowledge necessary to perform a task • Enabled to generate predictions about human behavior in real, naturalistic tasks

8. Framework • Two key components • Model Human Processor (MHP) • General characterization of the human information-processing system • System architecture • Quantitative parameters of component performance • GOMS • A way of describing what the user needs to know to perform computer-based tasks

9. GOMS • A family of models • Describes • The knowledge necessary • Four cognitive components of skilled performance in tasks • Goals • Operators • Methods • Selection rules

10. Original GOMS Framework Focus: • Selection of methods from memory • Time to specify and execute an action

11. The GOMS Method Assumptions • Skilled user • Serial sequence of independent cognitive operations and motor activities The method • Predict a time it takes a user to execute a task • A task is based on retrieving plans from long-term memory • A method is chosen from available methods depending on the features of the task • Execute motor movements necessary Time parameters for external actions were estimated from empirical data

12. Example Parameters: • k – keystroke: 280 msec • M - mental operator: 1.35 sec • P – pointing: 1.1 sec • H – moving hands: 400 msec Example: sum ( a , b ) Mkkk MkMkMkMkMk Total = 6M’s + 8 k’s = 6(1.35) + 8(.280) = 10.34 sec

13. Limitations of GOMS • Limited range of domains • Applied to skilled users only • Accounts for performance but neither learning nor recall • Focused on errorless performance • Gives little account of cognitive processes • Focused of sequential tasks while many processes occur in parallel • Does not address mental workload • Disregards fatigue that users experience • Does not account of individual differences among users

14. Advances in modeling specific serial components • 1983 – further research based on GOMS methodology • Serial processing • Time parameters are constant across tasks • Incorporated relevant cognitive psychology factors • Empirical work based on studies of entering editor commands, formulas in spreadsheets, etc.

15. Classes of parameters • motor movement • perception • memory • cognition

16. Motor Movements • Keying • Moving a mouse • Hand movement

17. Keying • Time to enter a keystroke in a normal typing task • Value depends on • The skill level of the typist • Frequency with which a key is used • Predictability and continuity of the text • Example • Skilled typist – 80msec/keystroke • User unfamiliar with the keyboard – 1200 msec /keystroke

18. Moving a Mouse • Pointing with a mouse at objects at various distances and of various target sizes. • Derived from empirical experiments. • Fitts’s law: • T = 1.03+.096 log2(D/S+.5) • applied to nested menus: T = .81+.21 log2(D/S+.5)

19. Hand Movements • Time needed to move from the space bar of the keyboard until the pointing control begins to move the cursor. • Large-muscle movement • Characterized by Fitts’s law • Empirically, T = 360 msec

20. Perception • Recognition of features of the current task and assessment of some parameters necessary to do a task • Examples: • Time to respond to a brief light = 100 msec (50-200 msec depending on intensity) • Time to recognize a 6-letter word = 340 msec • Time for the eye to jump to next location = 320 msec

21. Memory and Cognitive Processes • Memory retrieval • Executing steps in a mental procedure • Choosing among methods

22. Memory Retrieval • Time to retrieve the next unit of information • well-known units • from long-term memory to working memory • A repeated act speeds up memory access

23. Memory Retrieval Example

24. Executing Steps in a Task GOMS catalogues: • the retrieval of goal and its subgoals • the decision to select a method • the retrieval of the motor movements • the execution of each command component Production system formalism - explicit representation

25. Choosing Among Methods • The more choices – the longer the expected response time • Empirical estimations of time vary (1.3 – 4.6 sec)

26. Composite Performance A task: enter a block of values(2 digits) • Mouse method- enter each value, point to the next cell with a mouse • Menu method - <ret> key advances cursor automatically to the next cell. Use mouse only to go to the next line

27. Empirical solution Empirical results: • Mouse method 4.19 sec per cell • Menu method 2.46 sec per cell 2.81 sec to start each line

28. GOMS solution Mouse method moving the hand to the mouse 360 msec clicking the mouse 230 msec moving the hand to the keyboard 360 msec retrieving digits 1200 msec typing digits 460 msec retrieving the end action 1200 msec typing the <ret> key 230 msec Total 4040 msec 3% error of 4.19 sec empirical result

29. GOMS solution (cont.) Menu method – starting a new line: moving hand to mouse 360 msec pointing to a new line 1500 msec clicking the mouse 230 msec moving hand to keyboard 360 msec Total 2450 msec 13% error of 2.81sec empirical result

30. GOMS solution (cont.) Menu method – typing a number into a cell: retrieving two digits 1200 msec typing two digits 460 msec retrieving the end action 1200 msec typing the <ret> 230 msec Total 3090 msec 26% error of 2.46 sec empirical result

31. Pros and cons of the method Challenged based on inclusion or exclusion or an operation (esp. mental) Achievements • within an average of 14% error of the observed values • accurate enough to be useful

32. Summary • Problems with GOMS: • Serial process assumption • Independent task assumption • Served well in a variety of basic computer-based tasks

33. Extension of the Basic Framework • Learning and Transfer • Time to learn • Transfer from one system to the other • Analysis of errors: Workload in Working Memory • Parallel Processes • Modeling parallel processes with critical path analysis

34. Modeling in extended work • Modeling of grammatical rules • What knowledge a user must have before translating from goals to actions in a system? • Similar to goal decomposition and methods in GOMS. • Provide a countable entity: • The number of rules

35. Task-Action Grammar (TAG)

36. Task-Action Grammar (TAG)

37. Production system • Make underlying knowledge explicit • Once written, the accuracy and completeness can be checked by running the program • Program can be used to predict both errors and learning time behavior

38. Rules for writing a SQL join query Rule 1: StartUp. SeeIfJoinNeeded IF GOAL SeeIfJoinNeeded Not (NOTE SeeingIfJoinNeeded TRUE THEN Add NOTE SeeingIfJoinNeeded TRUE Add STEP CountTables

39. Rules for writing a SQL join query Rule 2: CountTables IF GOAL SeeIfJoinNeeded STEP CountTables THEN Do Task Count NumberOfTables * NumberOfTables Add NOTE NumberOfTables * NumberOfTables Delete STEP CountTables Add STEP AddJoinNote

40. Rules for writing a SQL join query Rule 4: IfNumberOfTablesNot =2, ThenCleanUp IF GOAL SeeIfJoinNeeded STEP AddJoinNote NOTE NumberOfTables 1 THEN Delete SETP AddJoinNote Delete NOTE NumberOfTables ?NumberOfTables Add STEP Cleanup

41. Use of Production Rules • IF part: Check for a match between the rule’s condition and the current goal and the current notes in working memory (WM). • THEN part: If there is a match, execute THEN part action to add and delete NOTES and STEPS to or from WM.

42. Time to Learn • Kieras & Polson • Study learning under highly restrictive and controlled condition • Determine the number of steps in a procedure • The time to learn each step is 30 second • The start-up time is 30 to 60 minutes.

43. Time to Learn (cont) • Results from from widely different situations and labs are at the same order of magnitude. • The number of rules is less critical than whether the features of those rules follow real-world features encoded in user’s memory. • Problem: how to quantify the learning time in more naturalistic situations.

44. Transfer of Training • Production is the unit of learning. • The number of productions shared by the two systems can predict the amount of transfer. • Time to master a new procedure is a function of the number of new production to be learned. • Specify the exact effects of consistent design across system and assess the relative costs of different degrees of consistency among procedures.

45. Transfer Predicted by the number of new rules to be learned

46. Analysis of Errors • Cause of errors: • Working Memory (WM) overload • Production systems are used to estimate the contents of WM and the resident duration of each piece of information in WM. • The more items in WM, the greater the likelihood of errors.

47. Systems with different WM workload • Lotus 1-2-3 • User has to find and remember the coordinates of cells in the formula. (e.g. D23) • Interactive Financial Planning System (IFPS) • User can refer to a cells by name with adjective (e.g. previous) to indicate relative location. No need to remember the coordinates.

48. WM Load for Different Systems

49. Position Naming (e.g. B22) Formula with cells in the same column WM Time

50. Keyword Naming (e.g. Previous Sales) Formula with cells in the same column WM Time