1 / 37

Predictive Evaluation

This article discusses simple models of human performance, including sensory perception, information processing, and the motor system. It also introduces the concept of predictive evaluation.

dsheridan
Download Presentation

Predictive Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Predictive Evaluation Simple models of human performance

  2. Recap I. Senses A. Sight B. Sound C. Touch D. Smell II. Information processing A. Perceptual B. Cognitive 1. Memory a. Short term b. Medium term c. Long term 2. Processes a. Selective attention b. Learning c. Problem solving d. Language III. Motor system A. Hand movement B. Workstation Layout

  3. Simple User Models • Idea: If we can build a model of how a user works, then we can predict how s/he will interact with the interface • Predictive model  predictive evaluation • No mock-ups or prototypes!

  4. Two Types of User Modeling • Stimulus-Response • Hick’s law • Practice law • Fitt’s law • Cognitive – human as interpreter/predictor – based on Model Human Processor (MHP) • Key-stroke Level Model • Low-level, simple • GOMS (and similar) Models • Higher-level (Goals, Operations, Methods, Selections) • Not discussed here

  5. Power law of practice • Tn = T1n-a • Tn to complete the nth trial is T1 on the first trial times n to the power -a; a is about .4, between .2 and .6 • Skilled behavior - Stimulus-Response and routine cognitive actions • Typing speed improvement • Learning to use mouse • Pushing buttons in response to stimuli • NOT learning

  6. Power Law: Tn = T1n-a If first trial (T1) takes 5 seconds, how long will future trials take? When will improvements level off? (a = -0.4)

  7. Uses for Power Law of Practice • Use measured time T1 on trial 1 to predict whether time with practice will meet usability criteria, after a reasonable number of trials • How many trials are reasonable? • Predict how many practices will be needed for user to meet usability criteria • Determine if usability criteria is realistic

  8. Hick’s law • Decision time to choose among n equally likely alternatives • T = Ic log2(n+1) • Ic ~ 150 msec • How can we use this? • Explanation on how to calculate base 2 logs: http://mathforum.org/library/drmath/view/55613.html

  9. Uses for Hick’s Law • Menu selection • Which will be faster as way to choose from 64 choices? Go figure: • Single menu of 64 items • Two-level menu of 8 choices at each level • Two-level menu of 4 and then 16 choices • Two-level menu of 16 and then 4 choices • Three-level menu of 4 choices at each level • Binary menu with 6 levels

  10. Fitts’ Law • Models movement times for selection (reaching) tasks in one dimension • Basic idea: Movement time for a selection task • Increases as distance to target increases • Decreases as size of target increases

  11. Original Experiment • 1-D d w

  12. Components • ID - Index of difficulty • larger target => more information (less uncertainty) ID = log2 (d/w + 1.0) width (tolerance) of target bits result distance to move

  13. Components • MT - Movement time • MT is a linear function of ID k1 and k2 are experimental constants MT = k1 + k2*ID MT = k1 + k2 *log2 (d/w + 1.0)

  14. Exact Equation • Run empirical tests to determine k1 and k2 in MT = k1 + k2* ID • Will get different ones for different input devices and device uses MT ID = log2(d/w = 1.0)

  15. Questions • What do you do in 2D? • h x l rect:one way is ID = log2(d/min(w, l) + 1) • Should take into account direction of approach

  16. Uses for Fitt’s Law • Menu item size • Icon size • Scroll bar target size and placement • Up / down scroll arrows together or at top and bottom of scroll bar • Example: what would Fitt’s say about multi-level menus? What about pop-up menus?

  17. Keystroke-Level Model (KSLM) • KSLM - developed by Card, Moran & Newell, see their book* and CACM * The Psychology of Human-Computer Interaction, Card, Moran and Newell, Erlbaum, 1983 • Skilled users performing routine tasks • Assigns times to basic human operations - experimentally verified • Based on MHP - Model Human Processor

  18. KSLM Accounts for • Keystroking TK • Mouse button press TB • Pointing (typically with mouse) TP • Hand movement betweenkeyboard and mouse TH • Drawing straight line segments TD • “Mental preparation” TM • System Response time TR

  19. Using KSLM - Step One • Decompose task into sequence of operations - K, B, P, H, D (no M operators yet; R can be used always or not at all)

  20. Step One : MS Word Find Command • Use Find Command to locate a six character word • H (Home on mouse) • P (Edit) • B (click on mouse button - press/release) • P (Find) • B (click on mouse button) • H (Home on keyboard) • 6K (Type six characters into Find dialogue box) • K (Return key on dialogue box starts the find)

  21. Using KSLM - Step Two • Place M operators Rule 0a. In front of all K’s that are NOT part of argument strings (ie, not part of text or numbers) Rule 0b. In front of all P’s that select commands (not arguments)

  22. Step Two : MS Word Find Command H (Home on mouse) MP (Edit) B (click on mouse button) MP (Find) B (click on mouse button) H (Home on keyboard) 6K (Type six characters) MK (Return key on dialogue box starts the find) Rule 0b: Pselects command Rule 0b: Pselects command Rule 0a: Kis argument

  23. Using KSLM - Step 3 Remove M’s according to heuristic rules (Rules relate to chunking of actions) Rule 1. Anticipated by prior operation • PMK ->PK (point and then click is a chunk) Rule 2. If string of MKs is a single cognitive unit (such as a command name), delete all but first • MKMKMK -> MKKK (same as M3K) (type “run rtn is a chunk) Rule 3. Redundant terminator, such as )) or rtnrtn Rule 4. If K terminates a constant string, such as command-rtn, then delete M • M2K(ls)MK(rtn) -> M2K(ls)K(rtn) (typing “ls” command in Unix followed by rtn is a chunk)

  24. Step 3 : MS Word Find Command H (Home on mouse) MP (Edit) B (click on mouse button) MP (Find) B (click on mouse button) H (Home on keyboard) 6K (Type six characters) MK (Return key on dialogue box starts the find) Rule 1 delete M H anticipates P Rule 4 Keep M

  25. Using KSLM - Step 4 • Plug in real numbers from experiments • K: .08 sec for best typists, .28 average, 1.2 if unfamiliar with keyboard • B: down or up - 0.1 secs; click - 0.2 secs • P: 1.1 secs • H: 0.4 secs • M: 1.35 secs • R: depends on system; often less than .05 secs

  26. Step 4 : MS Word Find Command H (Home on mouse) P (Edit) B (click on mouse button - press/release) MP (Find) B (click on mouse button) H (Home on keyboard) 6K (Type six characters into Find dialogue box) MK (Return key on dialogue box starts the find) • Timings • H = 0.40, P = 1.10, B = 0.20, M = 1.35, K = 0.28 • 2H, 2P, 2B, 2M, 7K • Predicted time = 8.06 secs

  27. Example: MS Windows Menu Selection • Get hands on mouse • Select from menu bar with click of mouse button • The “pull down” menu appears • Select desired item from the pull down menu

  28. Step 1: MS Windows Menu H (Home on mouse) P (point to menu bar item) B (left-click with mouse button) P (point to menu item) B (left-click with mouse button)

  29. Step 2: MS Windows Menu - Add M’s H (get hand on mouse) MP (point to menu bar item) B (left-click with mouse button) MP (point to menu item) B (left-click with mouse button) Rule 0b: Pselects command Rule 0b: Pselects command

  30. Step 3: MS Windows Menu - Delete M’s • H (get hand on mouse) • MP (point to menu bar item) • B (left-click with mouse button) • MP (point to menu item) • B (left-click with mouse button) Rule 1 Manticipated by P Keep M

  31. Step 4: MS Windows Menu Calculate Time • H (get hand on mouse) • P (point to menu bar item) • B (left-click with mouse button) • MP (point to menu item) • B (left-click with mouse button) • Textbook timings (all in seconds) • H = 0.40, P = 1.10, B = 0.20, M = 1.35 • H, 2P, 2B, 1 M • Total predicted time = 4.35 sec

  32. Macintosh Menu Selection • Operator sequence • H(mouse)P(to menu item)B(down)PB(up) • Now place Ms • H(mouse)MP(to menu item)B(down)MPB(up) • Selectively remove Ms • H(mouse)MP(to menu item)B(down)MPB(up) • Textbook timings (all in seconds) • H = 0.40, P = 1.10, B = 0.10 for up or down, M = 1.35 • H, 2P, 2 B, 1 M • Total predicted time = 4.15 sec • Macintosh is predicted to be .2 secs faster than MS Windows, about 5% Rule 0b Rule 0b Rule 1 Delete H anticipates P

  33. KSLM Comparison Problem • Are keyboard accelerators always faster than menu selection? • Use MS Windows to compare • Menu selection of File/Print (previous example estimated 4.35 secs.) • Keyboard accelerator • ALT-F to open the File pull down menu • P key to select the Print menu item • Assume hands start on keyboard

  34. KSLM Comparison:Keyboard Accelerator for Print • Use Keyboard for ALT-F P (hands already there) • K(ALT)K(F)K(P) • MK(ALT)MK(F)MK(P) • MK(ALT)K(F)MK(P) • 2M + 3K = 2.7 + 3K • Times for K based on typing speed • Good typist, K = 0.12 s, total time = 3.06 s • Poor typist, K = 0.28 s, total time = 3.54 s • Non-typist, K = 1.20 s, total time = 6.30 s • Time with mouse was 4.35 sec • Conclusion: Accelerator keys not necessarily faster than mouse for all users! First K anticipates second K

  35. Using KSLM • Skilled users • Performing routine tasks • The user has done it many times before • No real learning going on • Some modest “thinking” as captured by Ms • Rules for placing Ms are heuristics • Best use is for comparing alternatives • Sometimes predictions are off • But rankings of faster - slower tend to be accurate

  36. Now You Get to Do It • KSLM of the hierarchical menu selection example • Combine with Hick’s Law • Draw through text and make it bold • By pointing to BOLD icon in floating palette • By selecting BOLD from pull-down menu

  37. Cognitive models - many flavors More complex than KSLM Hierarchical GOMS - Goals, Operators, Methods, Selectors CCT - Cognitive Complexity Theory Linguistic TAG - Task Action Grammar CLG - Command Language Grammar Cognitive architectures SOAR, ACT

More Related