Lecture 5.2 Psychological Concepts Spring 2019

1. Lecture 5.2 Psychological Concepts Spring 2019 Lecture 5.2: Usability Engineering - Psychological Concepts

2. Cognitive Psychology and Information Processing A sub discipline of psychology exploring internal mental processes. It studies how people • Perceive • Remember • Think • Speak • and Solve problems Compares the human mind to a computer, suggesting that • We too are information processors • It’s possible (and desirable) to study the internal mental processes that lie between the stimuli in our environment and our responses to them. Lecture 5.2: Usability Engineering - Psychological Concepts

3. Cognitive Psychology and Information Processing Cognitive psychology attempts to answer questions like: • What makes us attend to one thing rather than another Selective attention • Why we switch our attention sometimes to something that was previously unattended Cocktail party syndrome • How many things we can attend to at the same time Attentional capacity • How much information (and in what depth) we can remember and process at any given time Capacity and levels of memory Lecture 5.2: Usability Engineering - Psychological Concepts

4. Human • User • Computer • System • User • Interface • Dual Processor Metaphor A good way to think about the user interface Think of humans as information processors who can only process a limited amount of information at a time without becoming overloaded. Cognitive processor Computer Processor Information Processing Approach Human thinking is an environment where information can be stored, retrieved and transformed using ‘mental programs’. • Input data are provided externally by our senses. • Results are our behavioral responses to the provided input. Lecture 5.2: Usability Engineering - Psychological Concepts

5. Dual Processor Metaphor • Example: EPICArchitecture (Executive-Process/Interactive Control) • A cognitive architecture developed at the University of Michigan in late 90’s • Computer • Human Lecture 5.2: Usability Engineering - Psychological Concepts

6. Input / Stimuli • Input • Process • Storage • Process • Output • Process • Output / Response Cognitive Psychology and Information Processing Basic model of human brain[suggested by Broadbent in 1950’s] “A limited capacity information processing system, through which external input is transmitted”. • Several complex Information processing models have been proposed. • All consist of a series boxes, which represent stages of processing connected with arrows that indicate the flow of information from one stage to the next. Lecture 5.2: Usability Engineering - Psychological Concepts

7. Indicative Information Processing Models Lecture 5.2: Usability Engineering - Psychological Concepts

8. Norman’s Human Action Cycle • Psychological model  proposed by Donald A. Norman • Describes how humans may form goals and then develop a series of steps required to achieve that goal. • The model includes both cognitive and physical activities (user does execute the steps) • Can be used to help evaluate the efficiency of a user interface. Lecture 5.2: Usability Engineering - Psychological Concepts

9. How people do things (Donald Norman) • To get something done, you start with some notion of what is wanted • – the goal to be achieved • Then you do something to the world • – take action to move yourself or manipulate someone or something • Finally, you check to see that your goal was made • Human action has two primary aspects • Execution: doing something • Evaluation: comparison of what happened to what was desired (to our goal)  Lecture 5.2: Usability Engineering - Psychological Concepts

10. Norman’s Action Cycle Lecture 5.2: Usability Engineering - Psychological Concepts

11. Action Cycle: Stages of Execution • Goals do not state precisely what to do • Where and how to move, what to pick up • To lead to actions, goals must be transferred into intentions • A goal is something to be achieved • An intention is a specific set of actions to get to the goal • Yet even intentions are not specific enough to control actions Lecture 5.2: Usability Engineering - Psychological Concepts

12. Action Cycle: Stages of Execution Lecture 5.2: Usability Engineering - Psychological Concepts

13. Action Cycle: Stages of Evaluation • Evaluation side, checking up on what happened, has three stages • Perceiving what happened in the world • Interpreting the state of the world • Evaluating the outcome (against our expectations) Lecture 5.2: Usability Engineering - Psychological Concepts

14. Action Cycle: Stages of Evaluation Lecture 5.2: Usability Engineering - Psychological Concepts

15. Seven Stages of Action • Onefor goals, three for execution and threefor evaluation • Note: only an approximate model • Forming the goal • Forming the intention • Specifying an action • Executing the action • Perceivingthe state of the world • Interpretingthe state of the world • Evaluatingthe outcome  • Goal • Execution • Evaluation Lecture 5.2: Usability Engineering - Psychological Concepts

16. Seven Stages of Action Lecture 5.2: Usability Engineering - Psychological Concepts

17. Seven Stages of Action - Example • “I am reading a book and decide that I need more light” • My goal: get more light • Intention: push the switch button on the side lamp • Action sequence to satisfy intention: (still a mental event) • must move my body • must stretch to reach the switch • must push the switch • Physical execution: action sequence executed • Note that I could satisfy my goal with other intention and action sequences • Instead of pushing the switch, ask another person to switch on the light • My goal hasn’t changed, but the intention and the resulting action sequence have • 5. Perception: there is more light in room (maybe not) • 6. Interpretation: the lamp has turned on • 7. Evaluation (decision): is the resulting amount of light sufficient? Lecture 5.2: Usability Engineering - Psychological Concepts

18. What the Seven Stage model Reveals • The difficulty in using everyday things and systems resides entirely in deriving the relationships between: • the mentalintentions and interpretations (‘knowledge in the head’) • and the physicalactions and states (‘knowledge in the world’) • Norman describes the problem faced by designers in meeting the goals of users as two gulfs between the user and the system • These gulfs present major problems for users Lecture 5.2: Usability Engineering - Psychological Concepts

19. Gulfs of Execution / Evaluation • Norman describes the problem faced by designers in meeting the goals of users as two gulfs between the user and the system. • Gulf of Execution: user's difficulty in translating a psychological goal into a physical action • Gulf of Evaluation: user's difficulty in evaluating whether the response of the system meets the desired goal Lecture 5.2: Usability Engineering - Psychological Concepts

20. Gulf of Execution • The gap between a user's goal for action and the means to execute that goal. • Does the system provide actions that correspond to the (mental) intentions of the person? • One measure of this gulf: • how well the system allows a person to do the intended actions directly, without an extra effort (e.g. Plug & Play) • “Good” system: direct mappings between intentions and selections Lecture 5.2: Usability Engineering - Psychological Concepts

21. Gulf of Evaluation • Amount of effort exerted to interpret the feedback (physical state of the system) and to determine how well the expectations and intentions have been met • Can feedback (perceived physical state) be interpreted in terms of intentions and expectations? • “Good” system: feedback easily interpreted as task expectations • e.g. graphical simulation of a document page being printed (WYSIWYG) • “Bad” system: no feedback or difficult to interpret feedback Lecture 5.2: Usability Engineering - Psychological Concepts

22. A Primary goal for Usability • Reduce / Bridge gulfs between the user and the system by removing • roadblocks and steps that cause extra thinking • actions that distract the user's attention from the task intended • Key: build a better bridge (interface) between them Lecture 5.2: Usability Engineering - Psychological Concepts

23. Example 1: A classic case of enlarging the Gulf of Execution • PC interfaces regularly make the mistake of not writing the action on the button. • This forces the user to read a question and then perform a mapping from his answer into a choice between the Yes and No buttons. • Far easier to use if the buttons were labelled with the desired actions • Read & understand question • Mentally answer question • Map your answer to either Yes or No • Press appropriate button • Read & understand question • Respond by pressing the appropriate action button Lecture 5.2: Usability Engineering - Psychological Concepts

24. Example 2: Reducing the Gulf of Execution • The gulf of execution can be reduced by limiting the number of widgets the user must manipulate to execute an action • Enter search text • Inspect radio buttons to determine current direction of search • Select desired direction of search (if not already selected) • Press the Search button • Enter search text • Press the desired Search button Lecture 5.2: Usability Engineering - Psychological Concepts

25. Summing Up • The gulfs of execution and evaluation refer to the mismatch between • Our internal goals • The expectations and the availability of information specifying • the state of the world • how we may change it • (Norman 1991) • Primary goal of Usability: • Bridge -or reduce- these gaps through clever UI Design • (remove roadblocks, unnecessary steps, distracting actions etc.) • The idea that the discrepancy between user and system should be conceived asgulfs came from Jim Holland and Ed Hutchins during a revision of a chapter on direct manipulation in the book User-Centered System Design (1986). Lecture 5.2: Usability Engineering - Psychological Concepts

26. Attention & Memory Lecture 5.2: Usability Engineering - Psychological Concepts

27. Attention • A cognitive process that lets you selectively focus and concentrate on certain stimuli of your surrounding and environment while disregarding others. • A brain function that allocates cognitive processing resources to focus on information or stimuli. • Deals with how you mentally process specific information present in your environment that you are experiencing through your five senses. Attention is often the initial stage to other cognitive functions. You first must pay attention to something before you can process it for meaning and understanding. Lecture 5.2: Usability Engineering - Psychological Concepts

28. Types (dimensions) of Attention • In terms of sensory organ involved: Visual Attention the ability to focus on a region of interest in the visual field Auditory Attention the ability to focus on specific sounds and process them to extract meaning Haptic Attention the ability to focus on specific touch stimuli on the skin Olfactory Attention the ability to focus on specific smells Lecture 5.2: Usability Engineering - Psychological Concepts

29. Types (dimensions) of Attention • In terms of stimuli origin: • Overt Attention: the act of directing sense organs towards a stimulus source (loud sound, quick movement etc.) • Covert Attention: the act of mentally focusing on one of several possible sensory stimuli. (listening to radio, while TV is also playing) • In terms of space/time perspective: • Spatial Attention: attention is studied from the perspective of how it is distributed across space (observing a picture) • Temporal Attention: attention is studied from the perspective of how it is distributed across time (video watching) Lecture 5.2: Usability Engineering - Psychological Concepts

30. Types (dimensions) of Attention In terms of focus target and duration: • Focused Attention : short term response to a stimulusthat attracts attention. • Sustained Attention : the ability to focus on one specific task for a continuous amount of time without being distracted. • Selective Attention: the ability to select from many factors or stimuli and to focus on only the one that you want while filtering out other distractions. • Alternating Attention: the ability to switch your focus back and forth between tasks that require differentcognitivedemands. • Divided Attention: the ability to process two or more responses or react to two or more different demands simultaneously. Divided attention is often referred to as multi-tasking. Lecture 5.2: Usability Engineering - Psychological Concepts

31. Focusing to one thing at a time • Focused attention allows you to briefly focus to a stimulus that attracts attention (6-8 sec) telephone ringing / hearing your name / seeing a ‘falling star’ • Sustained attention allows you to focus on one specific task or activity for a long period of time, listening to a lecture / reading a book / watching a video / fixing a car • Selective attention allows you to be able to “select” what you want to pay attentionto excluding (blocking-out) all other stimuli attending a loud party and you are focusing on one person’s voice trying to study in a noisy room • Watch a ‘classic’ selective attention test Lecture 5.2: Usability Engineering - Psychological Concepts

32. Focusing on multiple things at once • Both important and common in every day life • since it is rare for someone to be engaged in just one task • Alternatingattention allows you to alternate between tasks that require the use of different areas your brain • - reading a recipe (learning) and then performing the tasks of recipe (doing) • - cooking while helping your child with his/her homework. • Dividedattention allows you to split your attention instead of alternating it • checking email while listening in a meeting • talking on the phone while getting dressed • texting or talking while driving • In fact: Your brain can only process one task at a time. So you are really not “focused” on one task at a time, you are really continuously alternating your attention between tasks. • That is why it is so difficult and dangerous to text and drive or talk and drive. Lecture 5.2: Usability Engineering - Psychological Concepts

33. Attention Span • Focused attention: (e.g. telephone ringing) very brief attention span (6-8 sec) • Sustained attention: (e.g. attending class) several minutes to ??? • Most adults are unable to sustain attention on one thing for more than about 40 minutes at a time, although they can choose repeatedly to re-focus on the same thing. • This ability to renew attentionpermits people to "pay attention" to things that last for more than a few minutes, such as long movies. • Attention Span: The amount of time that a person can  concentrate on a task without becoming distracted Lecture 5.2: Usability Engineering - Psychological Concepts

34. Attention Span Statistics 2015 • http://www.statisticbrain.com/attention-span-statistics • Research Date: April 2015 • Source: National Center for Biotechnology Information, U.S. National Library of Medicine, The Associated Press • Avg. attention span dropped about 30% in 15 years Lecture 5.2: Usability Engineering - Psychological Concepts

35. Visual Attention • Where is Waldo? Lecture 5.2: Usability Engineering - Psychological Concepts

36. Visual Attention • Visual attention is thought to operate as a two-stage process. • Stage 1: attention is distributed uniformly over the external visual scene and processing of information is performed in parallel. (overview) • Stage 2: attention is concentrated to a specific area of the visual scene (i.e. it is focused), and processing is performed in a serial fashion. Lecture 5.2: Usability Engineering - Psychological Concepts

37. Visual Attention Models • Spotlight Model: • Attention is having a focus, a margin, and a fringe • Zoom-Lens Model • Expanded version of the spotlight model, with the added property of changing in size. • - Variable diameter field can be expanded or contracted • - Rate of processing inverselyrelated to the sizeof focus Lecture 5.2: Usability Engineering - Psychological Concepts

38. Assessing Visual Attention • Eye tracking and Image Heatmaps • Eye tracking: • Measurement of the eye activity to determine where / what a person is looking at (point of gaze) over time • Achieved by monitoring the motion of an eye relative to the head. • Eye tracker: a device for measuring eye positions and eye movement Lecture 5.2: Usability Engineering - Psychological Concepts

39. Eye-tracking technologies • Non-contact optical method:IR light is reflected from the eye and sensed by a video camera. • method of choice in most cases since it does not involve any hardware to be placed upon user Eye attachments such as a special contact lens with an embedded mirror or magnetic field sensor. The movement of the attachment is measured. • Must ensure that attachment does not slip significantly as the eye rotates. Electric potentials measured with electrodes placed around the eyes • eye movement can also be detected a) in total darkness b) if the eyes are closed How eye tracking devices work Lecture 5.2: Usability Engineering - Psychological Concepts

40. Gaze Plots • Display movement sequence, orderand durationof gaze fixation. • May display gaze motion of several respondents (different colors). • The image is displayed for a fixed amount of time (e.g. 10 sec.) Lecture 5.2: Usability Engineering - Psychological Concepts

41. Heat Maps • Display the hot spots of users’ focus in a certain period of time • (typically a few seconds). • The warmer the color the more attention user has paid. • Heatmaps are generated by combining hot spots from large number of users • Heat Map generation example Lecture 5.2: Usability Engineering - Psychological Concepts

42. More Heat Map Examples Lecture 5.2: Usability Engineering - Psychological Concepts

43. F-Shaped Pattern in Heatmaps when Reading Web ContentF-shaped pattern: two horizontal stripes followed by a vertical stripeHeavily favors the left side of the screen • At firstusers read in a horizontal movement, usually across the upper part of the content area. • Next, users movedown the page a bit and read across in a shorter horizontal movement • Finally, users scan the content's left side in a vertical movement. • Sometimes this is a fairly slow and systematic scan that appears as a solid stripe on eyetracking heatmap • Other times users move faster, creating a spottier heatmap. Lecture 5.2: Usability Engineering - Psychological Concepts

44. F-Shaped Pattern Examples • Left: an article in the "about us" section of a corporate website • Center: a product page on an e-commerce site • Right: a search engine results page. • Eyetracking of a Facebook page Lecture 5.2: Usability Engineering - Psychological Concepts

45. Computational Models of Visual Attention • "Participant-free eye tracking" • Heatmaps can alternatively be generated by computational models, eliminating the need for human subjects. • Models are generally based on bottom-up visual attention (Saliency) • combined with elements from • neuroscience studies of natural vision processing • computational attention studies • eye-tracking sessions (F-shaped patterns) • perception and cognition rules of humans • Models claim a 75 - 90% correlation with real eye tracking data • Several sites exist offering heatmap generation based on such models • EyeQuant Attention Wizard Crazy Egg Feng-GUI Lecture 5.2: Usability Engineering - Psychological Concepts

46. Definition of Saliency (or Salience) • Language: • The property of being noticeable or important • Neuroscience: • The perceptual quality by which an observable thing stands out relative to its environment • SocialPsychology • A set of reasons which draw an observer's attention toward a particular object (social salience) • . Basic principle behind computing visual saliency Detection of locationswhose local visual attributes significantlydiffer*from the surroundingimage attributes *along some dimension or combination of dimensions Visual saliency Example Lecture 5.2: Usability Engineering - Psychological Concepts

47. Examples of Visual Saliency • Stands out because of shape • Stands out because of color • Because of shape & color... but mainly because of human cognition rules • Stands out because of motion Lecture 5.2: Usability Engineering - Psychological Concepts

48. Visual Saliency Model Example • Example where a salience model immediately located a vehicle as being the most salient object in a complex scene (Itti & Koch, 2000). Lecture 5.2: Usability Engineering - Psychological Concepts

49. Real Eye Tracking v.s. Computational Model • Website analysis models claim a 75 - 90% correlation with real eye tracking data • This is not always true • read more about this example here • Real Eye Tracking • Shown for 6 seconds to 50 people • Computational model Heatmap (generated by Feng-GUI) Lecture 5.2: Usability Engineering - Psychological Concepts

50. Mouse Tracking • A – not so good – alternative method to visual eye tracking • Attention is measured with a mouse or similar pointing device. • Training period needed to ensure that user’s hand andeye movement aresynchronized. • Actual measurement starts directly after training. • Click data are interpreted as points of attention (visual fixation) • Not very trustworthy compared to eye tracking(Example 1 Example 2 ) Lecture 5.2: Usability Engineering - Psychological Concepts