The Human Processing and Memory

1. The HumanProcessing and Memory Human Computer Interaction, 2nd Ed. Dix, Finlay, Abowd, and Beale Chapter 1

2. Model Human Processor + AttentionRecall, “purely and engineering abstraction” • Sensory store • Rapid decay “buffer” to hold sensory input for later processing • Perceptual processor • Recognizes symbols, phonemes • Aided by LTM • Cognitive processor • Uses recognized symbols • Makes comparisons and decisions • Problem solving • Interacts with LTM and WM • Motor processor • Input from cog. proc. for action • Instructs muscles • Feedback • Results of muscles by senses • Attention • Allocation of resources

3. Overview • Will look at elements of human information processing from a slightly different orientation than “engineering abstraction” • A bit more fine grained analysis, following from psychological studies • But, it is these psychological studies from which the “engineering abstraction” is derived • 3 stage model of human memory • Iconic buffer, STM, LTM • Models of LTM • Reasoning • Problem solving

4. Model Human Processor + AttentionRecall, “purely and engineering abstraction” • Sensory store • Rapid decay “buffer” to hold sensory input for later processing • Perceptual processor • Recognizes symbols, phonemes • Aided by LTM • Cognitive processor • Uses recognized symbols • Makes comparisons and decisions • Problem solving • Interacts with LTM and WM • Motor processor • Input from cog. proc. for action • Instructs muscles • Feedback • Results of muscles by senses • Attention • Allocation of resources

5. 3-Stage Model of Human Memory • Sensory (here, iconic) memory – “very” short term memory • lasts 1-2 seconds, infinite capacity • Short-term memory (Working memory) • lasts ~ 18 seconds, holds 1.75 (7+/-2 items) • Long-term memory • infinite capacity; short of damage is permanent • Recall vs. Recognition (Remember vs. Know) • Retrieval cues • Will demonstrate later in class … http://www.if.uidaho.edu/~marbjm/class%202.pdf

6. “Executive” - Attention • Central “executive” controls tasking • Pays, or allocates, attention • Bandwidth of attention is limited • Tasks that require the same resources interfere with one another • Attention is both a low-level and high-level property of vision http://www.if.uidaho.edu/~marbjm/class%202.pdf

7. Sensory Memory: “Very” Short Term Memory • Sensory buffers for stimuli received through senses • iconic memory: visual stimuli • echoic memory: aural stimuli • haptic memory: tactile stimuli • Examples • “sparkler” trail • stereo sound • Continuously overwritten – demo follows

8. A Test – of Visual Iconic Memory • Will present figure briefly (~1/2 second) • Try to remember as many elements as you can • Write them down

12. The Phenomenon • After presentation, did you continue to “see” the items? • Some purely physiological based “seeing”: • Afterimage • Bleaching of pigments • “bright, or colored, stuff” • But also, there is a more “memory-based” image (process further downstream in memory system) • Iconic memory • “dark, or veridical, stuff” • Reading from the iconic buffer

13. Reading from the Iconic Buffer, 1 • Typically can list 3 – 7 items named • Short lived visual, or iconic, buffer • holds the image for a second or two • Read images and place in STM • 3-stage model • Can get about 5-7 items until run out of short term (working) memory capacity • Limitation of 5-7 comes from: • Decay of iconic memory • Rate can read from visual buffer • Capacity of working memory • In each fixation between saccadic eye movements, image of world captured Set of miscellaneous symbols

14. Reading from the Iconic Buffer, 2 • Again, Limitation of 7 comes from: • Decay of iconic memory • Rate can read from visual buffer • Capacity of working memory • From each image, • brain must identify objects, • match them with objects previously perceived, and • take information into working memory for symbolic analysis • Search light model of attention (for vision) • Visual information is acquired by pointing fovea at regions of visual field that are interesting • Then using a scanning process in which objects are read from an image buffer from more extensive processing Set of miscellaneous symbols

15. Attention • Spotlight metaphor • Spotlight moves serially from one input channel to another • Can focus attention (and perceptual processor) on only one input channel at a time • Location in visual field, voice in auditory field, …, anything • Visual dominance: • Easier to attend to visual channels than auditory channels • All stimuli within spotlighted channel are processed in parallel • Whether you want to or not • Can cause “interference” - demo

16. Say the Colors of the Words • Easy enough – didn’t take too long

17. Say the Colors of the Words • Took longer … Stroop effect • For design: • Choose secondary characteristics of display to reinforce message

18. Again, Human Memory Stages • Sensory (here, iconic) memory • lasts 1-2 seconds, infinite capacity • Short-term memory (Working memory) • lasts ~ 18 seconds, holds 1.75 (7+/-2 items) • Long-term memory • infinite capacity; short of damage is permanent • Recall vs. Recognition (Remember vs. Know) • Retrieval cues http://www.if.uidaho.edu/~marbjm/class%202.pdf

19. Short-term memory (STM) • “Scratch-pad” (or buffer) for temporary recall • rapid access ~ 70ms • rapid decay ~ 200ms • limited capacity - 7± 2 chunks • Chunking, recoding, etc. • affects amount of information retained, entering LTM

20. Example - Chunking HEC ATR ANU PTH ETR EET

21. Long-term Memory (LTM) • Repository for all our knowledge • slow access ~ 1/10 second • slow decay, if any • huge or unlimited capacity • Episodic and semantic memory • Episodic (episodes): Serial memory of events • Semantic (“meanings”): Structured memory of facts, concepts, skills • Also, procedural and declarative memory • “Processes” vs. “facts”

22. LTM – Models of Semantic Memory • Semantic memory structure • Contains LTM knowledge of world • Provides access to information • Generic knowledge -- specific details lost • Represents relationships between bits of information • Important for rule-based behavior • Supports inference • Many models, theories, accounts, schemata proposed • Semantic network model (example next slide): • E.g., Inheritance – child nodes inherit properties of parent nodes • Relationships between bits of information explicit • Supports inference through inheritance • Other Models (examples follow): • Scripts, frames, production rules

23. Early Model of Semantic MemoryCollins and Quillian • Collins & Quillian’s Teachable Language Comprehender • Semantic memory is organized as a network of interrelated concepts • Each concept is represented as a node • Concepts are linked together by pathways • Economy of representation • Activation of one concept spreads to interconnected nodes • Remind you of anything from computer science?

24. Early Model of Semantic MemoryCollins and Quillian • Collins & Quillian’s Teachable Language Comprehender

25. Early Model of Semantic MemoryCollins and Quillian • Spreading Activation • Working memory is activated LTM • When a concept becomes active, activation spreads to all other interconnected nodes • Activation spreads to all related nodes • How do you evaluate sentences like “Is a robin is an animal”?

26. Early Model of Semantic MemoryCollins and Quillian • Spreading Activation • Activation spreads from each of the concept nodes (Robin & Animal) • When two spreading activations meet, an intersection is formed • Robins ==> BIRD <== Animals • If no intersection, relatively fast no • If intersection, decision stage operates to determine if sentence is valid Is a robin an animal?

27. Tests of Spreading Activation • Sentence verification task • Time to respond yes or no • Takes time for activation to spread • Greater distances ==> longer RT • Verification time for items 0, 1, and 2 links

28. But, it’s not that simple ... • E.g. typicality effects - how many links separate: • A canary is a bird? • A robin is a bird? • A chicken is a bird? • An ostrich is a bird? • But, RT varied - less typical birds took longer than more typical birds

29. Semantic Relatednes and Semantic Priming • Semantic relatedness • Spreading activation between related concepts • Activation of one concept partially activates semantically related concepts • Semantic priming • Stimulus 1 ==> Stimulus 2 • (Prime) ==> (Probe) • Test spreading activation by manipulating semantic relationship between prime & probe • Concepts linked by spreading activation • Prime: Probe: • Doctor Nurse • Bread Butter • Doctor Butter • Bread Nurse • Sometimes prime facilitates processing

30. Semantic Relatednes • Recall, semantic relatedness • Spreading activation between related concepts • Activation of one concept partially activates semantically related concepts • So, can focus on relatedness, without explicitly indicating links

31. Spreading Activation - Fin • Semantic priming commonplace • Can exploit in design • Indeed, in design can exploit all information about how human operates • Spreading activation is thought to be automatic • Governed by data-driven aspects of processing • How do expectancies affect semantic access? • Automatic vs Conscious Strategies (attentional) • Fast vs Slow • Effortless vs Effortful • Benefits vs Costs & Benefits

32. Fyi – Another semantic network

33. Models of LTM – Frames, or Schemata COLLIE Fixed breed of: DOG type: sheepdog Default size: 65 cm Variable colour • Information organized in “memorial data structures” • Schemata • Stored framework or body of knowledge • Conceptual framework for interpreting information • Biased information processing to relate new material to what we already know • Alters way we perceive things • Individual differences in perception and memory • Frames • Slots in structure instantiated with values for instance of data • Type–subtype relationships DOG Fixed legs: 4 Default diet: carniverous sound: bark Variable size: colour

34. Script for a visit to the vet Entry conditions: dog ill vet open owner has money Result: dog better owner poorer vet richer Props: examination table medicine instruments Roles: vet examines diagnoses treats owner brings dog in pays takes dog out Scenes: arriving at reception waiting in room examination paying Tracks: dog needs medicine dog needs operation Models of LTM - Scripts • Model of stereotypical information required to interpret situation • Script has elements that can be instantiated with values for context

35. Models of LTM - Production Rules • Representation of procedural knowledge. • Condition/action rules if condition is matched then use rule to determine action. IF dog is wagging tail, THEN pat dog IF dog is growling, THEN run away

36. LTM - Storage of information • LTM much studied in psychology: • Rehearsal • information moves from STM to LTM • Total time hypothesis • amount retained proportional to rehearsal time • Distribution of practice effect • optimized by spreading learning over time • Structure, meaning and familiarity • information easier to remember

37. LTM - Forgetting • Decay • information is lost gradually but very slowly • Interference • new information replaces old: retroactive interference • old may interfere with new: proactive inhibition • So, ... may not forget at all, memory is selective …! • Also, affected by emotion – can subconsciously `choose' to forget

38. LTM - Retrieval • Should be familiar from heuristics • Recall • information reproduced from memory can be assisted by cues, e.g. categories, imagery • Recognition • information gives knowledge that it has been seen before • less complex than recall - information is cue

39. Thinking – Cognitive Processing • Humans reason, process information, like, well, humans • Recall, any theory is an abstraction and, thus, captures some elements of phenomenon, and misses others • Question is … • Is the account (theory, model) useful in the context and for the purpose for which it is used? • Basic forms of reasoning, or, forming inferences, are useful in understanding broad outlines of human cognition • Deduction • Induction • Abduction • Problem solving • Gestalt • Problem Space • Analogy • Skill acquisition

40. Reasoning • Deduction: • derive logically necessary conclusion from given premises • e.g., If it is Friday, then she will go to work - It is Friday, therefore she will go to work • Logical conclusion not necessarily true: e.g., If it is raining, then the ground is dry - It is raining, therefore the ground is dry • Induction: • Generalize from cases seen to cases unseen • e.g., All elephants we have seen have trunks - therefore all elephants have trunks. • Unreliable (but useful): • Can only prove false not true • Abduction: • Reasoning from event to cause • e.g., Sam drives fast when drunk. • If I see Sam driving fast, assume drunk. • Unreliable: • can lead to false explanations

41. FYI - Induction vs. Deduction • Induction: Make observations first, then draw conclusions • Organized data survey (structured analysis, visualization) of raw data provide basis for interpretation process • Interpretation process will produce knowledge that is being sought • Experience of individual scientist (observer) is crucial • Important: selection of relevant data, collection method, and analysis method • Data mining is an important knowledge discovery strategy • ubiquitious data collection, filtering, classification, and focusing is crucial • Deduction: Formulate hypothesis first, then test hypothesis • Via experiment and accept/reject • Data collection more targeted than in induction • Only limited data mining opportunities Mueller, 2003

42. Problem Solving • Process of finding solution to unfamiliar task using knowledge • Complex, time consuming process • Selections not immediately obvious • May require many steps • May involve insight • May use analogy • Solutions often counterintuitive • Several theories, or accounts • Gestalt • Problem solving both productive and reproductive • Productive draws on insight and restructuring of problem • Attractive but not enough evidence to explain “insight” etc. • Move away from behaviourism and led towards information processing theories • Others: Insight, Functional fixedness, Analogy

43. Problem Solving Cycle • One schema – consider “task performance”

44. Insight • Early - Kohler (a Gestalt psychologist) in Canary islands in WWI • Studied problem solving in chimpanzees • Sultan and the Banana: • Learned how to get banana with longer pole • Then given shorter poles that wouldn’t reach • Flash of “insight”, Sultan put the poles together • Sudden perception of useful or proper relations • Solutions will sometimes “spring to mind” • Pieces fall into place • First attempts to solve don’t work • Production hindered by unwarranted assumptions. • Insight occurs when the assumption is removed Strayer, Utah: http://www.psych.utah.edu/psych3120-classroom/

45. Problem solving (cont.) • Problem space theory • Problem space comprises problem states • Problem solving involves generating states using legal operators • Heuristics may be employed to select operators e.g. means-ends analysis • Operates within human information processing system e.g. STM limits etc. • Largely applied to problem solving in well-defined areas e.g. puzzles rather than knowledge intensive areas

46. Problem solving (cont.) • Analogy • Analogical mapping: • novel problems in new domain? • use knowledge of similar problem from similar domain • Analogical mapping difficult if domains are semantically different • Skill acquisition – e.g., “expert” performance • Skilled activity characterized by chunking • lot of information is chunked to optimize STM • Conceptual rather than superficial grouping of problems • Information is structured more effectively

47. Individual Differences • Long term – Sex, physical and intellectual abilities • Short term – Effect of stress or fatigue • Changing – Age • Dix says ask: • Will design decision exclude section of user population? • (or, more generally) How does design differentially affect sections of the population? • i.e., Universal Usability

48. The “Model Human Processor + Attention” isSimilar to Ware (2004) Model • We’ll look at one more model of cognitive (and visual) processing • All are in fact much the same, but focus on different goals

49. The “Model Human Processor + Attention” isSimilar to Ware (2004) Model • We’ll look at one more model of cognitive (and visual) processing • All are in fact much the same, but focus on different goals • Card et al. model was developed in context of predicting user performance • E.g., set parameters and perform simulation • Ware’s model includes much the same elements, but focuses on those which are most relevant for processing of visual information in context of task performance

50. The “Model Human Processor + Attention” isSimilar to Ware (2004) Model • Sensory store • Rapid decay “buffer” to hold sensory input for later processing • Perceptual processor • Recognizes symbols, phonemes • Aided by LTM • Cognitive processor • Uses recognized symbols • Makes comparisons and decisions • Problem solving • Interacts with LTM and WM • Motor processor • Input from cog. proc. for action • Instructs muscles • Feedback • Results of muscles by senses • Attention • Allocation of resources