2. Models for cognitive ergonomics

1. 2. Models for cognitive ergonomics 2.1. the concept of models 2.2. models in cognitive psychology 2.3. cognitive ergonomics models

2. 2.1. the concept of models “M is a model of A if M can be used to answer questions about A” (Ross, 1983) A modelis a representation of relevant characteristics of an object (Rohr & Tauber, 1984) A model is a description that can be communicated, of a certain aspect of part of the real world, viewed at a certain level of abstraction (Oberquelle, 1984)

3. representation of relevant characteristics of an object Mental model of system system User inter-face user

4. aspect of part of the real world, viewed at a certain level of abstraction Aspect 1, graphics Aspect 2, dialogue User interface system

5. the concept of modelsinternal models & external models External models are for communication, should be represented explicitly Internal models are for “execution”: there is an “agent” who uses the model to make decisions based on the behavior of the model, and to make predictions on the behavior of the modeled reality • if the agent is human: mental model • if the agent is a machine: program, database

6. 2.2. models in psychology 2.2.1. models of human information processing • modern theories, e.g., Barsalou • mental models in Cognitive Psychology 2.2.2. mental models in Cognitive Ergonomics - Norman

7. long term MEMORY THOUGHT working MEMORY input PERCEPTION output MOTOR/BEHAVIORAL 2.2.1. Model of human information processing TASK/ STIMULI

8. ACTIVITITES OF THE COGNITIVE SYSTEM • PERCEPTION: • complex sensorial processes • primary images containing all information about the concrete features • of objects and phenomena • that act direct action upon the sensory systems • (visual, acoustic, kinesthetic, olfactory, gustatory). • MEMORY: • ability to remember, recognize and recall • information is encoded, stored and retrieved. • active: structuring, constructive and creative psychic mechanism. • THOUGHT: the process of information processing in working memory.

9. VISUAL PERCEPTION - A COMPUTATIONAL THEORY (pattern recognition) PRIMARY PROCESSING SECONDARY PROCESSING - RECOGNITION Visual Stimuli Descendant processing Primary sketch 2.5 D sketch 3D Representation texture movement color distance position gestalt geons depth form principles segmentation Processing modules

10. VISUAL PERCEPTION - A COMPUTATIONAL THEORY (pattern recognition) Data-driven (bottom-up ) processing 1. mechanisms of edge detection, processing of texture, movement, color, distance and depth, position and form from processing of shadow of stimulus (2.5-D sketch,) - automated/ modular/preattentional/ unconsciousness processing 2. edge organization - gestalt principles of perception (applied to 2.5-D sketch) : proximity, similarity, good continuation, closure example: IwOuLdLiKeToDrInKaBeErInArOmAnIaNbAr. 3. geons generation (geometrical ions) segmentation of the 2.5-D sketch (intermediary image) in zones of maximal concavity 4. Recognition geons activate from memory the objects made-up by the respective geons, matching the 2.5-D sketch with the representation stored in memory

11. GESTALT PRINCIPLES OF PERCEPTION A. Proximity principle B. Similarity principle D. Closure principle C. Good continuation principle A D C B

12. VISUAL PERCEPTION - A COMPUTATIONAL THEORY • (pattern recognition) • Conceptually driven (top-down) processing • concepts and higher-level processes influence pattern recognition • applied to 2.5-D sketch • recognition in the case of: - verbal stimuli (word superiority effect; sentence superiority effect) • T E C T THE WORK MUST GET DONE. WORK

13. VISUAL PERCEPTION - A COMPUTATIONAL THEORY • (pattern recognition) • I’m zhizkizg tz enzoy zhiz wezk-ezd az thz sezsize. • - objects (object superiority effect) • - human faces • OBS.: Importance of implicit/tacit knowledge about the (statistical) regularities of the world in pattern recognition (physical support of things, reciprocal occlusion, occurrence probability, relative size, position and rigidity of objects etc.). • Violation of tacit assumptions causes visual illusions. This assumptions are not specific (they are applied automatically to any object).

14. The importance of context Illusions - at level of modular processing

15. AUTOMATIC PROCESSING • Learned automatic processing • Frequent association of a stimulus with a response produces a production having a relatively autonomous status. Then, the perception of the stimulus produces the response with no intention to do so (ex. classical conditioning). • in perception: orienting learned reflexes (shifting the attention to important stimuli in the environment - ex.: a speaker who utters your name) • (used by advertising industry) • in memory: upon encoding a particular stimulus, people may activate automatically information associated with it in memory

16. STRATEGIC PROCESSING • EXECUTIVE PRODUCTIONS • cognitive mechanism that establish and execute acquired goals • “if-then” productions • current state of the environment and the cognitive system determine which executive production is fired in order to pursue the high-level goals of the cognitive system • number of executive productions that can be fired at once is extremely limited(one or possibly few) because of centralized strategic resource

17. MECHANISMS OF STRATEGIC PROCESSING • executive productions operates on goals, scripts and reminded episodes to select and coordinate information processing subsystems during goal-directed behavior • repeating particular patterns of strategic processing produces new productions that automate repeated parts of relevant scripts, it freeing executive productions to work on more subtle aspects of task or to perform multiple tasks simultaneously • skills develops as increasing amounts of the processing (that executive production perform) become automated

18. PERFORMANCE CHARACTERISTICS OF INFORMATION PROCESSING • 1. Limited capacity • information processing resource that limits strategic processing was theorized as: • - limited processing energy (analogous with an electrical source) • (if one strategic task require all of the available processing energy, no other task can be accomplish simultaneously) • - a single executive processor that applies and manages executive productions - it can only execute one strategic task at a time but can switch quickly from one strategic to another. Limits on strategic processing arise from the limited ability of the executive processor to switch between the tasks. Elementary operations are: compare / choose / repeat / compute / transform

19. PERFORMANCE CHARACTERISTICS OF INFORMATION PROCESSING • 2. Selectivity • to achieve a goal a cognitive system must be able to select and coordinate information processing subsystems (ex. perceptual, motor, memory), locations in perceptual fields and categories in memory • corresponds to what theorists often mean by attention • selection can be specific (ex.particular ear or eye, particular information in memory)

20. DISCUSSIONS Presence of a single executive does not imply that it controls the entire cognitive system: The executive may primarily schedule and monitor (it may also direct processing when goals are new, difficult and dangerous). Many basic processes in perception and movement, many acquired productions which control many automated skills (ex. driving, typing), lie beyond its scope.

21. SENSORY MEMORY • consist in persistence (prolongation) of the sensorial representation of the stimulus after the stimulus is no longer acting on the receptors • specific to a certain type of sensation • format: neuro-physiological codes • capacity: unlimited but the cognitive system will process further only the relevant stimuli • duration: - visual memory 100 ms - auditory memory 200 ms / 2 s • automatic/ pre-attentional retention of the precategorical information (information is in an unprocessed form)

22. LONG TERM MEMORY (LTM) • all the knowledge the cognitive system owns • unlimited capacity • duration: whole life of neural system • memory systems in LTM: • explicit vs. implicit • semantic vs. episodic (memory of general knowledge about our environment vs. memory of personal events) • format (encoding) of the information: • verbal / analog (images) / semantic (propositional) • retrieval (activation) of knowledge: parallel search process

23. WORKING MEMORY (WM) • activated part of long-term memory • encoding: verbal, analogical (image), semantic (propositional) • capacity: • depending on the level of expertise (by chunking); maximum capacity of attention: 7 +/- 2 chunks • limited capacity for a certain type of information (auditory, visual, motor) • limits for certain types of information are independent one from another (ex. if maximum of visual information is in WM this does not decrease the maximum of auditory information that can be in WM).

24. Relation between LTM, WM and attention when there is a specific goal to be reached when there is no specific goal to be reached LTM LTM WM Attention WM = Attention

25. ORGANIZATION OF KNOWLEDGE IN • LONG TERM MEMORY • EXPLICIT MEMORY: content is accessible to consciousness and can be tested by recall and recognition tests • low level structures • propositional network (semantic memory) • semantic network (semantic memory) • high level structures (complex units of knowledge) • schema/script/frame/plan (semantic and episodic memory) • mental models (semantic and episodic memory) • IMPLICIT MEMORY: content is hardly accessible to consciousness and cannot be tested by recall and recognition tests. • production systems (cognitive and motor skills, priming, conditioned reflexes)

26. SEMANTIC NETWORK - represents semantic contents from well-structured knowledge domains - knowledge are represented by a network of nodes and relation between nodes - nodes represents the concepts and the relations between nodes are labeled - meaning of a concept (or node) is given by the pattern of its relations among which it participates. property breath animal type have skin bird property type fly swim fish have feather type shark dangerous type chicken predator eagle not eatable symbol of power

27. SCHEMAS (Rumelhart, 1980) • represents generic concepts stored in memory underlying objects, situations, events, sequence of events, actions and sequence of actions • they are used for a class of stereotypical situation • they vary the very simple to the very complex • are organized in a hierarchical fashion: • variables which have fixe value (the kernel) • slots with optional values (pheripheral) which can be filled in with particular instances of the concepts. If the instances are not specified then the slots will have default values (prototypes) • can be embedded one in other, e.g., • Human body (Head, Trunk, Limbs) • Head (Face, Ears, Hair) • relation between the elements are in spatial-temporal contiguity (ex. bread - butter) • active processing devices (top-down processing) which produce an interpretation of the world - they adapt reality to knowledge • they are assumed to be shared across individuals (in a culture?)

28. SPECIAL TYPES OF SCHEMAS • SCRIPTS (Shank and Abelson,1977) • schemas for frequently occurring sequences of events in a particular context • 2 categories of variables: roles (filled by persons) and props (filled by objects) • includes: - entries condition • - scenes • - results • scripts are the result of social learning • maintanance of the scripts is guaranted by a set of social contingencies • FRAMES (Minsky, 1975) - static representation • - schemas that do not possess active processors • - between the elements of the frame are enabling or causal relations • - framework that is adapted to fit reality • generic frames (class): • ex.: car ( color, brand, driver, engine, transmission, wheels) • specific frames (instantiated) - in a particular context • ex.: my friend’s car (black, Mercedes, John, 4 wheel drive)

29. FRAME FOR “CAR” gasoline John type Driver type buys Fuel diesel type type flows Liz operates type Engine four-cylinder operates rotates type six-cylinder Transmission standard Fixed values (kernel) rotates steel Specific instances (default values) Wheels alloy

30. Task: Arrange a project meeting Plan (meeting (project)) Consult (information source, information token, project meeting) Identify (information source, information token, project meeting) Search (information source, information token) Retrieve (information token, information source) Store (information token, project meeting, working memory) Select (media message) Identify (long-term memory,constrain, project meeting) Choose (media, constrains ) Send message (meeting (project)) Consult (information token, information source) Identify (information source, information token, letter) Search (information source, information token) Retrieve (information token, information source) Store (information token, information source) Represent (information token, message) Write (information token, message, media) Compare (message, information token) Edit (information token, message) Store (message copy, message file, media) Execute (transaction requirements, message) . frame-based representation: how to create and send message to arrange a project meeting (Keane and Johnson, 1987)

31. Script: eating at a restaurant Entry condition hungry, had money, restaurant open Roles diner, waiter, cashier Props tables, money, chairs, menu, cutlery, food Entry scene Diner enters restaurant. Waiter seats diner at table. Waiter places menu on table. Diner begins to read menu. Ordering scene Diner selects food from menu. Diner signals to waiter. Waiter approaches the table. Diner orders food. Waiter leaves. Eating scene Waiter brings food to the table. Waiter leaves. Diner eats food with cutlery. Diner finishes eating food. Leaving scene Diner signals to waiter. Waiter approaches table. Diner asks waiter for bill. Diner checks bill. Diner approaches cashier. Diner gives cashier bill and money. Cashier checks money. Diner leaves restaurant.

32. MENTAL MODELS - dynamic representation • frames in which the relations between (and attributes of) the elements are analogous to a physical/organizational/procedural structure in the world (component parts and relations between these) reflecting the actual state of affairs in the world. • parts of it become instantiated being triggered by an input (stimulus, problem, event) • can be run producing quasi-continuos simulation of the events (over space and time) and can explain how events occurred (comprises explanatory principles)

33. IMPLICIT MEMORY - TYPES OF KNOWLEDGE • Cognitive and motor skills • they develop from a script-like representation of knowledge • Conditioned reflexes • it develops by association of stimulus with response • Priming • it develops by frequent exposure to a stimulus, modifying the judgement value of the stimulus

34. IMPLICIT MEMORY - KNOWLEDGE REPRESENTATION • representation are in the form of production rules; a production is a condition-action (if-then) pair • IF (condition for triggering) THEN (do these actions) • production rules are organized in production systems • production systems can be general or specific (defining expertise in a certain domain) • they are triggered automatically by categorization of relevant stimuli by matching current state of problem-solving (as a pattern in working memory) or of a stimulus against the conditions of the productions rules • they are hardly accessible to consciousness • when needed, the script (on the basis on which production system has been developed) can be reconstructed

35. A variant of the model, Card, Moran, Newell

36. Some principles • perceptual processor cycle time varies inversely to stimulus intensity • cognitive processor cycle time: • shorter with more task load • shorter with more information • shorter with practice in task domain

37. A variant of the model, Card, Moran, Newell Some laws • Fitts’ law: time T to move hand to target of size S at distance D: Tpos = 100[70~120] msec/bit log2 (D/S + .5) • Power law of practice: time Tnneeded to complete a task at trial n: Tn = T1 nª , wherea = .4[.2~.6]

38. A variant of the model, Card, Moran, Newell Example: simple reaction time

39. A variant of the model, Card, Moran, Newell Example: symbol detection

40. Calculation symbol detection RT = Tperceptual processor + 2 Tcentral processor + T motor processor = 100[50~200] + 2*(70[25~170]) + 70[30~100] = 310[130~640] msec

41. 2.2.2. Mental models from the point of view of Cognitive Ergonomics • The functions of mental models in using complex systems • What type of mental models are needed for using complex systems • How to “measure” mental models

42. The functions of mental models in using complex systems • Planning • execution of task delegation • evaluation • interpretation

43. What type of mental models are needed for using complex systems