430 likes | 951 Views
The Architectural Role of Emotions in Cognitive Systems. Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute. Outline. Outline. Emotions are adaptive Can inform cognitive system design
E N D
The Architectural Role of Emotions in Cognitive Systems Jonathan Gratch USC Institute for Creative Technologies Joint work with Stacy Marsella USC Information Sciences Institute
Outline Outline Emotions are adaptive Can inform cognitive system design Ground in implemented cognitive system • Mission Rehearsal Exercise system • Cognitive Appraisal Theory • Illustrate impact on architecture design General implications for cognitive systems
Outline Adaptive Role of Emotions • Revolutionary progress in emotion research • Neurophysiology of emotion (Damasio, LeDoux) • Appraisal theories (Frijda, Lazarus, Scherer) • Emotions appear adaptive (in moderation?) • Decision-making ─ Focus of attention • Learning ─ Social relationships • Belief formation ─ Communication • Growing interest in “emotional” systems with focus on modeling human behavior • HCI (non-verbal recognition and generation) • User and human behavior modeling • Believability/Entertainment
Outline Architectural Perspective • Can inform intelligent behavior in general • Motivate behavior • Balancing competing goals • Balancing reaction and deliberation • Disambiguating stimuli in light of existing beliefs and commitments • Abstract and formalize as information processing • Not new idea: • Simon(1967), Oatley&Johson-Laird(1987), Sloman • Revisit in light of new findings • In intelligent systems • In theories of emotion
Grounding: Virtual Humans Face-to-face interaction Verbal & non-verbal behavior Swartout,Gratch, Hill, Hovy, Johnson, Marsella, Narayanan, Rickel, Traum, … Marsella, Johnson & Labore
Mission Rehearsal Exercise Social Training Simulation • Explore high-stakes social interactions in safety of VR
Mission Rehearsal Exercise • Team decision-making in crisis situations: • Non-scripted real-time interactions • Planning, replanning, and plan execution • Teamwork, distributed authority and responsibility • Collaborative, mixed initiative dialogue • Multi-party conversations • Verbal and non-verbal communication • Emotionally-biased behavior
Mission Rehearsal Exercise • Assumptions/Limitations: • Tightly focused task-related dialogue • Near-expert decision makers • Stylized vocabulary (military speak) • Stylized virtual environment
Soar Action Selection Dialogue Perception Planning Emotion NLU pragmatics NLG Voice Input World Simulator Speech Recognition (HTK) Semantic Parser Projection System Communication Bus Animation System Vega BDI Haptek Audio (Protools) Motion/ Gesture Scheduler (Beat) Speakers (10.2) Text to Speech (Festival)
Cognitive Representation Present Future Past Eagle 1-6 Assist Desire: LT Belief: False Child Healthy:False Accident Intend: False Blame: unresolved Assist Eagle 1-6:False Get Medevac Responsibility:LT Intend: True Medevac Available:True Child-Healthy Desire: SGT Belief: False Probability: 75% Soar’s Working Memory Planning Perception Dialogue Action Soar operators
Cognitive Representation Present Future Plans Past Events Eagle 1-6 Assist Desire: LT(+50) Belief: False Child Healthy:False • Causal Interpretation • Combines decision-theoretic plans with models of belief and intention • Uniform representation of past, present, future • Agent centric subjective view Accident Intend: False Blame: unresolved Assist Eagle 1-6:False Get Medevac Responsibility:LT Intend: True Medevac Available:True Child-Healthy Desire: SGT(+80) Belief: False Probability: 75%
Architectural Role of Emotion • Began with view “emotion as veneer” Ended up as central organizing construct • Initial problem: • how to convey emotion in interactive setting? • Built mechanism to infer plausible emotions • In response to simulation events • In response to user interventions • But discovered resolved architectural issues • Coherence is more than skin deep • Build it and they will come
How to convey emotion • Cognitive Appraisal Theory • Influential and well-established theory Arnold, Frijda; Lazarus; Ortony, Clore & Collins; Scherer; Smith • Emphasizes tight coupling between • Emotion • Cognition • Motivation
Cognitive Appraisal Theory Coping Problem-focused Emotion-focused Goals, Beliefs External Events Appraisal Emotion Coping Smith and Lazarus’ cognitive-motivational-emotive system
Appraisal Goals, Beliefs External Events Appraisal • Appraisal = Situation assessment • Compare beliefs, desires and intentions • with • external circumstances
Appraisal • Characterize via appraisal variables • Desirability • Likelihood • Urgency • Unexpectedness • Causal attribution (causality, agency, blame/credit) • Coping potential (controllability, adaptability) • Superset of criteria considered by cog systems • Decision theory: desirability, likelihood • Scheduling: desirability, urgency
Coping Strategies • Coping = Response strategy • Characterized by ontology of coping strategies Emotion External Events Goals, Beliefs Coping Problem-focused Emotion-focused
Coping Strategies • Problem-focused (act on the world) • Action execution • Planning • Seek instrumental social support • Analogous to: • Deliberative or reactive problem solving • Team negotiation
Coping Strategies • Emotion-focused (act on belief) • Denial • Find silver lining • Shift blame • Distancing • Not typically considered by cog systems systems • More than a decision (e.g. abandon current plan) • Provides self-justification for why • Related to motivational / explanatory coherence • Leads to persistent change in behavior
Modeling Appraisal and Coping Future Past Soar’s Working Memory Planning Perception Dialogue Action Soar Operators
Modeling Appraisal and Coping • Appraisal as plan-evaluation • Causal interpretation mediates agent-environment relationship • Define appraisal variables in terms of features of interpretation • Fast, reactive, parallel • Coping as generalized plan critics Map to operators that change interpretation • Problem-focused execute step, add plan step • Emotion-focused • Denial Change belief • Find silver lining Change utilities • Shift blame Change causal attribution Dialogue moves • Distancing Drop goal / intention
Émile: Architectural Manifestation The Emotional Octopus
Appraisal Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame:unresolved Child-Healthy Desire: SGT Satisfied: False Perspective: Self (Sgt) Desirability: -80 Likelihood: 100% Blame/Credit: unresolved Distress: 80 Sgt’s Appraisal of Accident from his perspective
Appraisal Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Child-Healthy Desire: SGT Satisfied: False Perspective: Lieutenant Desirability: -80 Certainty: 100% Blame/Credit: unresolved Distress: 80 Distress: 80 Sgt’s Appraisal of Accident from Lieutenant’s Perspective
Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Child-Healthy Desire: SGT Satisfied: False Distress: 80
Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Child-Healthy Desire: SGT Satisfied: False Sgt’s Own Perspective Distress: 80
Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Get Medevac Responsibility:LT Child-Healthy Desire: SGT Satisfied: False Probability: 75% Make Amends Distress: 80 Distress: 80 Problem-Focused Coping: Form intention to help Boy
Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: MOM Get Medevac Responsibility:LT Child-Healthy Desire: SGT Satisfied: False Probability: 75% Shift Blame Make Amends Distress: 80 Distress: 80 Emotion-Focused Coping: Blame Mother
Coping Eagle 1-6 Assist Desire: LT Satisfied: False Child Healthy:False Assist Eagle 1-6:False Accident Blame: unresolved Get Medevac Responsibility:LT Child-Healthy Desire: SGT Satisfied: False Probability: 75% Shift Blame Make Amends Distress: 80 Distress: 80 Personality
Architectural Implications • Emotion as central control construct • Planning (inform course-of-action selection) • NLU (inform reference resolution) • Dialogue (prompt dialogue initiative) • NLG (biases sentence generation strategies) • Non-verbal expression
General Implications • Emotion and Reflection • Appraisal is form of self-reflection / focus of attention • Emotion as decision-making • Generalization of decision-theory • More to the world than probabilities and utilities • Emotion and plausible reasoning • Emotion-focused coping motivate preference/beliefs • Attempt to construct coherent motivational explanation • Non-rational but adaptive?
General Implications • Emotion and Learning • Focus learning on “emotionally salient” events • Appraisal variables as features / case indexes
Conclusion • Emotion is form of information processing • Arguable adaptive • Juggling competing goals and commitments • Focusing cognitive resources • Enforcing coherence • Arguable unexplored by cognitive systems
Dialogue Example: Sgt’s Behavior Secure Area Secure 12-4 Secure 8-12 Render Aid Secure 4-8 Secure LZ Secure Accident 1 Focus=1 Lt: U9 “Secure a landing zone” Committed(lt,7), 7 authorized, Obligation(sgt,U9) Sgt: U10 “First we should secure the assembly area” Disparaged(sgt, 7), endorsed(sgt,2) Lt: U11“Secure the area” Committed(lt,2), 2 authorized, Obligation(sgt,U11) Sgt: U12 “Yes sir” Committed(sgt,2), Push(2) Goal7:Announce(2,{1sldr,2sldr,3sldr,4sldr}) Goal8: Start-conversation(sgt, {1sldr,2sldr,…},2) Goal8 Sgt: U13 “Squad leaders listen up!” Goal7 Sgt:U14 “I want 360 degree security” Push(3) Goal9:authorize 3 Goal9 Sgt:u15“1st squad take 12-4” Committed(sgt,3), 3 authorized Pop(3), Push(4) Goal10: authorize 4 Goal10 Sgt:u16“2nd squad take 4-8” Committed(sgt,4), 4 authorized Pop(4) … A10: Squads move A10: grounds U13-U18,… ends conversation about 2, realizes 2 Pop(2), Push(7) Decomposition Area Secure Squads in area A=Lt, R=Sgt A=Lt, R=Sgt 2 7 Decomposition 3 4 A=Sgt, R=1sldr A=Sgt, R=2sldr 5 6 A=Sgt, R=3sldr A=Sgt, R=4sldr
More than a theory of emotion • Appraisal as a mediating variable • Direct mappings (e.g. Hayes-Roth personality model) • Indirect mappings • Direct: More links, No insight on how to map • Indirect: more constrained. More modular World state Beliefs Behavior Desires Personality World state Appraisal Variables Beliefs Behavior Desires Personality
Mediating Variable • Appraisal Mediates Personality Personality Variable Appraisal Variables Behavior e.g Extroversion Control Hope Penley & Tomaka (2002) • Appraisal Mediates Culture Culture Variable Appraisal Variables Behavior e.g. Uncertainty avoidance Threat Fear Kupperbusch et al
Mediating variables • Coping mediated by appraisal • Undesirable & Controllable Distress Problem directed coping • Undesirable & Uncontrollable Distress Emotion directed coping