ALMA- A Layered Model of Affect

1. ALMA- A Layered Model of Affect Wenkai Dai

2. Motivation • Software should provide users with a more human-like interface • Popular application areas for virtual character • Like virtual training environment • Portable personal guides • Interactive fiction • Storytelling systems • E-commerce application in interface of consumer electronics

3. Introduction • In order to increase the believability of the virtual conversation partner, address the modeling and emulation of human-like qualities • Such as personality and affective behavior • The emotion modeling inspired by OCC model • Systems with multiple characters also was proposed • Shift from single to multiple agents bears advantages • Incorporate the personality and affective states to extends a character’s conversational and social repertoire

4. Introduction • Tree major affective characteristics • Emotions: short affect • Moods: medium affect • Personality: long affect

5. Introduction • The fact : different kinds of affect influence different aspects of behavior in humans • Emotions bias action whereas moods bias cognition • Personality understood as a general affective framework of individuals • Personality interferes other kind of affect like emotion and mood

6. The Virtual Human Project • Aim at development of concepts and techniques for human-like conversational characters • Realized by the combination of computer graphics technology with multimodel dialog generation • In VirtualHuman, the multimode dialog generation is extended by the affect generation module to produce a more human-like behavior

7. Introduction to System • A virtual learning system displaying human-like sized character in a 3D environment • In order to provide the group learning experience to human student acquires new knowledge together with the virtual student Sven and The Virtual teacher Valerie • Her main task is to provide information about the subject of lesson • Valerie asks questions about the recently conveyed information • Sven should support the human student by giving hints • Human student and Sven compete with each other for answering • The more intelligent Sven is defined the faster he answers

8. Introduction to System • At the beginning, a third part to specify the personality profile to both of virtual characters • Personality profile defines general behavior • Valerie’s personality parameters are: • severe versus tolerant and dry versus flowery which influence the wording of her explanations and her conversational gestures. • Sven’s personality parameters are: • ambitious versus lazy. They define the amount of time after that Sven gives an answer, but also influence the quality of his answers.

9. Introduction to System • During runtime, each dialog contribution • including the spoken or typed input of the human student • appraised by each character’s affect generation module, which then alters the character’s affective state. • Based on that state, the facial expressions, the wording, idle behavior and gestures of the virtual humans are generated.

10. A LAYERED MODEL OF AFFECT

11. A LAYERED MODEL OF AFFECT • Emotion help controlling behavior aspect but not enough • Emotion is the short term affect but behavior changes over a long period time • Which behavior should be considered • Verbal and non-verbal expressions i.e. wording the length of phrase, speech parameter and facial expression • Cognitive process: making decisions, motivation, appraisal

12. The Analysis of type behavior • Short-term behavior: facial expression, gestures, wording of verbal expression • Medium-term & long-term behavior: the process of making decision, motivation of characters • Mixed-term behavior: idle behavior like eye blink and posture-changes • Different kind of affects, differ in temporal characteristics

13. Emotions Characteristic • Emotions reflect short-term affect, • bound to a specific event, action or object • After its elicitation emotions usually decay and disappear of the individual’s focus

14. Moods Characteristic • Moods reflect medium-term affect, • Not related with a concrete event, action or object. • Moods are longer lasting stable affective states, influence on human’s cognitive functions

15. Moods Characteristic • mood changes can be divided into • (a) the onset of a mildly positive or negative event • (b) the offset of an emotion-inducing event, • (c) the recollection or imagining of emotional experience • (d) the inhibition of emotional responding in the presence of an emotion-inducing event.

16. Personality Characteristic • Personality reflects long-term affect. • Personality reflects individual differences in mental characteristics. • A common personality schema is the Big Five model of personality • It specifies the general (affective) behavior by the traits openness, conscientiousness, extraversion, agreeableness and neuroticism.

17. Define the Modeling Affect for Behavior • Question: how to define the affect to be computer • How they interact with each other • Solution: OCC model of emotions combined with the five factor model of personality • OCC is a cognitive model of emotions • OCC is based on the concept of appraisal and intensity

18. Main frame of System • Based on Emotionengine • Moods model as medium-term for this engine • describes mood with the three traits pleasure (P), arousal (A), and dominance (D)

19. Appraisal • Act based appraisal according to OCC-model • Acts reflect intention • DialogActs for utterance(e.g, Encourage) • AffectsActs for affect signals(e.g Smile) • Mapping on internal OCC-variables according to • Role • Contxt • Simplifies affect generation in script and plan based application

20. Emotions • Emotions defined as valenced reactions to events • The intensity of emotions underlies a natural decay, which can be configured by several decay functions • Generation by Emotion Engine in real-time • 24 types of emotion

21. Mood • mood as “an average of a person’s emotional states across a representative variety of life situation • Mood is distinguished from emotions in that it reflects an individual’s stable or longer lasting affective state • Mehrabian describes mood with the three traits pleasure (P), arousal (A), and dominance (D)

22. PAD Space • PAD mood space uses axes ranges from -1.0 to 1.0 for each dimension • Mood is described with the following classification of each of the three mood space axis • +P and –P for pleasant and unpleasant • +A and –A for aroused and unaroused • +D and –D for dominant and submissive

23. Example of Mood Octant • a person’s discrete mood description is relaxed • Value of the trait pleasure is positive • Value of the trait arousal is negative • Value of trait dominance is positive • The strength of a current mood by its distance to the zero point of the PAD mood space • values: 0.25 pleasure, -0.18 arousal, 0.12 dominance, its discrete mood description is slightly relaxed • A mood represents a point in the PDA space

24. Reliable mapping • When starting the medium-term affect computation, essentially to define an individual default mood for all characters, as a mood starting value • Mapping define a relationship between the big five personality traits and the PAD space

25. Reliable mapping • By mapping, EmotionEngine use the big five personality model to define a characters personality • Thereby able to compute a default mood for characters

26. Example of Reliable mapping • Using this mapping a person, whose personality is defined with the following big five personality traits: • openness=0.4, conscientiousness=0.8, extraversion=0.6, agreeableness=0.3, and • neuroticism=0.4 • Who has the default mood slightly relaxed • (pleasure=0.38, arousal=-0.08, dominance=0.50).

27. More Challenging task • It is the simulation of human-like mood changes • Instead of modeling moods changes, taking the emotions as the mood changing factor • In order to realize this, emotions must be related to a character’s mood • Put emotion into PAD space

28. Mapping of OCC Emotions into PAD space

29. Mapping of OCC Emotions into PAD space • How the intensity of emotions influences the change of the current mood • For example, if a person’s mood can be described as slightly anxious and several events let the person experience the emotion fear, the person’s mood might change to moderate or fully anxious. • The simulation of mood changes by the active emotions generated by the EmotionEngine

30. Mapping of OCC Emotions into PAD space • All active emotions are used as input of the pull and push mood change function. • It first computes the virtual emotion center of all currently active emotions in the PAD space • The virtual emotion center represents a point in the PAD space and has an intensity that is the average of all active emotions’ intensity • Its maximum intensity is 1.0. If no active emotions exist, no virtual emotion center exists and the current mood is not influenced by active emotions

31. The pull and push mood change function • Pull phase: If the current mood position is between the PAD space’s zero point and the virtual emotion center, the current mood is attracted towards the virtual emotion center • Push phase: If the current mood is beyond (or at) the virtual emotion center the current mood is pushed away, further into the current mood octant in which the mood is located

32. The pull and push mood change function • The intensity of the virtual emotion center defines how strong the current mood is attracted respectively pushed away • Usual mood change time: the amount of time the pull and push mood change function needs to move a current mood from one mood octant center to another one • In system usual mood change time is 10 minutes

33. Move Back to the Default Mood • our mood simulation is that the current mood has a tendency to slowly move back to the default mood • Generally, the return time depends how far the current mood is away from the default mood • the longest distance of a mood octant (√3) for defining the mood return time. Currently this is 20 minutes.

34. Affect Computation

35. Affect Computation

36. The appraisal rules • structured in hierarchical appraisal classes, which are related to input types • Basic appraisal rules. They define how a character appraises events, actions, and objects related to him • Act appraisal rules. They define how a character appraises its own acts and other characters acts. • Emotion display appraisal rules. They define how a character appraises its own emotion displays, but also other characters’ emotion displays • Mood display appraisal rules. They define how a character appraises its own mood displays, but also other characters’ mood displays

37. The Running Example

38. Examples of character behavior

40. Q & A