Survey of Affective Computing for Digital Home

1. Survey of Affective Computing for Digital Home Tsai-Yen Li Computer Science Department National Chengchi University

2. Outline • Affective computing overview • Research activities • Issues and approaches • Applications in digital home • Research challenges

3. Definition of Affective Computing • Affective computing: computing that relates to, arises from, or deliberately influences emotions • Multidiscipline across several academic fields: such as computer science, cognitive science, psychology, neuroscience, etc. • Multidiscipline inside Computer Science: machine learning, pattern recognition, signal processing, computer vision, speech analysis, computer animation, and protocol design.

4. Issues in Affective Computing • Emotion perception • Emotion modeling • Emotion expression • Emotion communication

5. Research Groups on Emotion • United Stats: • Affective Computing (M.I.T.), OZ (C.M.U.), Joe LeDoux's Laboratory (N.Y.U.), Responsive Virtual Human Technology (RTI), Affect Analysis Group (U. Pittsburgh), etc. • United Kingdom: • U. of Cambridge, U. of Birmingham, U. of York, etc. • Others: • Switzerland, Finland, Netherlands, Canada, Austria, etc.

6. Emotion Perception • Verbal: • Voice: magnitude, pitch, etc. • Dialog: limited vocabulary • Non-verbal: • Facial: feature-based, appearance-based • Physiological symptoms: wearable computer • heart rate variability, brainwaves, respiration, muscle tension, blood pressure and temperature • Input behaviors: emotional mouse • Gestures

7. Computational Models of Affect • Architecture-level models • Mostly based on the OCC model. • How emotions interact with other components such as goal, preference, stimulus, etc. • Task-level models • enhancing system performance on a particular problem-solving task such as NPL, Military applications. • Mechanism-level models • To emulate high-level or low-level aspects of the mechanism involved in emotional processing

8. Research Groups on Emotional Virtual Human • Center for Human Modeling and Simulation, UPenn: by Norman Badler • Kinematics, dynamics, NLP • Virtual Reality Lab, EPFL, Switzerland : by Daniel Thalmann • Motion, crowd, virtual environment • Miralab, Switzerland: by Nadia Magnenat-Thalmann • Facial, cloth, skin

9. Emotion Expression • Emotion mechnism (Oliverira 2003): • Appraisal: OCC model • Copping: expression • Expression types: • Verbal: speech, text • Non-verbal: facial expression, motion, gesture, actions, etc.

10. Emotion Modeling and Expression Language • PAR (Parameterized Action Representation ) • Badler2000, representation lang. • CML (Character Markup Language) • Arafa2003, scripting and representation lang., XML • AML (Avatar Markup Language) • Thalmann2002, scripting lang., XML, emotion • VHML (Virtual Human Markup Language) • Marriott2002, scripting and representation lang. • HML (Human Markup Language): • representation lang., XML • APML (Affective Presentation Markup Language): • scripting and representation lang., XML, emotion

11. Application in Digital Home • Levels of applications • Expression only: Mac, Office agent • Perception only: intelligent tutoring system • Perception and expression: David in AI • Applications in digital home: • Entertainment: emotional virtual characters • Instructional: context-aware virtual presenter • Environmental setting: temperature, light, decoration, emergent alert, etc. • Education: intelligent tutor/nanny • Ingredients for other home applications

12. Research Challenges • Are these results recognized people’s emotions accuracy? • Does it need to be accurate? (joy, anger) • Could the data collected in laboratories be placed a digit home ? • Analogy of computer vision. • Ethics issue • Consider at as low level as possible. • Other technical challenges • Distinguishability, Naturalness, etc.