The Affective Loop Kristina Höök

1. The Affective LoopKristina Höök

2. Kristina Höök • Swedish Institute of Computer Science www.sics.se • Professor in human-machine interaction at the IT-university, Stockholm University/KTH, www.dsv.su.se

3. Mostly hate?

4. Emotions do not hinder rational thinking! • Emotions are the basis for rational thinking • Emotions and reasoning is based both in the brain but also in our bodies

5. Gestures & tangible interaction • Laban: • Analysis of dance, factory work, human movement • Certain body postures and gestures ”encourage” certain emotions • Ishii: • Tangible interaction, embodied

6. Affective computing Make computers: • Express emotions • Influence user emotions • Understand user emotions • Allow users to express emotions • ”Have” emotions? • Express emotions

7. Facial expressions

8. Affective computing • Make computers: • Express emotions • Influence user emotions • Understand user emotions • Allow users to express emotions • ”Have” emotions? • Influence user emotions

9. AIBOs

10. Affective computing Make computers: • Express emotions • Influence user emotions • Understand user emotions • Allow users to express emotions • ”Have” emotions? • Understand user emotions

11. Recognize emotions

12. Affective computing Make computers: • Express emotions • Influence user emotions • Understand user emotions • Allow users to express emotions • ”Have” emotions? • Allow users to express emotions

13. SenToy

14. Affective computing Make computers: • Express emotions • Influence user emotions • Understand user emotions • Allow users to express emotions • ”Have” emotions? • ”Have” emotions?

15. Influencing Machine

16. Emotional in Florida?John Thackara A culture is surely in deep, if not terminal, crisis when engineers start talking about the design of emotions. Now even Don Norman is at it. "In the past year I've been catching up on the scientific work in emotion", he writes in the programme for an up-coming CHI conference; "the result is a new theory of emotion, with implications for the development of autonomous machines and robots as well as to design". Now Professor Norman is a great man, but this is both ridiculous and....sad. To talk about putting emotions into products, in the same way that Del Monte puts tomatoes into cans, is literally meaningless. I will spell it out: emotion is a quality of consciousness. Consciousness is something that people have, and machines don't. It follows that the only "implication for design" of emotion is: steer well clear of the subject, or risk looking ridiculous.

17. The affective loop... • Inspired by Influencing Machine by Phoebe Sengers & collegues and SenToy by Ana Paiva & collegues • Emotions reside in and are induced by both body and mind • Not recognise emotion as a one-step process – instead enter an interaction? Needs strong narrative (or context)

18. Influencing MachinePhoebe Sengers & others • Intimate relationship between humans and computational ‘beings’ • Child-like drawings drawn in real time project on the walls and music reflects the agent’s emotional state • Users can send the agent postcards in a mailbox • The postcard affects the agent’s emotional state, which in turn changes the pictures that are being drawn and music played

19. Affective inputMailbox

20. Affective output: childlike drawings & music Inspiration: kids’ drawings • Affective expressions: • Complexity of shape • Pressure of pen • Colour • Speed • Shape • Size • ....

21. The point? • To provoke our cultural notions of whether a machine can ”have” emotions and act on its own • (..but also to be charming)

22. The enthusiastic quartet ”We might have influenced his feelings, through holding up a pink cards. Think of how a child would react, sits there and feels really sad and is thinking about sad things, and then I give him a pink card, and he becomes happy, though he is still somewhat cold, so I give him a yellow card, so he becomes warm.”

23. What they do vs. what they say... Video observation: One of the women made an interesting comment: she pointed at the computer under the table with the table cloth, and asked the man in the group whether this computer was in fact connected to the machine? She meant that if it was connected, then the Influencing Machine was just a computer – not a machine in its own right. It seems that to her a computer cannot be what she perceives that the Influencing Machine is. If it is a computer, it must be predictable, not influenced by them.

24. Affective loop...? • No ”model” of users’ emotion • Tangible input • Interaction over a longer time span: 20 – 40 minutes

25. SAFIRA demo: SenToy in FantasyA • A role-playing game • Agents in the game have emotional states • To advance in the game, the user has to understand the agents’ emotional states, get magical tools, and fight magical fights • The user has a plush toy, SenToy, with sensors that s/he can use to direct his/her character and express its emotions

26. ... ... (( )) Affective input: SenToy Gloat Sad Happy Fear Angry Surprise

27. User experience

28. Affective loop...? • No ”model” of users’ emotion • Tangible input • Gestures – induce emotional state through body posture? • Make narrative in the game, avatar expressions, and gesture input into a coherent whole that captures the mind & body • Interaction over a longer time span 30 minutes – 2 hours

29. Affective loop? • The user expresses an emotion to a system (intentionally, not true emotion) • The system computes an affective response making sense in the application (be it a game, a narrative, an e-commerce application, ...) • The response speaks to our senses and mind (music or an activity in the game that induces a state of stress or fear) • The user reacts affectively

30. New group at SU • Affective bandwidth • Gestures • Multimodal output • MMS

31. Narrative intelligence • Narrative context for emotions • Collaborative storytelling • Interactive storytelling with synthetic characters

32. MUD Centre of the Cassiandora square marketplace. A small, ivy-covered milestone at your feet reads “Mercinae”. - goods We have nothing to sell right now; please come back later. – sigh You sigh heavily. – citizens Allanon; Grim, the Alchemist; Mage Sparky, the Incandescent; Cinaed y Llawen; Feanor. – tell cinaed excuse me, milord, but I cannot find any clothes in the city stalls, and I’m freezing. Can you help? You tell Cinaed y Llawen, “Excuse me milord, but I cannot find any clothes in the city stalls, and I’m freezing. Can you help?”. – Cinaed y Llawen tells you, “Hmm, let me check the city stores. One moment please…”. – Cinaed y Llawen tells you, “I’m afraid the city is very low on cotton and wool at the moment, but I have put a simple robe in the clothing stall for you <smile>”. – tell cinaed oh, that is all I need. My thanks for your aid, baron You tell Cinaed y Llawen, “Oh, that is all I need. My thanks for your aid, baron”. – buy robe You buy a grey robe for 40gp. – wear robe You are now wearing a grey robe. Cinaed y Llawen tells you, “Any time, Feanor <smile>. Life be with you”.

33. Palace

34. Sam Imagine a 3-D animated character who can pass multiple toys backand forth through the seemingly immutable boundary of a projectionscreen, and thus tell stories and share experiences together with a child.

35. Pirates (Göteborg!)

36. Research questions • Artistic freedom versus proper models • Models: appraisal versus discrete emotions • Understanding user emotions from body sensors? • New input/output modalities • Emotions as social constructs • Relationship narrative - character

37. Summary Cool expressions: • Affective computing • Pervasive computing • Synthetic characters • Narrative intelligence • Social capital and communities

38. Jakub Wejchert ”We live in a time in which objective reality is given much more importance than the subjective. It is in this context that most technological and scientific developments are carried out. The danger is that the personal, the subjective and the world of inner experience is becoming reduced to an insignificant by-product.”

39. kia@dsv.su.se www.sics.se/~kia www.sics.se/safira/