Peggy Wu - Incorporating Psychology Theories into Simulations & Serious Games

1. Incorporating Psychology Theories into Simulations & Serious Games SIFT – Smart Information Flow Technologies Peggy Wu (PWu@siftech.com), Tammy Ott, Sonja Schmer-Galunder, Christopher Miller, Jeff Rye, July 2015

2. SIFT Introduction Smart Information Flow Technologies, LLC (SIFT) is a Consulting Research & Development company: Founded in 1999 Headquartered in Minneapolis MN Offices in Boston, San Diego, Washington DC $7.0M+ per year in revenues 35 Full time employees 30+ Advanced Degrees (Psychology, Computer Science, Engineering) Advancing research to enhance information flow in human-computer and computer mediated human-human interactions 2

3. Technical Personnel Advanced degrees in: Artificial Intelligence Aviation Autonomous systems Control Theory Cognition Cultural Anthropology Linguistics Psychology NLP User Experience 27 July 2015 3

4. Capabilities • User Interfaces and Human Centered Interaction Design Anthropological analyses Linguistic analysis Modeling: – Human and Human + System performance – Tasks – Functional Representations – Cognition – Software Structure – Information Flow – Context-to-presentation Matching – Etiquette/Politeness – Cultural Perceptions • Intent/Plan Recognition and Task Tracking Intent specification and use Context Sensitivity Adaptive and Adaptable User Interfaces and Automation Mixed Initiative System Design Associate Systems Decision Aiding Systems RT Planning and Scheduling Formal and Testbench security analysis Software Development Experimental designs & statistical analysis • • • • • • • • • • • • • = Smart Information Flow Technologies 27 July 2015 4

5. Customers & Collaborators Our customers and collaborators include: BBN, BAE, CMU, Cornell University, GMU, DARPA, Honeywell, Lockheed, NASA, NIST, Oxford, UofM, UMD, USAF, USC, US Army, US Navy, VA 5

6. SIFT’s Human Behavior Modeling Work 2014 SAGA 2015 NAPP 2009+ ELADIS–affective reactions occur to unconsciously learned stimuli and increase recognition of people. Utilized SCR and HR. 2014 R3–Reading, Remembering, Revising 2014 ATHENA–zero intrusion workload detection 2009+ ADMIRE–Assessing how people in a social network feel about each other given their politeness behaviors 2007 TLTS 2012+ AD ASTRA–non- intrusive assessment of individual and team psycho- social health 2010 GRASP– Keyboard dynamics for cybersecurity 2014 CAMO–zero intrusion workload detection via text and Keyboard dynamics 2008 SUPPORT 2010 EVA Multi Cultural Interactions 2006 Phrasebook 2000 RPA Independent LifeStyle Assistant (ILSA) A NIST ATP Program 2013+ ANSIBLE 2011 CALM 2007 E4D2 2003 ILSA 6

7. SIFT’s Human Behavior Modeling Work Games for Health 2014 SAGA 2015 NAPP 2009+ ELADIS–affective reactions occur to unconsciously learned stimuli and increase recognition of people. Utilized SCR and HR. 2014 R3–Reading, Remembering, Revising 2014 ATHENA–zero intrusion workload detection Workload Detection Human 2009+ ADMIRE–Assessing how people in a social network feel about each other given their politeness behaviors Sociolinguistic Modeling 2007 TLTS 2012+ AD ASTRA–non- intrusive assessment of individual and team psycho- social health Modeling Humans for Language & Culture Training 2010 GRASP– Keyboard dynamics for cybersecurity 2014 CAMO–zero intrusion workload detection via text and Keyboard dynamics VR for Social & Psychological 2008 SUPPORT 2010 EVA Multi Cultural Interactions 2006 Phrasebook 2013+ ANSIBLE Health 2000 RPA Independent LifeStyle Assistant (ILSA) A NIST ATP Program 2011 CALM 2007 E4D2 2003 ILSA 7

8. Language & Culture Training Since 2003, developing Computational Model of Human-Human Interaction – Etiquette EngineTM Software Module driving individualized interpretation of politeness and behavior selection “swappabl e cultural modules” Visualizati on of Etiquette Validation of results with naïve student ratings 27 July 2015 8

9. 27 July 2015 9

10. Etiquette, Politeness and Compliance Grice’s sociological concept of Face  Universal, uniquely human Brown and Levinson– cross-cultural, socio- linguistic, human-human, politeness model  Positive Face & Negative Face  Each interaction poses “Face Threat” which is a function of Power, Familiarity and Imposition  Politeness is used to redress Face Threat We used this concept to create Embodied Conversational Virtual Agents for Culture & Language Training What about other applications?  Natural Language Interfaces for other uses?  Does Politeness affect performance? 27 July 2015

11. Do machines have “etiquette”? Should they? Reeves & Nass (1996)  Humans are equipped with schema for interaction with complex agents  Complex machines activate those schema– unless we fight hard to counteract them. Parasuraman & Miller (2004)  “Polite” (non-nagging) decision aids improve:  Trust  Perceived workload  Overall performance  Over and above automation reliability Wu, Ott, & Miller (2009)  “Polite” Automation affects:  Compliance  Trust  Perceived workload  Reaction time GOOD COMMUNICATION STYLE POOR COMMUNICATION STYLE 100 80 60 40 20 0 HIGH LOW AUTOMATIONRELIABILITY 11

12. NASA’s Vision for Deep Space Exploration 12

13. Earth to ISS 354 km / 220 miles. No COMM delay, virtually always connected. 27 July 2015 13

14. Mission Control flies ISS, not Astronauts 27 July 2015 14

15. Earth to Moon 383,000 km / 238,000 miles 1.2s one way delay 27 July 2015 15

16. Earth to NEO Asteroid 951 Gaspra 1,929,378 km / 1,198,860 miles 6.5s one way delay 27 July 2015 16

17. Earth to Mars 56,005,100 km / 34,800,000 miles, 3min (when its closest) 402,000,000 km / 250,000,000 miles, 24min Current Mars DRM: 6 months, 18 months on surface, 6 months back, plus high crew autonomy! 27 July 2015 17

18. Orion Interior? 27 July 2015 18

19. 27 July 2015 19

20. 27 July 2015 20

21. Motivation Sensory Monotony, Social Monotony known to cause: Adjustment disorder, Fatigue, drops in productivity, anxiety, hostility, risk-taking and rule-breaking behaviors 27 July 2015 21

22. ANSIBLE 27 July 2015 22

23. ANSIBLE World Map 27 July 2015 23

24. ANSIBLE 27 July 2015 24

25. 27 July 2015 25

26. Virtual Environment as Extension to Real World? Humans can interact with Virtual Agents in socially meaningful ways Reeves and Nass (1996) “The Media Equation” Brown and Levinson (1987) Politeness Theory Socially appropriate VA behaviors can affect Human Performance Miller et al (2010) Human Computer Etiquette 26

27. Virtual Environment as Extension to Real World? Learning is a cognitive process that occurs in a social context, and VEs can provide the social context, so perhaps it’s possible to learn positive behaviors in VEs Virtual Environment with social context Bandura (1986) Social Cognitive Theory “Immersive Entrainment” E.g. Yee & Bailenson (2007) “The Proteus Effect” 27 27 July 2015

28. Virtual Environment as Extension to Real World? Hasson (2008) “Neurocinematics” Konigsberg (2007) Activation of Mirror Neurons Ramachandran (2012) The Tell-Tale Brain Iacoboni (2008) Mirroring People 28 27 July 2015

29. Virtual Environment as Extension to Real World? Köhler effect: phenomenon that occurs when a person works harder as a member of a group than when working alone. Industrial psychologist Otto Köhler found members of Berlin Rowing Club worked harder when part of a group vs. as individuals 29

30. Operationalized Evidence Based Strategies in VE Review of 80+ publications, seeking evidence based strategies for promoting psychosocial health, brainstormed implementation ideas for VE+COMM delay Combat Sensory Monotony • Weather, Lighting, Sun Rise/Set • Virtual Plant life, Nature Scenes • Virtual spaces for crew discretionary events, providing different visual stimuli Combat Social Monotony • Virtual pets • Virtual agents as actors • People watching tied to Earth • Vary types of interactions e.g. support personal pursuits and hobbies (3D modeling) Recall positive memories, memories of gratitude, acts of kindness (Lyubomirski) • Game - pictionary for shared memories • Scavenger Hunt • Virtual care package Interpersonal skills training • Motivational interviewing • VA to consult and provide guidance on avoiding global criticisms, contempt, defensiveness, stonewalling etc. Continuing rituals (Xygalastas et el., 2011) • Birthdays, thanksgiving, christmas Shared experiences, laughter and humor (Cousins, 1976) • Virtual vacations • Comedy club • Turn taking games Meaningful work: reflecting on past and future, expectations and responsibilities (Baumeister et al., forthcoming, Frankl, 1991) • Public Interaction area • Educational outreach • Music, Creative expressions, personal pursuits Mere belonging to increase connectedness • Sharing random/superficial commonalities e.g. Sports jersey Mindfulness and meditation 30

31. Is it going to work? “If we knew what we were doing it wouldn’t be research.” - Albert Einstein 27 July 2015 31

32. Validation Testing HISEAS: Hawaii Space Exploration Analog and Simulation Hawaii actually looks a lot like Mars… 27 July 2015 32

33. HI-SEAS Missions 1 & 2: 4 months Mission 3: 8 months (Oct 2014-June 2015) Control group for ANSIBLE Mission 4: 12 months ANSIBLE group 30min 3x/week 27 July 2015 33

34. Surveys and Measures (Crew & Family) Pre and Post: - STAI (20 Qs) - Sensory Seeking Survey (40 Qs) - Outgoing debrief (30min) In hab (both crew & designated family/friend): - Modified Circles of Closeness (3x/week 4 Qs ~1min) - Connectedness & Sensory (3x/week 11 Qs ~2min) - Journal + 6 questions (3x/week ~25min) - Perceived Stress Questionnaire (weekly 30 Qs ~8min) - Modified Social Support (SF) (weekly 18 Qs ~3min) - Usage (ANSIBLE group only) - Earth-Space communications in open forum - PANAS (daily, crew only) - Sleep (3x week) - HRV - Cortisol 34

35. Experiences or Things you miss most 27 July 2015 35

36. PWu@Siftech.com 27 July 2015 36