1 / 40

What If I Speak Now ? A Decision-Theoretic Approach to Personality-Based Turn- Taking

This research explores the relationship between personality, speech behavior, and turn-taking in conversations. It addresses the challenges of consistent personalities, robust handling of uncertainty, and the need for transparent and adaptable behavior models. Psychological background and related work in virtual agents decision theory are also discussed.

harville
Download Presentation

What If I Speak Now ? A Decision-Theoretic Approach to Personality-Based Turn- Taking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WhatIf I SpeakNow?A Decision-Theoretic ApproachtoPersonality-Based Turn-Taking Kathrin Janowski, Elisabeth André Human Centered Multimedia Institute of Computer Science Augsburg University Universitätsstr. 6a 86159 Augsburg, Germany

  2. Motivation

  3. Motivation Speech behavioriscloselycoupledwithpersonalitiesandrelationships. Rogers & Jones, 1975: “Effects Of Dominance Tendencies On Floor Holding And Interruption Behavior In Dyadic Interaction"

  4. Motivation Speech behavioriscloselycoupledwithpersonalitiesandrelationships. Goldberg, 1990: "Interrupting the discourse on interruptions: An analysis in terms of relationally neutral, power- and rapport-oriented acts"

  5. Motivation But onlyforhumans.

  6. Challenges Theyneedconsistentpersonalitiesand interpersonal attitudes.

  7. Challenges Theyneed robust handlingofuncertainty.

  8. Challenges Theyneed transparent, re-usableandadaptablebehavior models.

  9. Psychological Background Speakers needtotakevariousgoalsintoaccount. Goldberg, 1990: “Interrupting the discourse on interruptions: An analysis in terms of relationally neutral, power- and rapport-oriented acts” Horowitz et al., 2006: „How Interpersonal Motives Clarify theMeaningof Interpersonal Behavior: A RevisedCircumplexModel“

  10. Psychological Background DeYoung et al., 2013: “Unifying the Aspects of the Big Five, the Interpersonal Circumplex, and Trait Affiliation” Horowitz et al., 2006: "How Interpersonal Motives Clarify theMeaningof Interpersonal Behavior: A RevisedCircumplexModel“

  11. Psychological Background DeYoung et al., 2013: “Unifying the Aspects of the Big Five, the Interpersonal Circumplex, and Trait Affiliation” Horowitz et al., 2006: "How Interpersonal Motives Clarify theMeaningof Interpersonal Behavior: A RevisedCircumplexModel“

  12. Psychological Background DeYoung et al., 2013: “Unifying the Aspects of the Big Five, the Interpersonal Circumplex, and Trait Affiliation” Horowitz et al., 2006: "How Interpersonal Motives Clarify theMeaningof Interpersonal Behavior: A RevisedCircumplexModel“

  13. Psychological Background McCrae & Costa, 1989: “The structure of interpersonal traits: Wiggins’s circumplex and the five-factor model” DeYoung et al., 2013: “Unifying the Aspects of the Big Five, the Interpersonal Circumplex, and Trait Affiliation”

  14. Related Work - Virtual Agents Decision Theory What is the best timing? Bohus & Horvitz, 2011: "Decisions About Turns in Multiparty Conversation: From Perception to Action" Conati, 2013: "Virtual Butler: What Can We Learn from Adaptive User Interfaces?"

  15. Related Work - Virtual Agents Decision Theory What is the best timing? Incremental Processing When can we start responding? Bohus & Horvitz, 2011: "Decisions About Turns in Multiparty Conversation: From Perception to Action" DeVault et al., 2011: "Incremental interpretation and prediction of utterance meaning for interactive dialogue" Visser et al., 2012: "Toward a model for incremental grounding in spoken dialogue systems" Conati, 2013: "Virtual Butler: What Can We Learn from Adaptive User Interfaces?" Chao, 2015: "Timing multimodal turn-taking in human-robot cooperative activity"

  16. Related Work - Virtual Agents Decision Theory What is the best timing? Incremental Processing When can we start responding? Bohus & Horvitz, 2011: "Decisions About Turns in Multiparty Conversation: From Perception to Action" DeVault et al., 2011: "Incremental interpretation and prediction of utterance meaning for interactive dialogue" Visser et al., 2012: "Toward a model for incremental grounding in spoken dialogue systems" Conati, 2013: "Virtual Butler: What Can We Learn from Adaptive User Interfaces?" Chao, 2015: "Timing multimodal turn-taking in human-robot cooperative activity" Barge-In Handling When should we back down? Crook et al., 2010: "Handling user interruptions in an embodied conversational agent" Selfridge et al., 2013: "Continuously predicting and processing barge-in during a live spoken dialogue task" Smith et al., 2010: "Interaction Strategies for an Affective Conversational Agent"

  17. Related Work - Virtual Agents Decision Theory What is the best timing? Incremental Processing When can we start responding? Bohus & Horvitz, 2011: "Decisions About Turns in Multiparty Conversation: From Perception to Action" DeVault et al., 2011: "Incremental interpretation and prediction of utterance meaning for interactive dialogue" Visser et al., 2012: "Toward a model for incremental grounding in spoken dialogue systems" Conati, 2013: "Virtual Butler: What Can We Learn from Adaptive User Interfaces?" Chao, 2015: "Timing multimodal turn-taking in human-robot cooperative activity" Personality and Stance How is it perceived? Barge-In Handling When should we back down? ter Maat et al., 2011: "How Agents’ Turn-Taking Strategies Influence Impressions and Response Behaviors" Crook et al., 2010: "Handling user interruptions in an embodied conversational agent" Ravenet et al., 2015: "Conversational Behavior Reflecting Interpersonal Attitudes in Small Group Interactions" Selfridge et al., 2013: "Continuously predicting and processing barge-in during a live spoken dialogue task" Cafaro et al., 2016: "The Effects of Interrupting Behavior on Interpersonal Attitude and Engagement in Dyadic Interactions" Smith et al., 2010: "Interaction Strategies for an Affective Conversational Agent"

  18. Our Approach Wemodelpersonality-basedbehavior in an InfluenceDiagram.

  19. First Prototype other Speech State own Role other Role other Speech State Duration own Extraversion own Status Exert Control own Agreeableness own Affiliation own Speech Behavior

  20. First Prototype other Speech State own Role other Role other Speech State Duration own Extraversion own Status Exert Control own Agreeableness own Affiliation own Speech Behavior

  21. First Prototype other Speech State own Role other Role other Speech State Duration own Extraversion own Status Exert Control own Agreeableness own Affiliation own Speech Behavior

  22. First Prototype other Speech State own Role other Role other Speech State Duration own Extraversion own Status Exert Control own Agreeableness own Affiliation own Speech Behavior

  23. First Prototype

  24. First Prototype

  25. First Prototype

  26. Evaluation Stimuli: 2x2 Extraversion configurations

  27. Evaluation Online Survey: 116 participants ♂ ♀ 20-29 years

  28. Evaluation Extraversion: Agents are perceivedasmoreextraverted when configured as extraverted. CONFIRMED

  29. Evaluation Status: Agents are perceived asmore dominant when configured as extraverted.

  30. Evaluation Status: Agents are perceived asmore dominant when configured as extraverted. CONFIRMED

  31. Evaluation Agreeableness: Agents are perceived aslessagreeable when configured as extraverted.

  32. Evaluation Agreeableness: Agents are perceived aslessagreeable when configured as extraverted. CONFIRMED

  33. Conclusion An influence diagram can generate turn-taking behavior in line with the configured personality.

  34. Future Work

  35. Future Work Gaze

  36. Future Work User Interaction Gaze

  37. Future Work User Interaction Training Data Gaze

  38. Future Work More Goals User Interaction Training Data Gaze

  39. Future Work More Goals Semantics User Interaction Training Data Gaze

  40. Questions?

More Related