1 / 50

Advancing Conversational AI: Semantics, Consistency, Interactivity

Explore the nuances of building more intelligent conversational systems, focusing on semantics, consistency, and interactivity to enhance user engagement and experience. Learn about the components, challenges, and advancements in conversational AI technology for a more personalized interaction. Discover cutting-edge models and techniques for language generation and understanding in open-domain conversational systems.

yolandak
Download Presentation

Advancing Conversational AI: Semantics, Consistency, Interactivity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TowardsBuildingMoreIntelligentConversationalSystems:Semantics,Consistency,&InteractivenssTowardsBuildingMoreIntelligentConversationalSystems:Semantics,Consistency,&Interactivenss Dr.MinlieHuang AssociateProfessor TsinghuaUniversity http://coai.cs.tsinghua.edu.cn/hml

  2. Types of Conversational AI Systems Limitedsocial backandforth Ideal Bot Virtual Assistant Accomplish Tasks Limitedcontent totalkabout executecommands, answerquestions Chat Bot chitchat Social Conversation Fromprof.MOstendorf,withedits

  3. Issues Vary for Different Paradigms ASSISTANT SOCIALBOT Conversational AI System Components *Social & info *Intents *Grounding *Task intents *Slotfilling Speech/language understanding *Narrow options & execute tasks *Reward= timely task completion *Learn about interests *Make suggestions *R=userengagement Dialog management Constrained domains Language generation Open domains Back-end application Unstructured Data&Info Structured KB+DB Fromprof.MOstendorf,withedits

  4. GoodExamplesfromDLModels • User：最幸福的人不是拿到了最好的东西，而是学会去珍惜手上已经拥有的人。 • Thehappiestpersonisnottheonewhogetsthebestthing,insteadwhocherisheswhatshe/healreadyhas. • Machine:是的，所以我也要珍惜自己的幸福 • Agree,Iwillcherishmyhappinesstoo. • User:时间啊，你这把杀猪刀。。。Timeisakiller… • Machine:我是被杀的 Iamtheonewhowaskilled. • User: 怕孤獨，又捨不下自己的小自由。 • Afraidtobelonely,butwont’giveupmylittlefreedom. • Machine: 我想去流浪，但没有人陪我去。 • Iwanttowander,butnooneaccompanyme.

  5. BadExamplesareMore! User: 其实，他没有那么爱你。 Ohh, he does not love you that much. Machine:幸福的一天开始了。 A happy day starts. User: 回北京了！很冷很高兴！ Back to Beijing now. So cold so happy! Machine:什么时候来北京啊？ When will you come to Beijing? User: 我来自上海。 I come from Shanghai. Machine：真巧，我来自海南，老乡啊 So cool, I am from Hainan, the same province!

  6. ControllabilityofLanguageGeneration • Threefundamentalproblemsincurrentneurallanguagegenerationmodels • Semantics • Consistency • Logic

  7. BadExamplesareMore! XiaoicebyMSRA 语义理解的问题 Semanticunderstanding 微 软 小 冰 User: 你家多大？ Howlargeisyourhouse? 小冰: 这么快就要爆年龄了？Sosoontorevealmyage? User: 你是素食主义者还是吃肉啊？ Areyouveganormeatatarian? 小冰: 怎么减肥 HowcanIloseweight? 个性身份一致性问题 Inconsistencyinpersonality User: 你几岁了？ 小冰: 我啊。。。反正是九零后！ User: 那你是几几年出生的？ 小冰: 错，75年阳历一月出生。 User: 你是九零后，怎么会75年出生？ 小冰: 生在九零后，在深圳只能被当做八零后了。 上下文理解的问题 Contextissue User: 早饭不好吃 小冰: 那明早一起吃 User: 行，你推荐的应该不错 小冰: 是吗？ User: 哈哈，我相信你 小冰: 呃呃

  8. ChallengesinConversationalSystems • One-to-many:oneinput,manymanypossibleresponses • Knowledge&Reasoning:realunderstandingrequiresvariousknowledge,worldfacts,commonsense,etc. • SituationalContext • Whoareyoutalkingwith? • Stranger,orfriend? • Hismoodandemotion? • Sharedbackgroundsthatareonlyaccessiblebytwoacquaintances

  9. ChallengesinConversationalSystems Consistency Interactiveness Semantics Emotion& Sentiment Content, Context, Scene Strategy, Behavior Personality, Personalization, LanguageStyle Open-domain+Open-topic

  10. Open-domainConversationalSystems • Behavingmoreinteractively: • Perceiving and Expressing Emotions(AAAI2018) • ProactiveBehavior by AskingGoodQuestions(ACL2018) • ControllingSentenceFunction(ACL2018) • TopicChange(SIGIR2018) • Behaving more consistently: • ExplicitPersonalityAssignment(IJCAI-ECAI2018) • Behavingmoreintelligentlywithsemantics: • Better Understanding and Generation Using CommonsenseKnowledge(IJCAI-ECAI2018distinguishedpaper) • Discourseparsinginmulti-partydialogues(AAAI2019)

  11. Open-domainConversationalSystems • Behavingmoreinteractively: • Perceiving and Expressing Emotions(AAAI2018) • ProactiveBehavior by AskingGoodQuestions(ACL2018) • ControllingSentenceFunction(ACL2018) • TopicChange(SIGIR2018) • Behaving more consistently: • ExplicitPersonalityAssignment(IJCAI-ECAI2018) • Behavingmoreintelligentlywithsemantics: • Better Understanding and Generation Using CommonsenseKnowledge(IJCAI-ECAI2018distinguishedpaper) • Discourseparsinginmulti-partydialogues(AAAI2019) Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory.AAAI2018. Assigning personality/identity to a chatting machine for coherent conversation generation.IJCAI-ECAI2018. Commonsense Knowledge Aware Conversation Generation with Graph Attention.IJCAI-ECAI2018. Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders.ACL2018. Generating Informative Responses with Controlled Sentence Function. ACL 2018. Chatmore:deepeningandwideningthechattingtopicviaadeepmodel.SIGIR2018. A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues.AAAI2019.

  12. Interactiveness:EmotionPerceptionandExpression

  13. EmotionalChattingMachine PerceivingandExpressingemotionbymachine Closertohuman-levelintelligence SocialInteractionData Emotion Tagged data Post Response EmotionalChatting Machine Emotion Classifier Post Response Post Response … … Post Response Sad Happy Angry OurworkwasreportedbyMITTechnologyReview,theGuardian,CankaoNews,XinhuaNewsAgencyetc. Prof Björn Schuller: “an important step” towards personal assistants that could read the emotional undercurrent of a conversation and respond with something akin to empathy. Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory.AAAI2018.

  14. MoreExamples Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory.AAAI2018.

  15. EmotionInteractionPatterns LikeLike(empathy) Sadness Sadness(empathy) Sadness Like(comfort) Disgust  Disgust(empathy) Disgust  Like(comfort) Anger  Disgust HappinessLike Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory.AAAI2018.

  16. Interactiveness:BehavingMoreProactivelybyAskingGoodQuestions

  17. AskingQuestionsinConversationalSystems 我昨天晚上去聚餐了 Iwenttodinneryesterdaynight. Learningtoaskquestionsinopen-domainconversationsystems.ACL2018.

  18. AskingQuestionsinConversationalSystems • Askinggoodquestionsrequiressceneunderstanding Scene:Diningatarestaurant 我昨天晚上去聚餐了 Iwenttodinneryesterdaynight. Friends? Food? … Persons? Bill? Place? WHERE HOW-ABOUT HOW-MANY WHO WHO Learningtoaskquestionsinopen-domainconversationsystems.ACL2018.

  19. AskingQuestionsinConversationalSystems • Responding+asking(Li et al., 2016) • Keyproactivebehaviors(Yuetal.,2016) • Askinggoodquestionsareindicationofmachineunderstanding • Keydifferencestotraditionalquestiongeneration(eg.,readingcomprehension): • Differentgoals:Informationseekingvs.Enhancinginteractiveness and persistence of human-machine interactions • Variouspatterns:YES-NO,WH-,HOW-ABOUT,etc. • Topictransition:fromtopicsinposttotopicsinresponse

  20. AskingQuestionsinConversationalSystems • Agoodquestionisanaturalcompositionof • Interrogativesforusingvariousquestioningpatterns • Topicwordsforaddressinginterestingyetnoveltopics • Ordinarywordsforplayinggrammarorsyntacticroles

  21. AskingQuestionsinConversationalSystems • Typeddecoders:softtypeddecoder

  22. AskingQuestionsinConversationalSystems • Typeddecoders:hardtypeddecoder • Foreachpost: • Asetofinterrogatives • Alistoftopicwords • Othersforordinarywords • Topicwords: • Training--nouns,verbs • Test–predictedbyPMI

  23. AskingQuestionsinConversationalSystems • Typepredictionateachdecodingposition 6 1 2 3 4 5 Decodingsteps

  24. Interactiveness:AchievingDifferentSpeakingPurposesbyControllingSentenceFunctionInteractiveness:AchievingDifferentSpeakingPurposesbyControllingSentenceFunction

  25. Controlling Sentence Function • Sentence function indicates different conversational purposes. Interrogative Acquire further information from users (e.g. WHY, WHAT, …) Imperative Make requests, instructions or invitations (e.g. LET’S, PLEASE, …) User: I’m really hungry now. Make statements to state or explain (e.g. AND, BUT, …) Declarative Generating Informative Responses with Controlled Sentence Function. ACL 2018.

  26. Controlling Sentence Function • Response with controlled sentence function requires a global plan of function-related, topic and ordinary words. What did you have at breakfast? (Acquire further information from users) Interrogative Let’s havedinner together! (Make requests, instructions or invitations) Imperative User: I’m really hungry now. Declarative Me, too. But you ate too much at lunch. (Make statements to state or explain) Function-related words Topic words Ordinary words

  27. Controlling Sentence Function • Keydifferencestoother controllable text generation tasks: • Global Control:adjust the global structure of the entire text, including changing word order and word patterns • Compatibility:controllable sentence function + informative content • Solutions: • Continuous Latent Variable:project different sentence functions into different regions in a latent space + capture word patterns within a sentence function • Type Controller:arrange different types of words at proper decoding positions by estimating a distribution over three word types

  28. Semantics:BetterUnderstandingandGenerationwithCommonsenseKnowledgeSemantics:BetterUnderstandingandGenerationwithCommonsenseKnowledge

  29. CommonsenseKnowledge • Commonsense knowledge consists of facts about the everyday world,thatallhumansareexpectedtoknow.(Wikipedia) • Lemonsaresour • Treehasleafs • Doghasfourlegs • Commonsense Reasoning ~ Winograd Schema Challenge: • The trophy would not fit in the brown suitcase because it was too big. What was too big? • The trophy would not fit in the brown suitcase because it was too small. What was too small?

  30. CommonsenseKnowledgeinChatbots respiratory disease lung disease IsA IsA asthma air pollution Prevented_by Caused_by avoiding triggers Caused_by chest tightness

  31. CommonsenseKnowledgeinChatbots lung disease respiratory disease IsA IsA asthma air pollution Caused_by Prevented_by Caused_by avoiding triggers chest tightness

  32. CommonsenseKnowledgeinChatbots Post:Ihaveanasthmasincethreeyearsold. Triplesinknowledgegraph: (lungdisease,IsA,asthma) (asthma,Prevented_by,avoidingtriggers) respiratory disease lung disease IsA IsA asthma air pollution Prevented_by Caused_by avoiding triggers Caused_by FromConceptNet chest tightness

  33. CommonsenseKnowledgeinChatbots Post:Ihaveanasthmasincethreeyearsold. Triplesinknowledgegraph: (lungdisease,IsA,asthma) (asthma,Prevented_by,avoidingtriggers) Response:I am sorry to hear that. Maybe avoiding triggerscan prevent asthma attacks. respiratory disease lung disease IsA IsA asthma air pollution Prevented_by Caused_by avoiding triggers FromConceptNet Caused_by chest tightness

  34. CommonsenseKnowledgeinChatbots Post:Ihaveanasthmasincethreeyearsold. Triplesinknowledgegraph: (lungdisease,IsA,asthma) (asthma,Prevented_by,avoidingtriggers) Response:I am sorry to hear that. Maybe avoiding triggerscan prevent asthma attacks. respiratory disease lung disease IsA IsA asthma air pollution Prevented_by Caused_by avoiding triggers FromConceptNet Caused_by chest tightness

  35. CommonsenseKnowledgeinChatbots Output:BecauseI’mabrittleman. Decodingwordsbyattendingtoknowledgegraphsandthentotriples Encodingtheretrievedknowledgegraphsforeachword Input:whyareyousobreakable?

  36. CommonsenseKnowledgeinChatbots Staticgraphattention:encodingsemanticsingraph, Feedingknowledge-enhancedinfo.intotheencoder

  37. CommonsenseKnowledgeinChatbots Dynamicgraphattention:firstattendagraph,thentoatriple withinthatgraph,finallygeneratewiththewordsinagraph

  38. CommonsenseKnowledgeinChatbots Automaticevaluation Manualevaluation (Sign-test,p-value<0.005) Commonsense Knowledge Aware Conversation Generation with Graph Attention. IJCAI-ECAI 2018, Stockholm, Sweden.

  39. GenerationExamples1 Post: He proposed March 5th. We will be married October 10th. So 7 months Knowledge: (engaged, RelatedTo, married), (wedding, RelatedTo, married), (miss, DistinctFrom, married), (March, RelatedTo, months) • Seq2Seq: I'm not sure how to answer that. I'm not sure how to make a date. • MemNet: I'm assuming he's married to a gay marriage? • CopyNet: I'm sorry, I'm not sure what you mean by that. • CCM: I'm pretty sure he's going to be engaged in the next 6 months.

  40. Logic:StoryEndingGeneration

  41. Logic:Story Ending Generation Find context clues: plan the order of events and entities. Story Ending Generation with Incremental Encoding and Commonsense KnowledgeAAAI2019 ReportedbyTechXplore

  42. Logic:Story Ending Generation Commonsense knowledge

  43. Logic:Story Ending Generation Incremental Encoding Multi-Source Attention

  44. Logic:Story Ending Generation Attention to the knowledge base: static graph attention Graph attention

  45. Experiment • ROCStories,90,000fortraining,8912fortest

  46. Logic:Story Ending Generation Story 1: Context: Taj has never drank an espresso drink. He orderedone while out with his friends. The shot of espressotasted terrible to him. Taj found that he couldn't stop talking or moving. Generated Ending: He decided to never drink again. Story 2: Context: Martha is cooking a special meal for her family. She wants everything to be just right for when they eat. Martha perfects everything and puts her dinner into the oven. Martha goes to lay down for a quick nap. Generated Ending: When she gets back to the kitchen, she sees a burning light on the stove.

  47. Logic:Story Ending Generation Buildingcontextcluesincrementally

  48. Summary • Semantics,consistency,interactiveness • Emotion,behaviors,personality,andknowledge • Stillalongwaytogo:existingconversationalsystemsarestillfarfromhuman-like

  49. FutureResearchProblems • Multi-modalityemotionperceptionandexpression(voice,vision,text) • Personality,identity,style “human-likerobot” • Introvertorextrovert • Personalized(style,orprofile) • Learningtolearn(lifelonglearning) • Growupfrominteractionswithhumanpartnersandenvironment

  50. ThanksforYourAttention • http://coai.cs.tsinghua.edu.cn/ds/ 对话系统技术平台 • Contact: • MinlieHuang,TsinghuaUniversity • aihuang@tsinghua.edu.cn • http://coai.cs.tsinghua.edu.cn/hml

More Related