1 / 27

Dialogue Manager Senior Companion

Dialogue Manager Senior Companion. University of Sheffield March 2008. review. Senior Companion Dialogue Manager. 2-tier planning where High-level plans built using the Cognitive Model Topic specific plans built using Dialogue Action Forms DAFS Dialogue Action Forms

nascha
Download Presentation

Dialogue Manager Senior Companion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dialogue ManagerSenior Companion University of Sheffield March 2008

  2. review

  3. Senior Companion Dialogue Manager • 2-tier planning where • High-level plans built using the Cognitive Model • Topic specific plans built using Dialogue Action Forms • DAFS Dialogue Action Forms • Augmented Transition networks • Represent domain knowledge • Modularizable to encapsulate sub-topics within a conversation domain • Back channel response while working on more complete system utterances (hm, uh-huh)

  4. Dialogue Manager Architecture Cognitive Model NLU output of user utterance Semantic content of system utterance + emotion tags Natural Language Understanding Dialogue Manager Generation Ontology Dialogue history Photo app

  5. DM input closeup Dialogue History Cognitive Model NLP output • Emotion Tags • Planning • System Model • Record of User and system • utterances • Dialogue Acts • Named Entities • Person • Family Relations • Sister • Mother • grandmother • Location • Celebration event • Birthday • Anniversary • Wedding • Christmas • Pronoun references • Semantic categories for • User input objects (nouns) • User input actions(verbs) • Original user input string • Modalities for user input Dialogue Manager Ontology Photo App • Hierarchy of Semantic • tags for Photos • User instances • Photo/s id • Photo tags • Photo attributes

  6. Speech Analysis • Expected user • response category Fission/Generation • Number of output Utterances • for each: • Semantic Representation • objects • actions • Dialogue Act • Emotional content • Output modality DM output closeup Dialogue Manager Photo App Ontology Dialogue History Cognitive Model

  7. Dialogue Action Forms • GUI editor for creating DAFs • Composed of nodes and arcs containing tests and actions • DAFs pre-stacked, but can be overidden by matching indexing terms (semantic classes, significant words) • Essential for mixed-initiative conversation

  8. SC Dialogue Manager Stack Run Greeting DAF Pop greeting DAF Push photo DAF Run people DAF Pop people DAF System start Run photo DAF People DAF people event event date date special memory special memory greeting photo location location goodbye goodbye goodbye goodbye

  9. DAFs (1)

  10. DAFs(2)

  11. SC Output (1) • Natural Language Generation • Currently choosing from a selection of pre-defined surface forms. • A Generation module to be developed (CU) • To be shared with the Health and Fitness Companion • To include input from the Cognitive model for emotion and conversational style (based on user profile) • Will contain lists of alternative question realisations derived from corpora • TTS • Loquendo TTS • Avatar AAA, CrazyTalk, Nabaztag (2-D)

  12. more detail

  13. Current Dialogue Manager Architecture Input Queue Input Manager NLU Photo Watcher DAF Stack DAF Engine Output Queue Output Manager NLG

  14. Output of ASR In the format of Galaxy Communicator frame. For example: { c output :hypothesises ( {c hypothesis …} {c hypothesis …} … ) } {c hypothesis :ROName “Language Model Name” :Likelyhood -159.035736 :Confidence 0.758829 :String “nice one” :NumOfWords 2 :WordsInfo: ( {c word …} {c word …} ) } {c word :String “nice” :AcousticScore 3.791455 :Confidence 0.750377 :StartPoint 0 :EndPoint 42 :Language: “en-gb”}

  15. NLU • Input A textual string. For example: “Kate is the mother of Marry”. • Output • GATE Annotation Set. • A snippet output for the sentence: “Kate is the mother of Mary.” • [AnnotationImpl: id=0; type=Token; • features={category=NNP, kind=word, orth=upperInitial, length=4, string=Kate}; • start=NodeImpl: id=0; offset=0; end=NodeImpl: id=1; offset=4 • , AnnotationImpl: id=1; type=SpaceToken; • features={kind=space, length=1, string= }; • start=NodeImpl: id=1; offset=4;end=NodeImpl: id=2; offset=5 • , AnnotationImpl: id=2; type=Token; • features={category=VBZ, kind=word, orth=lowercase, length=2, string=is}; • start=NodeImpl: id=2; offset=5; end=NodeImpl: id=3; offset=7 • , AnnotationImpl: id=3; type=SpaceToken; • features={kind=space, length=1, string= }; • start=NodeImpl: id=3; offset=7; end=NodeImpl: id=4; offset=8 • ……]

  16. NLU 2. A populated ontology in RDF format A snippet output for the sentence: “Kate is the mother of Mary.” ……………………… <owl:FunctionalProperty rdf:ID="hasFather"> <rdf:type rdf:resource="http://www.w3.org/2002/07/owl#ObjectProperty"/> <rdfs:range rdf:resource="#Man"/> <rdfs:domain rdf:resource="#Person"/> <rdfs:subPropertyOf rdf:resource="#hasParent"/> </owl:FunctionalProperty> <Person rdf:ID="someone2"> <hasMother> <Person rdf:ID="someone1"> <hasName rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >Kate</hasName> <hasSex rdf:resource="#female"/> </Person> </hasMother> <hasName rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >someone2</hasName> </Person> </rdf:RDF>

  17. DAFs • DAFs are either triggered by • Being on the top of the stack • Through index term matching • Each DAF has an associated set of terms associated with it (semantic categories)

  18. Dialogue Manager Strategy • Very simple for now, discusses variety of things in and about the photos • People - their names, ages and relationship to the user • Location of the photo and when it was taken and for what purpose • DM strategy at present connects all photos that contain the same person and discusses the similarities and differences.

  19. DAFs • Created using a GUI editor • Set of nodes and arcs • A node represents a dialogue state • Each arc contains a test and an associated action

  20. DAF Editor Each node represents a dialogue state Each arc contains a test with an associated action

  21. DAF tests invoke protégé Rules The main knowledge base will be an ontology which will hold all the world knowledge. There is another ontology which holds the current session knowledge. We chose ontologies to represent the knowledge because in the same file we have a structure, instances and logical rules. The tests invoke rules predefined in the ontology. If the rule evaluates to true, then the relative action is invoked, otherwise not. Like this, a person creating the rule needs not know anything on programming. The rules will look like ... Person(?p) ^ hasAge(?p,?age) ^ greaterThan(?age,200) -> TooOld(?p) They can be written and tested in Protege using a graphical interface.

  22. SWRL http://protege.cim3.net/cgi-bin/wiki.pl?SWRLLanguageFAQ • SWRL is an acronym for Semantic Web Rule Language. It is intended to be the rule language of the Semantic • Web. SWRL is based on the OWL Web Ontology Language. It allows users to write rules to reason • about OWL individuals and to infer new knowledge about those individuals. • SWRL rule contains an antecedent part, which is referred to as the body, and a • consequent part, which is referred to as the head. • Both the body and head consist of positive conjunctions of atoms • SWRL rules are written in terms of OWL classes, properties and individuals • SWRL supports Open World Reasoning • One of the most powerful features of SWRL is its ability to support user-defined • built-ins. A built-in is a predicate that takes one or more arguments and evaluates • to true if the arguments satisfy the predicate. • ･Person(?p) ^ hasAge(?p, ?age) ^ swrlb:greaterThan(?age, 17) -> Adult(?p)

  23. Not all user input can be structured • If the NLU module cannot fit data into a structured form of the type we are expecting then the NLU module outputs • an Annotation Set which has coordinates and properties for those coordinates.

  24. Dialogue Manager uses the Knowledge Base to • Access individual pieces of Knowledge • Static system knowledge • Dynamic user instances • Make inferences about the knowledge *relies on the knowledge being structured

  25. NLG • Input The input of NLG is a logical representation which has three properties: domain, type and Value (optional). To generate a sentence of requesting some general information of a photo, following representation will be sent to NLG: Domain: PHOTO Type: REQUEST_GENERAL Value: EMPTY

  26. NLG • Output A predefined utterance from a template file. Below is a snippet of the template file. (The $ will be replaced by the value specified in the logical representation.) ……………… <Template domain="PHOTO"> • <Sentences type="REQUEST_AGE"> • <Sentence>How old is $?</Sentence> • <Sentence>What is $'s age?</Sentence> • <Sentence>What is the age of $?</Sentence> • <Sentence>$ is how old?</Sentence> • </Sentences> • <Sentences type="REQUEST_RELATION"> • <Sentence>How do you know $?</Sentence> • <Sentence>What is your relationship with $?</Sentence> • </Sentences> • ……………..

  27. additions to the DM (near future) • Add emotions to the SC system • Incorporate a cognitive model component • Machine Learning for improving the dialogue strategy

More Related