Dialogue systems

1. Dialogue systems Radhika Mamidi IASNLP 2014

2. Outline • Natural Language Dialogue • Computational Pragmatics • Pragmatics • Discourse Analysis • Conversation Analysis • Spoken Dialogue Systems • Types, models, domains • Comparing human-human vs human-system dialogues • Speech Act interpretation IASNLP 2014

3. A Dialogue could be…… • Two parties or more (Let’s call them John and Mary) • John and Mary talk about politics • Goal: an exchange of ideas about something • John and Mary talk about how to solve a problem. • Goal: try to solve a problem • John and Mary talk about nothing. “A random chitchat” • (Actually very common in human dialogue) • Goal? : an exchange of sentiment?, make themselves feel better by talking? or even …… there is no goal? IASNLP 2014

4. An Irish goes to a doctor saying he has mild heartache. The doctor gives him some medicines and says, “These are very strong. Take two pills a day, skip next day and take two the following day, skip and so on...” After a month the doctor meets the patient’s wife and asks her about her husband. She says, “Oh, he is dead.” The doctor says, “The medicines I gave did not work?” “Oh, the medicines worked. It was all the skipping that killed him.” IASNLP 2014

5. NL Dialogue Communication involving: – Multiple contributions – Coherent Interaction – More than one participant Interaction modalities: – Input: Speech, typing, writing, menu, gesture – Output: Speech, text, graphical display/presentation, animated body IASNLP 2014

6. Understanding intention Correct inference • Given any utterance, its speech act can play a role in many plans. • “Do you know the time?” . . . goal of learning the time. . . . goal of hurrying up the hearer . . . asking about a potential birthday present, etc. [Sentence meaning vs Speaker meaning] IASNLP 2014

7. Why is Natural Language Processing so difficult? • Human language is: • Complex and Ambiguous • We use language creatively • We don’t mean what we say! • Language Understanding needs contextual and general knowledge apart from linguistic knowledge. • To know what we mean shared knowledge is necessary. Representing all this knowledge computationally is THE challenge. IASNLP 2014

8. Human language is complex and ambiguous • When shot at, the dove dove into the bushes. • The insurance was invalid for the invalid. • They were too close to the door to close it. • The buck does funny things when the does are present. • There was a row among the oarsmen about how to row. • Upon seeing the tear in the painting I shed a tear. IASNLP 2014

9. Language understanding: Parsing problem! • Gene Autry is better after being kicked by a horse. • The women included their husbands and their children in their potluck suppers. • Two cars were reported stolen by the Groveton police yesterday. (Steven Pinker. 1994. The language instinct. Morrow. 102.) IASNLP 2014

10. We use language creatively… Example recommendations: • A man like him is hard to find. • He's an unbelievable worker. • You would indeed be fortunate to get this person to work for you. • There is nothing you can teach a man like him. • I can assure you that no person would be better for the job. IASNLP 2014

11. What we say and what we mean • A man like him is hard to find. [For a chronically absent employee] • He's an unbelievable worker. [For a dishonest employee] • You would indeed be fortunate to get this person to work for you. [For a lazy employee] • There is nothing you can teach a man like him. [For a stupid employee] IASNLP 2014

12. Cooperative model: various types of knowledge (Greene, 1986) IASNLP 2014

13. Computational Pragmatics • A subfield of Computational Linguistics • Pragmatics + Discourse Analysis + Conversation analysis IASNLP 2014

14. Pragmatics • Study of how utterances have meanings in situations. (Leech, 1983) • Study of how more gets communicated than is said. (Yule, 1996) • How people comprehend and produce a communicative act or speech act in a concrete speech situation. • It distinguishes two intents or meanings in each utterance or communicative act of verbal communication. Informative intent = the sentence meaning Communicative intent = speaker meaning (Sperber and Wilson, 1995). IASNLP 2014

15. Pragmatic competence • the ability to comprehend and produce a communicative act • Includes one's knowledge about the social distance, social status between the speakers involved, the cultural knowledge such as politeness, and the linguistic knowledge explicit and implicit. IASNLP 2014

16. Topics in Pragmatics • deals with relations between linguistic aspects and aspects of context. • Conversational Implicature A: Coffee? B: It will keep me awake. • Presupposition “I bought this book in Italy last summer” • Speech Acts “Why don’t you call Mary?” IASNLP 2014

17. Discourse Analysis Anaphora resolution John and Mary bought new cars. They are good friends. John and Mary bought new cars. They are 2008 models. Rhetorical relations John fell. Jack pushed. John went to work. He works at IBM. John went to work. He took a taxi. Ellipsis Mary bought a new car. So did Susan. Mary bought a new dress. So did Susan. Deixis “I’d like you to leave that over there and come here now” IASNLP 2014

18. Conversation Analysis • Turn Constructional Component Turns composed of one or more smaller utterance units • Turn Allocational Component Self and other selection • Sequence Organization • Adjacency pairs: greeting-greeting, question-answer pairs • Preferred, Dispreferred 2nd parts • Pre-sequences • Preference Organisation: agreement and acceptance are promoted over their alternatives • Repair: who initiates repair (self or other) and by who resolves the problem (self or other) Sacks, H., Schegloff, E. A., & Jefferson, G. (1974) IASNLP 2014

19. Do we always have smooth conversations? Johnnie: Mom! I am going out to play. Mom: With those holes in your socks!! Johnnie: No, with the kids next door. Boy(in a romantic mood): Say something soft and sweet. Girl: Custard pudding. IASNLP 2014

20. Challenge • Computing dialogues • Developing human-computer interactive systems • Based on human-human interactions IASNLP 2014

21. Computational Dialogue • Task-oriented. • Restricted Domain. • Dialogue management component to interpret the goals of incoming utterances and plan an appropriate response: IASNLP 2014

22. Computational Pragmatics “Computational pragmatics studies, from an explicitly computational point of view, how relations between linguistic phenomena and their context of use govern speakers’ abilities to interpret and generate utterances in conversation” How to compute these relations in terms of explicit representations. . . • given a linguistic expressions, how to compute the relevant contextual properties • given a particular context, how to compute the relevant linguistic expression (Bunt & Black, 2000) IASNLP 2014

23. Application of computational pragmatics Work on computational pragmatics often takes place within research ondialogue systems. Systems that are able to interact with human users in natural language. Helps us make decisions on how to deal in a computational way with all phenomena related to language use. IASNLP 2014

24. What is a dialogue system? • An artificial agent like robot or a computer system that can interact with human beings. • Helps us understand the nature of dialogue and test theories • Helps us understanding the collaborative nature of interaction • Helps us access information and services more efficiently IASNLP 2014

25. Uses of dialogue systems • Phone-based applications: timetable info or flight-booking • Personal assistant: understand user needs and tasks • Intelligent tutoring: student engagement • Embodied conversational agents – Engagement via realistic and affective physical and facial gestures • Intelligent environments: home or car – Understanding user situation and activity IASNLP 2014

26. Architecture (Mamidi and Khan, 2005) IASNLP 2014

27. Dialogue Management Tasks • Maintaining & Updating Context • Deciding what to do next • Interface with back-end/task model • Provide expectations for interpretation IASNLP 2014

28. Using Data Corpus Collection • Human-Human • Wizard of OZ • Human-System Annotation • Coding Scheme • Coding • Automatic • Tool-assisted IASNLP 2014

29. Available intelligent dialogue systems • Interactive Voice Systems [e.g. railway enquiry system asking the user to press certain numbers for accomplishing the task] • Question-Answering Systems [e.g. START, a web-based QA system answering user's questions on movies, places, people etc.] • Natural Language Interfaces [e.g. tutoring systems, trip planning systems] • Task-oriented • Restricted Domain • Easier to implement Ultimate goal to have Dialog Systems that talk like human beings and display intelligence in understanding the complex cognitive structure of language. IASNLP 2014

30. Dialogue domains • Travel information (SUNDIAL, ATIS) • Transport (TRAINS) • Business Appointments (VERBMOBIL) • Car-Navigation • Access to on-line information (SUN Speech Acts) IASNLP 2014

31. Dialogue models Dialogues can be: • Non-machine-mediated: ordinary every-day human dialogue analysed by computational means (dialogue data is recorded, transcribed and analysed to build automated systems). • Machine-mediated: The computer offers assistance to the participants (VERBMOBIL). • Simulated: Both participants are human, but one pretends to be a computer system. • Non-simulated: Genuine interaction between human and computer (fully-fledged Dialogue Systems) IASNLP 2014

32. Types of dialogue systems • Single initiative system Guide user through a series of scripted prompts. Eg. Telephone enquiry • Mixed initiative system Cooperative and flexible responses to user queries. Eg. Restaurant database IASNLP 2014

33. Approaches to Dialogue Modelling • Pattern based • model recurrent interaction patterns or regularities in dialogues at the illocutionary level of speech acts • dialogue grammar, adjacency pair • Benefits from DA and CA studies • Plan based • Relates speech acts performed in utterances to plans and complex mental states • Uses AI planning techniques • Complex representation and reasoning

34. IASNLP 2014

35. Dialogue Task Complexity • Finite-state Script (Least complicated) • Example: Long Distance Dialing • Dialogue Phenomenon handled: • User answers questions • Frame-based • Example: Getting trained arrival and departure information • Dialogue Phenomenon handled: • User ask questions, simple clarification by system

36. Dialogue Task Complexity (cont.) • Sets of Contexts • Example: Travel Booking Agent • Dialogue Phenomenon handled: • Shift between predetermined topics • Plan-based Models • Example: Kitchen design consultant • Dialogue Phenomenon handled: • Dynamically generated topic structures, collaborative negotiation subdialogues

37. Dialogue Task Complexity (cont.) • Agent-based Task • Example: Disaster Relief Task • Dialogue Phenomenon handled: • A dynamically changing world • Different modalities involved • TRIPS focused on • “…… primarily interested in design of the last two-levels of dialogue systems ……”

38. Hypothesis of Dialogue Systems • The Practical Dialogue Hypothesis • “The conversational competence required for practical dialogues, while still complex, is significantly simpler to achieve than general human conversational competence.” • The Domain-Independence Hypothesis • “Within the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management is independent of the task being performed.”

39. Four challenge mentioned • Parsing Language in Practical Dialogues • Integrating Dialogue and Task Performance • Intention Recognition • Mixed-Initiative Dialogue

40. Architecture - TRIPS

41. Evaluation • Objective Metrics • Task success • Resources used (time, turns, attention,..) • Subjective Evaluation • Issues • On-line vs off-line • Black Box vs. Glass Box • Class of User (Expert, Novice) • Feedback into system design IASNLP 2014

42. Sample dialogue: CSLI Dialogue System for in-car devices U: What music do you have? S: I have rock, soul … // aggregation U: What rock songs? Sys: Just a second: you have to turn left at the next intersection // interruption U: OK. S: I have songs by U2, Cake, Green Day, Jet, … // return to previous thread U: Play something by them. // pronominal reference S: OK. Playing xxx. < plays random song by Jet > U: Play American Idiot instead. S: Do you mean the song or the album? // disambiguation U: The album please. // fragment U: I need somewhere for lunch. S: There is a highly rated vegetarian restaurant near here. // knows preferences U: What other cuisines? S: There are Italian, Japanese, Malaysian, Nepali, and Thai restaurants close by. IASNLP 2014

43. Sample… • A: And what time would you like to leave Pittsburgh? (relevant next contribution) • C: Uh hmm, I don’t think there’s many options for non-stop • A: Right. There’s three non-stops today. IASNLP 2014

44. Comparing human-human vs computer-human dialogues • Human-Human • Hums, grunts, pauses, false starts, hesitations • Barge in conversations • Elliptical constructions • Context is important • Computer-Human dialogues • Rigid; turn by turn pattern • User speaks a word or two IASNLP 2014

45. Intelligent agent components • perception - the agent must be able to perceive the world around it • beliefs - the agent must have a representation of the present state of the world • desire/wants - the agent should have positive or negative responses to various states of the world, creating a way to compare the desirability of states • planning/reasoning - the agent must be able to reason about ways to attain other states • commitment - the agent must be able to decide to act to get to a different state • intentions - the agent must be able to maintain the course of action decided on • acting - the agent must be able to act and thus change its state (Allen,1995) IASNLP 2014

46. Illocutionary speech acts Searle (1975): • Assertives • Directives • Commissives • Expressives • Declarations IASNLP 2014

47. Challenges • Speech recognition errors • Parsing language in practical dialogue • Need to capture what was said • Spoken language is not sentence based • A single utterance realises a sequences of speech act. • Intention recognition • Mixed initiative • Integrate dialogue and task performance • Context-dependent interpretation • Dialogue strategies (turn-taking mechanisms) IASNLP 2014

48. If computers were to speak like us… • Recognise intention of speaker • A1: Lend me your umbrella. It is cloudy. [Request] • A2: Don't water the plants now. It is cloudy. [Warning] • A3: It will rain today. It is cloudy. [Assertion] • A4: I hope the pictures will come out well. It is cloudy. [Doubt] • Make proper inference • B1: Did you look at the sentence I sent you to translate. • C1: Yeah. It was such an easy sentence! • B2: Was it easy? • C2: No, I meant it was tough. IASNLP 2014

49. And… • Ellipsis • Retaining the logical form of previous sentence. • Reconstructing full content. • Turn management: determining when the turn is over and who talks next • Grounding - acknowledgement, repetition • Clarification: question to resolve some lack of understanding • Anaphora resolution IASNLP 2014

50. Speech act interpretation • BDI model • Cue based model IASNLP 2014