1 / 20

Machine Learning of Discourse

Machine Learning of Discourse. Piroska Lendvai ILK Research Group Tilburg University. Overview: Spoken Dialogue Systems: goal, architecture, need for error handling Machine Learning for SDS: what, where, how

natala
Download Presentation

Machine Learning of Discourse

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Machine Learning of Discourse Piroska Lendvai ILK Research Group Tilburg University

  2. Overview: • Spoken Dialogue Systems: goal, architecture, need for error handling • Machine Learning for SDS: what, where, how • Machine Learning for discourse in SDS: an empirical study on shallow semantic-pragmatic data

  3. I. Spoken Dialogue Systems • From 60s: communication with a machine in natural language. Recent advances in speech technology facilitate development of SDS • Create systems that enable interaction with an application (eg. software, TV, database) using natural language through a voice interface • SDS: programs that communicate with a user in order to perform some task (e.g. database query) -> "information systems“, “conversational agents”

  4. Practical use • Save money with automated telephone interfaces that provide info (calling, transport, weather, booking, banking) • Create automated help systems/manuals • Implement in smart household appliances / industrial robots • (Provide support for customers in) using e- commerce • E-mail, voice mail access

  5. Man-machine communication demands combining techniques of speech analysis and generation + linguistic analysis/generation (syntax, semantics) + task planning • Desirable to meet discourse requirements: satisfy social obligations, handle world knowledge and dynamically changing context • So: apart from understanding the literal meaning of the user’s utterance, must interpret subtle and implicit dialogue acts (does user request / provide / acknowledge info, correct misunderstanding, etc.)

  6. Discourse operates on all linguistic levels from phonetics to pragmatics -> implementing discourse info in SDS implies formalization of all these levels (VERY hard) • Current SDS not very efficient, operate in very restricted domains • Examples of dialogue systems: • Demo SDS: GoDIS (written input, limited slot-fillers, rigid management) • Test version SDS: OVIS (more flexible slot filling, bad language understanding)

  7. GoDis $S>Welcome to the travel agency! $U>I want to fly from Amsterdam to Budapest $S>What city do you want to go to? $U>Budapest $S>I didnt understand what you said. Please rephrase. $U>Gothenburg $S>What city do you want to go from? $U>Amsterdam $S>I didnt understand what you said. Please rephrase. …

  8. OVIS Q: goedenavond. dit is het automatische informatiesysteem van openbaar Vervoer Reisinformatie. dit systeem geeft uitsluitend informatie over treinreizen van NS-station naar NS-station. van welk station naar welk station wilt u reizen? A: ik moet volgende week dinsdag van schiphol naar nijmegen Q: van waar naar waar wilt u op dinsdag twaalf december reizen? A: van schiphol naar nijmegen Q: hoe laat wilt u vanuit schiphol naar nijmegen reizen? …

  9. SDS architecture lang understanding speech recog dialogue manager application speech synthesis lang generation

  10. Interface: speech (telephone); GUI (WAP/WWW protocol); animated agent (multimodal, 3D) • NLU unit: STT: ASRecogn (acoustic processing), Language model (grammar, lexicon, phrase segmentation), Semantic representation • Dialogue Manager: intermediate agent between user and application, responsible for the interaction. Built-in domain model, session model, user model (competence of user), task planner (carry out strategy, monitor state, update tasks / memory), problem solver, communicates with application / database • NLG: domain templates, TTS - speech synthesis

  11. Pragmatic info in SDS • Conversation is one type of discourse • Special notions: turn-taking, info grounding • Dialogue acts (ask, answer, acknowledge, correct, verify, suggest, reject,…) expressed by word order, intonation, ellipsis, context, … • Represent communicative content of utterances as • Q “On which day do you want to travel?” • E “So you want to go to Rotterdam.” • Q;I “What time do you want to travel to Rotterdam?” • S “Tomorrow.” • N;S “No, not to Rotterdam but to Roermond.”

  12. Semantic info in SDS We want to know what U says to S & vice versa • Simpler: want to know what slot Sys talks about with what value • Q_VA “From where to where do you want to travel?” • Q_DTH; I_A “When do you want to arrive in Rotterdam?” • Q_Oc “Do you want to know another connection?” • Want to know what slot User fills with what value • S_T “Somewhere in the evening” • S_DTH “On Monday morning at 10” • Y;S_VA “Yes, the return trip” • N “You don’ t need to repeat the connection”

  13. Error handling in SDS • Central issue to keep interaction running. • Typical problems in SDS are different from those in human-human communication: humans comm. multimodally but ling-ly not necessarily explicitly, in system everything needs to be explicitly stated • Deal with problems due to: • NLU technicalities: poor speech recognition as ASR trained in lab, limited vocabulary and lang. model, user idiosyncrasies (accent, intonation, reaction time, comprehension) • Poor DM engineering yields bad dialogue management:

  14. DM problems: • inefficient strategy wrt task completion (only 2 prompts to fill slot) • wrong default assumptions (user wants to travel today) • unintegrated social behaviour (user's greeting unrecognized) • unintegrated world knowledge (tweede kerstdag) If cues that signal problems (_prob) or seamless communication (_ok) are identified, we can perform error detection and recovery.

  15. II. ML for SDS • Goal:train / optimize / adapt the SDS modules by processing corpus data of H-C / H-H interactions • Issues: Which ML algorithms for which tasks? combining features means processing continuous and symbolic and set-valued data, induce rules vs store instances, ... • Data: Reduced hypothesis space, often artificial / small data (WOZ, test users)

  16. ML for NLU: • predict speech recognition performance • predict and adapt to poor speech recognition in SDS • identify understanding errors, user corrections • predict user reactions to system error (both prob and sem) • represent utterances’ semantic content • topic recognition

  17. ML for NLG: • optimize sentence planner for generating system prompt • ML for DM: • predict problematic situations during the interaction • optimize general strategy • identify dialogue acts in utterances • enhance system design by modelling interactions (both prag and sem)

  18. III. Empirical study of ML of discourse in man-machine dialogues • Noisy real-data Dutch corpus: OVIS • 3738 turns • Class: user utterance represented as tag incorporating shallow pragmatics + semantics + problem awareness [Sys: Q_DTH;I_A “When do you want to travel to Tilburg?”] • S_D_ok “Tomorrow” • N;S_A_prob “Not to Tilburg but to Schiphol!” • A;S_DTH_prob “Today at eight in the evening”

  19. Attribute picking: which characteristics of the dialogue are most predictive of the problem/non-problem class? • Shallow features, automatically extractable from SDS • symbolic (last 10 sysQ types, “history”) • numeric (user prosody: pitch, loudness, duration, tempo phenomena) • binary (759 bag-of-words output of ASR) • lexical (best string of ASR)

  20. Tasks • Check “baseline”: performance based on only most-predictive-feature (latest prompt) • Classifiy discourse (prag+sem+prob class) • Classifiy separately • Pragmatics and Semantics • Problem awareness

More Related