1 / 24

Dialogue and grammar

Dialogue and grammar. Karin Cavallin, Robin Cooper David Hjelm, Staffan Larsson. Göteborg University Dialogue Systems Lab Language Technology in Sweden 2004-05-12. Speech recognition and NL interpretation & generation. Options for SR: Statistical language model Grammar model

vonda
Download Presentation

Dialogue and grammar

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Goteborg University Dialogue Systems Lab Dialogue and grammar Karin Cavallin, Robin CooperDavid Hjelm, Staffan Larsson Göteborg University Dialogue Systems Lab Language Technology in Sweden 2004-05-12

  2. Goteborg University Dialogue Systems Lab Speech recognition and NL interpretation & generation • Options for SR: • Statistical language model • Grammar model • Options for NL interpretation: • Word/phrase spotting • Grammar & parsing • Options for NL generation • Template-based • Grammar-based • Can be combined in various ways

  3. Goteborg University Dialogue Systems Lab Are grammars useful for dialogue systems? • Most working systems don’t use a grammar! • Instead: • keyword or phrase-spotting techniques for interpretation • template-based generation • Also, most working systems are built more or less from scratch • do not fully separate application-specific knowledge from general and reusable knowledge

  4. Goteborg University Dialogue Systems Lab GoDiS • A dialogue system built using TrindiKit and based on the information state approach • Implements a theory of Issue-based Dialogue Management • Some goals with GoDiS: • Provide a domain-independent theory covering several dialogue genres • Modularity and reusability of system components • Enable rapid prototyping of new applications • Focus on dialogue management • These goals have been achieved on the level of dialogue management... • ... but not for NL processing

  5. Goteborg University Dialogue Systems Lab SR and interpretation in GoDiS • Started out as text-based system • Phrase spotting for interpretation; strings to dialogue moves • Templates for generation; dialogue moves to strings • Added SR component • Did not have a corpus, so needed to write a speech recognition grammar by hand

  6. Goteborg University Dialogue Systems Lab tvGoDiS • GoDiS application for programming a VCR over the telephone • schedule recordings • remove planned recordings • TV card used for recording television programs as video files • Small corpus collected of users calling the system • approx. 100 sessions (calls) and 1100 utterances • misrecognised utterances not yet transcribed • GUI enables immediate access to corpus

  7. TrindiGui displaying a tvGoDiS session Audio files linked with utterances Access to information state at different stages of the dialogue Goteborg University Dialogue Systems Lab Example from tvGoDIS corpus

  8. Goteborg University Dialogue Systems Lab Problems with GoDiS • Mismatch between SR module and interpretation module • correctly recognised speech was misinterpreted • Leads to strange and unexpected behaviour (unexpected interactions with dialogue management)

  9. S> Välkommen till videon! För att få hjälp, säg hjälp. S> Låt oss se. Vad kan jag hjälpa dig med? U> jag vill spela in ett program (0.6) S> Vill du lägga till en inspelning? U> ja (0.98) S> Låt oss se. Menar du från noll ett eller till noll ett? jag vill spela in ett program Covered by SR grammar Not covered by phrase spotting lexicon However spela in and ett are recognized by interpretation module Correct interpretation: jag vill spela in ett programrequest(add_rec_job) Actual interpretation: spela in request(add_rec_job ett  answer(time(1:00)) System assumes ett specifies either when the recording should stop or end – thus the follow-up question Goteborg University Dialogue Systems Lab ASR/interpretation mismatch

  10. Goteborg University Dialogue Systems Lab Problems with GoDiS • Mismatch between interpretation module and generation module • System said something that led the user to believe they could say X • but the system does not understand X

  11. S> Du kan lägga till en inspelning, ta bort en planerad inspelning, och lista planerade inspelningar. För att börja om, säg börja om. För att sluta, säg sluta. S> huvudmenyn. Vad kan jag hjälpa dig med? U> lista inspelningar (0.75) S> Ockey. gå till inspelningar. S> Låt oss se. Vill du lägga till en inspelning , radera dina inspelningar eller få information om dina inspelningar? Help message suggests that the user can say lista inspelningar But help message is not up-to-date, and neither is SR grammar. Instead interpretation module finds word inspelningar The user is asked to choose between roughly the same alternatives that was presented in the help message Goteborg University Dialogue Systems Lab Interpretation/generation mismatch

  12. Goteborg University Dialogue Systems Lab • Bugs can of course be fixed... • would take lots of time • no guarantee that new bugs won’t be introduced • no guarantee that all bugs have shown up • These solutions are not reusable • Instead, we want a general and principled solution • enable reuse and rapid prototyping of applications, including grammars

  13. Goteborg University Dialogue Systems Lab Solution, part 1: single grammar for SR, interpretation, generation • No mismatches between the grammars • Only one grammar needs to be written for each application • However: • the different modules may require different kinds of grammars, in different formats • So: • We need to be able to generate all 3 grammars from a single grammar • However, we may not want the exact same coverage for SR, interpretation and generation grammars • May not want to understand everything that can be generated

  14. Goteborg University Dialogue Systems Lab Solution, part 2: use GF! • GF is a powerful tool for mutilingual grammar development • Can be used to generate grammars in various other formats • Subgrammars can be extracted from larger grammars • SR, interpretation and generation subgrammars may have different coverage • In addition, GF offers resource grammars in several languages • This promises to further decrease the time needed for writing a new application grammar

  15. Goteborg University Dialogue Systems Lab GF GoDiS Grammar • Application grammars • User grammar • System grammar • Shared grammar, used by user grammar and system grammar • Utility grammars • Weekday, Time, Channel • Application grammars may use • Utility grammars • Resource grammars

  16. Goteborg University Dialogue Systems Lab GF VCR grammar • One abstract syntax • GoDiS dialogue moves as GF abstract trees • Three concrete syntaxes • Swedish • English • GoDiS dialogue moves in prolog notation • Statistics: • Abstract syntax : ~420 lines of code • Concrete syntax : • Prolog: ~440 lines of code • Swedish: ~440 lines of code

  17. Goteborg University Dialogue Systems Lab Parsing Swedish to abstract syntax > p -cat=DMove -lang=Swe "spela in från kvart över tolv" request (vcr_add_rec_job_args1 (startTimeToStoreReq (startTimeToStore (time hour12 minute15))))

  18. Goteborg University Dialogue Systems Lab Linearising abstract syntax to Prolog notation > l -lang=Prolog request (vcr_add_rec_job_args1 (startTimeToStoreReq (startTimeToStore (time hour12 minute15)))) request(add_rec_job , answer(start_time_to_store(12 : 15))

  19. Goteborg University Dialogue Systems Lab Translating Prolog to Swedish > t -cat=DMove Prolog Swe "request(add_rec_job , answer(start_time_to_store(12 : 15))" spela in från tolv femton spela in från tolv och femton spela in från femton över tolv spela in från femton minuter över tolv spela in från kvart över tolv

  20. Goteborg University Dialogue Systems Lab Time Grammar • A multilingual grammar module • Swedish, English, Prolog, started on French and Catalan.

  21. Goteborg University Dialogue Systems Lab Constructing concrete syntaxes for time expressions isn’t always trivial • Abstract: time hour12 Minute30 • Prolog: 12 : 30 • English: half past twelve / half twelve • Swedish: halv ett • klockan två • klockan två och femton • ?klockan fjorton • two o’clock • *two fifteen o’clock • *fourteen o’clock Corpus studies needed

  22. Goteborg University Dialogue Systems Lab Another problem • am or pm? • på morgonen • på eftermiddagen • på kvällen • på natten

  23. Goteborg University Dialogue Systems Lab Combining GF and statistical methods for robustness • Use GF to generate a corpus to use as a baseline for SLM training • Use GF-generated SR grammars in initial system and collect corpus to be used for training SLMs for speech recognition • For recognized out-of-grammar string, try to find nearest in-grammar string; confirm this with user (Gorrell 2003)

  24. Goteborg University Dialogue Systems Lab Conclusion • Keeping track of separate SR, interpretation and generation grammars in a dialogue system is time-consuming • GF offers the possibility of rapid prototyping of grammars for use in dialogue system applications • Generate other grammar formats from GF • Resource grammars for several languages • GF can be combined with statistical methods for robustness

More Related