Siridus Specification, Interaction and Reconfiguration in Dialogue Understanding Systems

1. SiridusSpecification, Interaction and Reconfiguration in Dialogue Understanding Systems an information state approach to flexible spoken dialogue systems

2. Overview • Siridus project • Information State Approach • Main research areas • Demonstrators

3. Siridus Project • Partners • University of Gothenburg (technical coordinator) • University of the Saarland (administrative coordinator) • University of Seville • Telefonica I+D • Linguamatics • SRI (original coordinator) • Duration • Jan 2000 - Dec 2002

4. Motivation • Provide flexible, user-centred dialogue systems • Improve reconfigurability of dialogue systems for • new tasks • new domains • Provide architectures allowing the dialogue state to be accessible for speech recognition, generation of prosody etc.

5. IS transitions IS for System IS for System IS for System Generate U’ Interpret U • dialogue as transitions between IS states • what information do we need in the IS? • how should it be structured? • can this information be reused e.g. synthesis/recognition? • examples • underspecified commands (DELFOS) • questions under discussion (GODIS)

6. Home manager VCR manager TeleDELFOS Travel Agency Home manager GoDiS-I GoDiS-A DELFOS-NCL IBDM/ KOS GoDiS ISU approach ISU approach Delfos basic system TrindiKit Delfos framework

7. Research Areas • Dialogue types • Dialogue phenomena • Enhancing Contextual Appropriateness of System Output • Enhancing speech recognition through IS • Robust interpretation

8. Dialogue Types • Natural command dialogues • user initiated commands e.g. • transfer my calls to Mr. Jones • turn off all the lights upstairs and lock the front door • Negotiative dialogues • Propose, negotiate & agree • parameters for a task e.g. arrival-time, destination-time • alternative solutions to a problem e.g. different flight options • Tutorial dialogues • guide student through task • non collaborative • hinting (to encourage active learning)

9. Dialogue phenomena • Conditional responses • Enhancing collaboration by: • Indicating source of failure: • U: Can I fly on the second? • S: Not if you want to fly economy class. • Indicating contingency of success: • S: Yes, if you fly with SAS. • Clarifications • Lack of specificity • S: Which account do you want to transfer to? • U: My bank account • S: Do you mean your deposit account or current account? • Unspecified task e.g. • U: Channel five • S: Do you want to add a program or change channel?

10. Enhancing Contextual Appropriateness of System Output • In order to make dialogue system utterances sound natural and contextually appropriate, it is important • to control intonation of spoken output • to control other aspects of realization, such as generation of short utterances • We handle these aspects using Information Structure • what speaker means to address vs. what she wants to say about it • what is the same vs. what discriminates between similar bits of information in context • Information Structure is determined from the IS

11. Enhancing speech recognition using IS • In flexible dialogue, may have dynamically generated prompts • Can’t rely on pre-defined grammars to recognise the response for each prompt • Potential sources of information: • previous move/dialogue history(choice from n-best) • syntactic/semantic coherence of fragments (direct parsing of word lattices) • reference resolution • state of the world • Combine/contrast sources of evidence to decide • most likely utterance in word lattice / n-best • when to clarify

12. Robust Interpretation • Phrase spotters go to a particular depth of analysis even if a full parse is possible • When syntactic information is available should use it • When not, should do at least as well as keyword/phrase spotting • Two novel techniques: • distribute the semantic representation using a semantic chart • ontological knowledge to propose combinations via semantics in competition with syntax: semantic-based composition turn on the light and the kitchen • Reconciling traditional syntax and compositional semantics with robust approaches

13. Main Demonstrators • TrindiKit 3 • Automated Telephone Operator

14. Trindikit 3 • Toolkit for dialogue researchers • Developed from TrindiKit 1 built in EU Trindi Project • IS update rules allow you to write your own dialogue manager • IS +move(s)  IS • Update rules suited for domain and language independent dialogue management • Global information state • all modules can access all information • composed from library of datatypes

15. Automated telephone operator • Natural language telephone-based access to company telephone directory/PABX • Functionality • Dial by name • Multi-party conference • Call transfers • Database search (e-mail address, office, phone number) • KQML based message passing • Delfos dialogue management • User trials at Telefónica I+D

16. Telephone Operator GUI Corporate Directory / PABX

17. Further Demonstrations • Specific modules • Intonation • Conditional responses • Robust interpretation • Applications to illustrate reconfigurability • Travel agency • VCR • Command and control of multiple devices (D’Homme project)

18. Demonstrators: VCR Task clarification Issue clarification + Feedback S> “What can I do for you? " U> five thirty S> "five thirty. I dont quite understand. Do you want to add a program or set the clock?“ U> add a program today S> "Okay. add a program…. Lets see. today… Do you want from five thirty or until five thirty?“ U> from five thirty S> "Okay. from five thirty… What channel do you want? " U> set the clock S> "Okay. Do you want to set the clock?“ U> yes S> "Lets see. What time? " U> four fifteen S> "Okay. four fifteen… The clock was set…Returning to add a program. What channel do you want? “ U> channel five S> "Okay. channel five." U> pardon S> "Okay. channel five." U> yes S> "What time do you want to stop recording? " U> turn on the t v S> "turn on the TV… Sorry, I cannot control the TV set. What time do you want to stop recording? “ U> seven thirty S> "Okay. until seven thirty…The program has been added. " Load plan Requesting other action ? Feedback Reraising - Feedback (by User) Rejection

19. Demonstrators: Home device control

20. Siridus Summary • IS approach has provided a stimulus for theoretical work • use of IS to improve • intonation of spoken output • recognition • exploration of different dialogue genres and phenomena • systems which are • modular • reconfigurable • between FS/form filling and BDI/planning • an automated telephone operator demonstrator • research tool for dialogue developers (TrindiKit 3)