Spoken Dialogue Systems and the GALAXY Architecture 29 October 2000

1. Spoken Dialogue Systems and the GALAXY Architecture29 October 2000 Advanced Technology Laboratories 1 Federal Street • A&E Building 2W • Camden, New Jersey 08102 Jerry FrankeSenior Member, Engineering Staffjfranke@atl.lmco.com 856.338.3341

2. Talk Outline • Spoken Language Development • GALAXY II System • SUMMIT (speech recognition) • TINA (natural language parsing) • GENESIS (natural language generation) • Turn Management • GALAXY II Domains • Overview/Demonstration

3. Spoken Language Development • Universities • MIT, CMU, Colorado • Basic research labs • ATT, SRI • Software developers • Nuance, SpeechWorks • Domain developers • Lockheed Martin - ATL

4. GALAXY II System • Developed by MIT Spoken Language Systems group • Multiple servers performing parts of the dialogue process S o u n d SUMMIT TINA GENESIS S o u n d Speech Recognition Language Understanding Turn Management Language Generation Speech Synthesis

5. SUMMIT (speech recognition) • Three elements: vocabulary, language models, acoustic models • Pause words stripped out • AM: segment-based models and boundary-based diphone models • LM: Forward Viterbi search with a class bigram model, followed by a backward A* search with a class trigram model • Produces N-best list or word graph of possible utterances • Models trained on domain corpus • Models achieve speaker-independence

6. TINA (natural language parsing) • Selects from N-best list depending on grammatical parse • Grammars reflect both syntactic and semantic structure • Result is a semantic frame • Example: “Where is the library in Swain Hall?” Clause: LOCATE Topic: PUBLIC-BUILDING Quantifier: DEF Name: library Predicate: IN Topic: HALL Name: Swain

7. GENESIS (natural language generation) • Processes semantic frames • Embeds semantic frame components into context-dependent message templates • Two types of output: • natural language messages • messages are sent to some speech synthesis module • possibility of output in multiple languages • keyword-value pairs • useful structure for the turn management backend • Can be used to map between (translate) languages

8. Turn Management • Manages the system’s part of the dialogue • Fuses current utterance with dialogue history for full context • Five main tasks: • Answer user’s requests (information retrieval) • Initiate sub-dialogues to clarify the user’s request • Track progress through the dialogue • Control response to the user • Provide assistance in using the system when needed

9. GALAXY II Domains From MIT: • Jupiter - weather forecasts • Pegasus - airline scheduling • Voyager - Cambridge, Massachusetts city guide • Dinex - Boston restaurant guide • Wheels - automobile classified ads • Mercury - airline flight booking From Lockheed Martin - ATL: • DARPA Communicator - airline flight, hotel, car rental booking • DARPA LCS-Marine/Marine Small Unit Logistics - supply request • DARPA LCS-Army - data collection during equipment tests

10. Overview/Demonstration • Booking airline flights (round trip and one way) • Hotel, car rental • Uses user profile to fill in some information about reservation • Uses some real-world knowledge • Information retrieval via mobile agents

11. Try It Yourself • CMU 1-412-268-1084 • MIT 1-877-527-8255 • Colorado http://communicator.colorado.edu/