Spoken Dialogue Technology

1. Spoken Dialogue Technology

2. Acknowledgements • Prof. Mctear, Natural Language Processing, http://www.infj.ulst.ac.uk/nlp/index.html, University of Ulster.

3. Natural Language Processing: A Definition • A 'natural language' (NL) is any of the languages naturally used by humans, i.e. not an artificial or man-made language such as a programming language. • 'Natural language processing' (NLP) is a convenient description for all attempts to use computers to process natural language.

4. Computational Linguistics Computational linguistics (CL) is the scientific study of language from a computational perspective • Theoretical / scientific perspective: • Deals with formal theories about the linguistic knowledge that a human needs for generating and understanding language. • Involves development of formal models simulating aspects of the human language faculty and implementing them as computer programmes. • Applied / technological perspective (language engineering or (human) language technology) • focuses on the practical outcome of modelling human language use • methods, techniques, tools and applications • software products that have some knowledge of human language

5. NLP: Models and Algorithms • Many of the key issues in (spoken) natural language processing can be captured through the use of a small number of formal models: • state machines, formal rule systems, logic, probability theory • and algorithms: • state space search (parsing), dynamic programming

6. NLP and other disciplines • linguistics • theory of language, general principles of language structure • psycholinguistics • NLP as a functional model of human language processing • philosophy of language • language as action (speech act theory), issues of meaning and language use • computer science • logic programming, data structures e.g. object hierarchies, list processing, compilers (theory of grammars)

7. APPLICATIONS OF NLP • spelling and grammar checking • natural language interfaces to database • machine translation • information retrieval, information extraction, question answering • text and audio mining • augmentative and alternative communication • spoken language systems

8. Machine Translation Example • Large public Web sites and services such as AltaVista and CompuServe are beginning to experiment with MT, allowing users to translate documents and Web content on the fly. • Les grands sites publics et les services de Web tels qu'AltaVista et CompuServe commencent à expérimenter avec la TA, permettant à des utilisateurs de traduire les documents et le contenu de Web en marche.

9. Spoken Language Systems (1) • Command and control • Controlling equipment e.g. computers, machinery, appliances • Useful for hands-busy / eyes-busy applications • Recent developments: speech to control appliances in cars • Small vocabulary, restricted combinations • Data Entry • Entering data into spreadsheets, databases, etc. • Form completion, package sorting, equipment maintenance, traffic accident reports, … • Small vocabulary (+ application specific words, e.g. street names)

10. Spoken Language Systems (2) • Dictation • Translating spoken word into written text • Systems often include command and control functions • Large vocabulary • No natural language understanding involved • Information retrieval using speech (and telephone) • Weather, travel schedules, share prices, … • Example: MIT’s Mercury system (flights) • Vocabularies for domain e.g. airport names, company names, etc. • Spoken dialogue technology

11. Language generation Dialogue management Audio Back end Speech recognition Language understanding A spoken dialogue system There’s a flight leaving at 7.45am arriving at 8.50am Where would you like to travel to? To London on Friday arriving at 9

12. Dialogue Interfaces System: Welcome to the Electronic Restaurant Guide for London, brought to you by Philips Dialogue Systems.Would you like to hear an introduction? Caller: No System: At what type of restaurant would you like to eat? Caller: I’m looking for a Thai restaurant in the West End for fifteen pounds System: Are you looking for a Thai restaurant in the West End at around fifteen pounds per person? Caller: Yes System: There is the following Thai restaurant: the Barn Thai restaurant located at 21 Frith Street in the West End. Would you like a full description? Caller: Yes

13. Dialogue Management Basic Idea: • Controlling the dialogue flow Examples of dialogue management tasks: • deciding which question to ask next • deciding whether to ask for clarification of unclear input • deciding when to look up the database

14. Problem detection: Speech recognition? Ask to repeat Problem detection: Not speech recognition Out of domain query? Leading prompt: hint Example Clarifying unclear input VZ: Is it sunny anywhere in the Caribbean? Jupiter: Sorry, I am not sure what you said. VZ: Is it sunny anywhere in the Caribbean? Jupiter: I am sorry, I may not know the city you are asking about. Try asking for the state or country. For example, what cities do you know about in Massachusetts?

15. Range of Spoken Dialogue Systems • simple query which department? men’s shoes (give number or transfer call) • system directed dialogue • system elicits relevant parameters for database query • mixed initiative dialogue with intelligent conversational agent • route planning, disaster management, ... • both system and user control the dialogue • complex dialogue modelling

16. Statement of problem Meaning of ‘vehicles’ Meaning of ‘available’ Detection of possible obstacle to proposed plan: Reasoning about plan Advanced dialogue example (TRIPS) User: We need to get the woman in Penfield to Strong. System: OK. User: What vehicles are available? System: There are ambulances in Pittsford and Webster. User: OK. Use one from Pittsford. System: Do you know that Route 96 is blocked due to construction?

17. Why spoken dialogue? • limitations of simple applications • can have over-complicated menu structures • allows users to express their request more directly, not navigating through a menu • problems with large numbers of names of persons or destinations when using a digit string from a pre-published sheet • can express multiple parameters within one utterance or work through the required parameters to formulate a query

18. Touch-tone Dialogues (DTMF): Example S: Thank you for calling the Spare Parts Corporation. If you wish to place an order, please key number ‘1’ on your telephone now, otherwise please hold on. U:<Presses ‘1’ key> S: Using the telephone keypad, please enter the stock code of the first item you wish to order U:<Presses 5-7-3-2-8-4> S: Please enter the number of items you require U: <Presses 1-0-0> S: One hundred items of stock number 5-7-3-2-8-4. Press ‘1’to confirm or ‘2’ to cancel U: <Presses ‘1’> S: Please enter the stock code of the next item you wish to order, or press ‘star’ if your order is complete.

19. Requirements for Dialogue Systems • natural - like human conversation • mixed initiative - each participant can take the initiative • co-operativity - system should try to satisfy the user’s goals, even where they are not expressed directly • robustness - system should be able to process ill-formed input