Spoken Dialogue Technology

1. Spoken Dialogue Technology How can Jerry Springer contribute to Computer Science Research Projects?

2. Objectives: • Review the article, “Spoken Dialogue Technology: Enabling the Conversational User Interface” by Michael F. McTear • Introduce Spoken Dialogue Technology • Discuss potential areas of research in this field

3. What is spoken dialogue technology? Spoken dialogue technology allows users to interact with computer-based applications such as databases and expert systems by using natural spoken language.

4. What problem does spoken dialogue technology solve? • Makes human-computer interaction more natural for human beings • Why do people get carpal tunnel syndrome? • Reaches differently-abled end users

5. How do we enable conversational user interfaces? • McTear’s paper addresses this question by: • Introducing spoken dialogue systems • Classifying these systems • Explaining the required components • Referencing current development trends • Reviewing development toolkits • Suggesting future research possibilities

6. How are spoken dialogue systems classified? Spoken dialogue systems are classified according to the system’s dialogue control method. • Finite State-based • Frame-based • Agent-based

7. Finite State-based systems • A dialogue consists of a predetermined sequence of steps. • User input is highly constrained, consisting of single words or phrases • The simplicity of such systems accounts for both the advantages and disadvantages of this approach • Example: Nuance Demo Banking System

8. Frame-based Systems • These systems gather all of the information necessary to complete a task • The dialogue flow depends on the user’s input and the system’s required data as specified in a template • Example: Philips Train Timetable

9. Agent-based Systems • These systems permit complex communication between the system, the user, and underlying applications. • Mixed-initiative systems: communication is viewed as interaction between two agents. • Example: Circuit-Fix-It Shop System

10. Required Components • Speech Recognition • Language Understanding • Dialogue Management • Communication w/ External Systems • Response Generation • Speech Output

11. Current Development Trends • Analyze Human-Human dialogues • Design based on Simulations • Usability Analysis • Guidelines and Standards • Evaluation Metrics

12. Development Toolkits • Many toolkits exist to develop spoken dialogue systems • Examples: • CSLU toolkit: Center for Spoken Language Understanding • SpeechMania • More examples: • ASDA: Anthropomorphic Spoken Dialogue Agent • Psyclone

13. Future Research Possibilities • McTear lists many possible areas for research: • Incorporation of a prosody analyzer • Multimodal dialogue systems • Perhaps more work on Theory • Classification • Architecture and Components

14. Prosody Analyzer • Why do most ancient epic poems conform to strict rhyme, rhythm, and metrical forms? • Rhyme and rhythm were used to “encode” and “transmit” knowledge. • Example from Cheers: Albania!

15. Prosody Analyzer • Researchers refer to prosody as a separate channel of communication distinct from the lexical channel. • If we could capture the information communicated on this channel, our spoken dialogue systems would be more robust.

16. Prosody Analyzer: Research • Bell Labs Tutorial: http://prosodies.org/tutorial2002 • Prosody Analysis for Speaker Affect Determination • Researchers concluded that the best source of prosody data was daytime talk shows like Jerry Springer!!!

17. Multimodal Dialogue Systems