Towards Conversational Human Computer Interaction

1. MOIS 466- Human Computer Interaction Mini-Project 1 Dept. of Computer Science University of Rochester Rochester, NY 14627 By: James F. Allen, Donna K. Byron, Myroslava Dzikovska George Ferguson, Lucian Galescu, Amanda Stent Present By: Nourhan Younis Towards Conversational Human Computer Interaction

2. What is a dialogue system? • A computer intended to converse with a human, with a coherent structure.

3. Dialogue System • Limit communication • Increasing efficiency of service • Making speech more predictable • Easier recognition and language processing

4. Human Conversation • Basing HCI on human conversation • More interaction and complexity BUT • People claim it won’t be as effective as graphic user interfaces (GUIs)

5. Conversational Interfaces Vs. GUIs Limitations of Graphic User Interfaces: • The size of the device used • Some tasks requires the use of hands and eyes • More flexible • Reduces the training time • The provide a set of operations with many complex, sequential tasks • Adds high level commands with many options • Requires lots of training

6. Considerations • It can’t totally replace GUIs because: • The user will have to remember a long set of verbal commands to perform a task (ex. Menus) The Conversational inter- faces, however, would provide the opportunity for the user to state what he/she wants to do in his/her own terms, just as he/she would do to another person, and the system takes care of the complexity.

7. Conversation Interfaces • Conversational Interaction will not depend on a series of commands But • Interaction involves: • Defining and discussing tasks • Exploring ways to perform the task • Collaborating to get it done All interactions are contextually interpreted with respect to the interactions performed to this point, allowing the system to anticipate the user’s needs and provide responses that best further the user’s goals

8. Dialogue Task Complexity

9. Hypotheses • H1: Practical Dialogues Conversational competence required for practical dialogues, although still complex, is significantly simpler to achieve than general human conversational competence. • H2: The Domain Independence Within the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management are independent of the task being performed

10. A Fragment of Practical Dialogue

11. Challenges For Practical Dialogue • Parsing language in practical dialogues • Integrating dialogue and task performance • Intention recognition • Mixed initiative dialogue

12. 1.Parsing Language In Practical Dialogues • Need to produce a detailed semantic representation the captures what the user meant by the utterance • Put limitations on the different meanings of the words • Encode restrictions to make sure words are in the right context • Spoken language is not sentence based, rather a single utterance may realize a sequence of communicative acts called Speech Acts • The grammar used is an act description rather than sentence structures

13. Speech Acts EXAMPLE: “OK, let’s do that then send a truck to Avon” • OK: an acknowledgement • Let’s do that: an acceptance • Send a truck to Avon: a request

14. 2. Integrating Dialogue and Task Performance • We need dialogue systems that can be adapted to any practical tasks • Requires an AGENT BASED FRAMEWORK • Dialogue Systems: • Knows about the tasks that needs to be accomplished • The generic system is specialized to the particular domain by integrating domain specific information • Broker : link between the dialogue system and backend • Backend: a set of agents providing services

15. The Agent Based Architecture

16. Abstract Problem Solving and Domain-Specific Task Models • The key to this enterprise is the development of an abstract problem solving model with: • Objectives • Solutions • Resources • Situations • As utterances can mean different things in different domains: • A domain specific task model: provides mapping from the abstract problem solving model to the operations in a particular domain by specifying what things count as objectives, solutions, resources, and situations

17. 3. Intention Recognition • Determining what the user wants to say by saying the utterance • Example: Emergency System that will be used to evacuate people from an island in the face of a hurricane

18. Example

19. Intention Recognition (Cntd) • That example shows different interpretations of those utterances • Reasoning used, requires information about the task to identify which makes sense rationally in the current situation • The challenge here is to match the intentions with the context of the the question asked

20. 4. Mixed Initiative Dialogue • Practical human dialogue involves mixed initiative interaction i.e. dynamic exchange of control of the dialogue flow, increasing efficiency, effectiveness; enabling both participants'’ needs to be met • For systems that ask questions it can work in simple tasks, like long distance dialing • For complex tasks: you may often need to get through irrelevant interactions before getting the information you need.

21. Limitations of Frame Based System • The user has to supply information for the system, until there is sufficient information for the system to perform the task BUT • The user might not know what information he might still needs to supply So • Currently system provide limited mixed initiative interactions

22. Conclusion • There are still lots of technical issues remaining to overcome to increase the efficiency and effectiveness of the dialogue based user interfaces. Thank You