1 / 16

Conversational role assignment problem in multi-party dialogues

Conversational role assignment problem in multi-party dialogues. Natasa Jovanovic Dennis Reidsma Rutger Rienks TKI group University of Twente. Outline. Research tasks at TKI Interpretation of multimodal human-human communication in the meetings

waseem
Download Presentation

Conversational role assignment problem in multi-party dialogues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Conversational role assignment problem in multi-party dialogues Natasa Jovanovic Dennis Reidsma Rutger Rienks TKI group University of Twente

  2. Outline • Research tasks at TKI • Interpretation of multimodal human-human communication in the meetings • Conversational Role Assignment Problem (CRAP) • Towards automatic addressee detection

  3. A framework for multimodal interaction research

  4. Multimodal annotation tool

  5. Who is talking to whom? • CRAP as one of the main issues in multi- parity conversation (Traum 2003.) • Taxonomy of conversational roles (Herbert K. Clark) speaker addressee side participant bystander all participants all listener eavesdropper

  6. Our goal: • Automatic addressee identification in small group discussions • Addressees in meeting conversations: single participant, group of people, whole audience • Importance of the issue of addressing in multi-party dialogues

  7. Addressing mechanisms • What are relevant sources of information for addressee identification in the face-to-face meeting conversations? • How does the speaker express who is the addressee of his utterance? • How can we combine all this information in order to determine the addressee of the utterance?

  8. Sources of information • Speech • Linguistic markers • word classes: personal pronouns, determiners in combination with personal pronouns, possessive pronouns and adjectives, indefinite pronouns, etc. • Name detection ( vocatives) • Dialogue acts • Gaze direction • Pointing gestures • Context categories(features)

  9. Dialogue Acts and Addressee detection (I) • How many addresses may have an utterance? • According to dialog act theory an utterance or an utterance segment may have more than one conversational function. • Each DA has a addressee ==> an utterance may have several addresses

  10. Dialogue Acts and Addressee detection (II) • MRDA (Meeting Recorder Dialogue Acts)– tag set for labeling multiparty face to face meetings (ICSI) • We use a huge subset of the MRDA set which is organized on two levels: • Forward looking functions (FLF ) • Backward looking functions (BLF)

  11. Non-verbal features • Gaze • Contribution of the gaze to the addressee detection is dependent on: participants’ location (visible area), utterance length, current meeting action • Turn-taking behavior and addressing behavior • Gesture ( pointing at a person) • TALK_TO (X,Y) AND POINT_TO (X,Y) • TALK_TO( X,Y) AND POINT_TO (X,Z) – X talk to Y about Z

  12. Context categories • Bunt: “totality of conditions that may influence understanding and generation of communicative behavior” • Local context is an aspect of context that can be changed through communication • Context categories: • Interaction history ( verbal and non-verbal) • Meeting action history • Spatial context (participants’ location, distance, visible area, etc. ) • User context (name, gender, roles, etc. )

  13. Towards an automatic addressee detection • Manual or automatic features annotation? • An automatic target interpreter has to deal with uncertainty • Methods: • Rule-based method • Statistical method ( Bayesian networks)

  14. Rule-based method • Processing information obtained from the utterance ( linguistic markers, vocatives, DA). The result is a list of possible addressees with corresponding probabilities • Eliminate cases where target is completely determined (for instance, name in vocative form) • Set of rules for BLF • Set of rules for FLF • Processing gaze and gesture information adding the additional probability values to the candidates

  15. Meeting actions and addressee detection • Automatic addressee detection method can be applied to the whole meeting • Knowledge about the current meeting action as well as about meeting actions history may help to better recognize the addressee of a dialogue act.

  16. Future works • Development of multimodal annotation tool • Data annotation for • training and evaluating statistical models • obtaining inputs for rule-based methods • New meeting scenarios for research in addressing

More Related