1 / 9

Workshop « Multimodal Corpora »

Workshop « Multimodal Corpora ». Jean-Claude MARTIN ( MARTIN@LIMSI.FR ) Patrizia PAGGIO Peter KÜEHNLEIN Rainer STIEFELHAGEN Fabio PIANESI. « Multimodal Corpora» Workshops at LREC. 2006 Summary. Several types of settings / corpora required Human-Computer Interaction: GUI, ECA, robot

pamelab
Download Presentation

Workshop « Multimodal Corpora »

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workshop« Multimodal Corpora » Jean-Claude MARTIN (MARTIN@LIMSI.FR) Patrizia PAGGIO Peter KÜEHNLEIN Rainer STIEFELHAGEN Fabio PIANESI

  2. « Multimodal Corpora» Workshops at LREC

  3. 2006 Summary • Several types of settings / corpora required • Human-Computer Interaction: GUI, ECA, robot • Human-Human: TV, Cartoon, Meetings, Conversations, scenario based or not • Human modalities: hand gestures, audiovisual speech, head, gaze, interpersonal distance • Top-down vs. bottom-up approaches • Evaluation & validation of annotation • Duration of videos data, Nb of video taped people • Media : audio, image, sensors

  4. 2006 Summary • Research questions • Capturing semantics and pragmatics of modalities • Social behavior • Dynamics of individual vs. group behavior • Topic of attention, functional role • Turn management, sequencing, feedback • Interaction loops at multiple levels vs. high level dialogue • Level of embodiment (robot / ECA) • Temporal relations between modalities • Cultural & individual differences • Naturaleness of corpora • Mapping between different schemes • Complementarity between manual & automatic annotations • Emotion related states & mm behavior (expressivity) • Workshop on Emotional Corpora (L. Devillers, J.C. Martin, E. Cowie, R. Cowie, A. Batliner)

  5. 2006 Summary • Practical questions • Methodological steps (basic functions vs. Full-fledged complexity) • Dissemination/availability of corpora (Grenoble, Norway, …) • Cooperations (e.g. meetings, multilinguality)

  6. 2006 Summary • Cross disciplinary: linguists, computer sc., … • International projects: HUMAINE, Similar, AMI, CHIL, CALO, ATR • Towards « standards » • Abstract coding: gesture, facial expr., • Language • W3C EMMA • Emotion EARL (cf WS Emotions) • Tools : Anvil used a lot • Decrease of participation ?

  7. Next steps • Related meetings • LREC 2008 / Interspeech / Intelligent Virtual Agents • Other Communities (e.g. expressive robotic) • IPRA07 : more theoretical incl. WS Multimodal pragmatics & corpus • ICMI07 : more technological, evaluation + application of mm corpora • Topic of next Multimodal WS • From corpora to application / studying phenomena • Data driven models • Standardisation • Tools usability • Animated character as perceptual tools • Focused scheme vs exploratory scheme • Machine Learning (ex export from annotations to WEKA) • Manual and automatic annotations • Cooperation HH vs HCI • Scientific maturity, Special issue of journal with open call

  8. Workshop« Multimodal Corpora » Jean-Claude MARTIN Patrizia PAGGIO Peter KÜEHNLEIN Rainer STIEFELHAGEN Fabio PIANESI

  9. Multimodal ParticipationMain conference • Numbers from Khalid and Joseph (incl. Speech and multimodal) • LREC • 1998 : 77 / 197 • 2000 : 86 / 280 • 2002 : 123 / 365 • 2004 : 142 / 525 • 2006 : 103 or 115 / 512

More Related