1 / 12

Markup of Multimodal Emotion-Sensitive Corpora

Markup of Multimodal Emotion-Sensitive Corpora. Berardina Nadja de Carolis, Univ. Bari Marc Schröder, DFKI. Our Background: Generation of ECA Behavior. Believable ECA: multimodality, affect, personality, human-like, etc. Two approaches: Bari APML (Affective Presentation Markup Language)

chip
Download Presentation

Markup of Multimodal Emotion-Sensitive Corpora

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markup of Multimodal Emotion-Sensitive Corpora Berardina Nadja de Carolis, Univ. Bari Marc Schröder, DFKI

  2. Our Background:Generation of ECA Behavior • Believable ECA: multimodality, affect, personality, human-like, etc. • Two approaches: • Bari APML (Affective Presentation Markup Language) • NECA RRL (Rich Representation Language)

  3. Overview • Markup Schema Definition • What to Annotate • What is a Markup Language • What is XML • Dealing with complexity of human communication • Labelling Tools

  4. Markup Scheme Definition • What to annotate? (verbal, non-verbal behavior, emotions, context, …) • Units of consideration (utterances/ turns/ paragraphs/ time units/…) • Descriptive Scheme • Categorical • Dimensional • Functional • …

  5. The Question of Coding • How to Share and Access Data Annotation? • Standardization • Format Readable for Computers (statistics, inference, …) • XML-based Markup Languages!

  6. What is a Markup-Language • Dictionary Definition: a set of symbols and rules for their use when doing a markup of a document nomenclature, terminology, language - a system of words used in a particular discipline; • Computer Domain Examples: • HTML • LATEX • RTF • … • But … • proprietary language • not generic • not standard • fixed syntax and semantics <?XML!>

  7. <pitch level=“high”> <emotion type=“reprimanding”> </pitch> </emotion> What is XML XML (eXtensible Markup Language) is • a language for creating markup languages that describe data thus, a meta language used to define new markup languages (domain- or industry-specific). • Simple, open standard, extensible, ready to use, … XML has a lot of friends that makes computation easy (XSL, DOM, SAX, …) and interoperable (RDF, RDF Schema, OWL, …) No No No! I said turn left!!!

  8. Dealing with complexity of human communication • Communication at different non-exclusive abstraction levels • Mental state (Beliefs, Goals, Intentions, Affective State, …) • Discourse (Communicative Goals, Focus, Relations) • Meanings (Performative, Affect, Turn-Taking, Emphasis) • Signals (Physical Communicative Expressions) • Defining Different Markup Layers • Separate levels of meaning into different layers • Define a Markup Language for each layer • Define Interaction between layers • Develop Easy-to-use tools

  9. Labelling tools:Text-centered annotation: MMAX http://mmax.eml-research.de

  10. Labelling tools:Video-centered annotation: ANVIL http://www.dfki.de/~kipp/anvil

  11. Feedback into Generation Systems • Annotated Corpora can be used for improving ECA systems: • extracting knowledge • markup language for specifying ECA behaviors • …

  12. Overview of the session • Introduction to XML structure • Demos • Use of APML in an ECA system • XML markup in MARY Text-to-speech • Hands-on: The ANVIL tool and the resulting XML markup

More Related