1 / 16

On Defining Cephalic Gesture Categories

On Defining Cephalic Gesture Categories. Natalya V. Sukhova , Y ulia V. Nikolaeva Institute of Linguistics RAS, Lomonosov Moscow State University NUST MISiS. Outline of the Talk. About the Corpus. Annotation of Kinetic Phenomena. General Concept.

pcasto
Download Presentation

On Defining Cephalic Gesture Categories

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On Defining Cephalic Gesture Categories Natalya V. Sukhova, Yulia V. Nikolaeva Institute of Linguistics RAS, Lomonosov Moscow State University NUST MISiS

  2. Outline of the Talk • About the Corpus. • Annotation of Kinetic Phenomena. General Concept. • Cephalic Gestures. Approaches to Analysis. • Annotation of Cephalic Gestures. Our Approach. ICMC 2017, Osnabrück

  3. 1a Language As It Is: Multimodal (Multichannel) Resource • Goal is to create a resource approaching the actual richness of human discourse • Registered phenomena: • verbal structure • prosody • eye gaze • head movements • hand gestures • torso position ICMC 2017, Osnabrück

  4. 1b Spoken Multimodal Discourse • Discourse Language? • Kinetic / visual modality • Vocal modality Proxemics Other channels Facial expression Other modalities Many other components Intonation Hands Head • Posture • Prosody • Verbal channel • Gesture • Eye gaze Most multimodal studies ICMC 2017, Osnabrück 4

  5. 1с RuPeCS Russian Pear Chats and Stories (Pear Film, Chafe ed. 1980): • 24 conversations • 96 native speakers • total duration is 10 hours • 110 000 words • 11 media files for each conversation • 24 written retellings Project web-site: www.multidiscourse.ru ICMC 2017, Osnabrück 5

  6. 1dThe Scene Tobii glasses Listener Listener Narrator Reteller Narrator Reteller Commentator Commentator 6 6 6

  7. 2a Manual Gestures • Communicatively significant spontaneous hand/arm movements, accompanying speech • Discourse functions • Adding to speech contents, facilitating verbalization, appealing to an interlocutor, regulating turn-taking, supporting rhythmical structure • Standing out as a figure against kinetic background (other movements and stillness periods) • Kinetic structure • Distinct articulatory effort, velocity, trajectory, movement direction, hand shape and orientation, gesture range, location in gesture space ICMC 2017, Osnabrück

  8. 2b Distinguishing Movements and Positions Neutral (resting) positions: set of preferred hand/arm positions a speaker regularly returns to. Hand/arm posture changes: hand/arm resting position changing movements. Self-adaptors (adaptors, manipulators): physiologically motivated movements or movements with practical purposes. Gestures Typical kinetic patterns of a speaker form a gesticulation portrait (cf. ‘prosodic portraits’ in Kibrik 2009). ICMC 2017, Osnabrück

  9. 3a Cephalic Gesture Categories • MOVEMENT is all motor activity of a head. • GESTURE is all motor activity that potentially has a meaning. • HEAD POSTURE (point, position) brings about a change of a head position in at least one direction and may (or may not) last for some time. ICMC 2017, Osnabrück

  10. 3b Levels of Analysis • Form – physical description • Meaning – e.g. nods, shakes • Function – a role in a discourse, e.g. backchannelling SeeHadaret al. 1985; Bull 1987; McClave 2000; Allwood, Cerrato 2003; Benoit, Caplier2005; Kousidiset al. 2013. ICMC 2017, Osnabrück

  11. 3c Our Approach to Cephalic Gesture Categories. Tiers in ELAN ICMC 2017, Osnabrück

  12. 4a Physical Form ICMC 2017, Osnabrück

  13. 4b Formal Vocabulary ICMC 2017, Osnabrück

  14. Picture in ELAN ICMC 2017, Osnabrück

  15. CONCLUSIONS & PERSPECTIVES Current annotation scheme: • provides detailed annotation of movements relevant in communication; • takes into account most frequent complications; • makes a part of a consistent annotation system for several kinetic channels (presuming that they are used simultaneously to pass along one message); • allows comparing head and manual gestures to vocal channel units and other kinetic channels’ units. Further research: • extending annotated subcorpus; • measuring gesticulation portrait features. ICMC 2017, Osnabrück

  16. Thank you for attention! sukhova.natalya@gmail.com ICMC 2017, Osnabrück

More Related