1 / 23

So nification of the facial expressio ns

So nification of the facial expressio ns. New Interaction Techniques Department of Computer Science University of Tampere. Guo Zhiguo Ioulia Guizatdinova Zhiguo.Guo@uta.fi ioulia_gizatdinova@pochtamt.ru. 27 February 2003. Contents. Introduction to the problem

abeni
Download Presentation

So nification of the facial expressio ns

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sonification of the facial expressions New Interaction Techniques Department of Computer Science University of Tampere Guo Zhiguo Ioulia Guizatdinova Zhiguo.Guo@uta.fiioulia_gizatdinova@pochtamt.ru 27 February 2003

  2. Contents • Introduction to the problem • Background & related research • Objectives • Methods • Resources & experimental setup • Preliminary results • Prospectives p 02_23 27.02.03Sonification of the facial expressions

  3. Introduction to the problem Sonification is the use of non-speech audio to convey information more specifically sonification is the transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation [1] p 03_23 27.02.03Sonification of the facial expressions

  4. Introduction to the problem The main question for sonification of the facial expression is: How to convey the emotional expressions/facial features/ of some person presented like any visual imaging into the sounds (e.g. earcons, music chords) or whatever ? p 04_23 27.02.03Sonification of the facial expressions

  5. Introduction to the problem The sonification system for facial expressions should allow the transformation of the visual information (attributes specifically describing emotional expression/state) into sound patterns This kind of sounds (“eARmoticons”) should evoke similar emotional feelings of the listener that is, to provide an emotional access through “audio snapshot” when visual perception is blocked or imaging is hidden for observer p 05_23 27.02.03Sonification of the facial expressions

  6. Background & Related Research Some facial expressions can hardly be transformed by sound (e.g., congenital or non-classified features) Blind people have a wide experience with real environmental sounds, while the congenital blinded can have not any association to visual facial imaging; only tactile memory of some facial features can be insufficient or non-adequate p 06_23 27.02.03Sonification of the facial expressions

  7. Background & Related Research • There is not signs’ system/auditory or tactile/specifically & briefly describing facial features/expressions. Only some general emotional states are presented like artificial set of static visual/tactile symbols [2, 11]. Happiness Frustration Madness Sadness : ( : / >:-( : ) p 07_23 27.02.03Sonification of the facial expressions

  8. Background & Related Research There have been a lot of approaches to data sonification of like math functions, simple graphics, diagrams, … The vOICe Java Applet http://www.visualprosthesis.com/javoice.htm p 08_23 27.02.03Sonification of the facial expressions

  9. Background & Related Research Relevantly, there are a lot of earcons used in MS Windows and other software, sound signaling for special equipment and application… sound sound sound sound p 09_23 27.02.03Sonification of the facial expressions

  10. Objectives The goal of this project is designing and usability investigation of the sonification system for attributes specifically describing facial expression to provide an emotional access through eARmoticons when visual perception is blocked or imaging is hidden for observer p 10_23 27.02.03Sonification of the facial expressions

  11. Schedule p 11_23 27.02.03Sonification of the facial expressions

  12. Research Methods • constructive research to design and simulate the new techniques, software tools and doing usability evaluation of novel sound objects – eARmoticons • empirical research is based on crossmodal information transfer and perception of intermodal events p 12_23 27.02.03Sonification of the facial expressions

  13. As a whole Facial landmarks Preliminary Modelling Recognition of the sound patterns to facial expression Facial expression (image, picture, video, etc) Video processing Facial expression-to-sound mapping transformation system sounds Sound Database p 13_23 27.02.03Sonification of the facial expressions

  14. Resources & Experimental Setup • Subjects (people with normal vision) • Stimuli – eARmoticons – earcons specifically & briefly describing facial features/expressions FACS – Facial Action Coding System [ 7, 8] a prototype of the sonification system for facial expression transformation p 14_23 27.02.03Sonification of the facial expressions

  15. Resources & Experimental Setup Experimental procedure p 15_23 27.02.03Sonification of the facial expressions

  16. Resources & Experimental Setup Data analysis: reaction time accuracy = error number p 16_23 27.02.03Sonification of the facial expressions

  17. Preliminary results Classification of the emotions by Ekman & Friesen (1975) NEUTRAL NEGATIVE POSITIVE anger sadness fear disgust surprise happiness http://www.paulekman.com/ p 17_23 27.02.03Sonification of the facial expressions

  18. Preliminary results Facial landmarks. Eye & eyebrow Inner brow raiser Lid tightener Outer brow raiser Lid droop Slit Brow lowerer Upper lid raiser Eyes closed Cheek raiser Squint p 18_23 27.02.03Sonification of the facial expressions

  19. Preliminary results Facial landmark. Mouth Lip suck Lip corner puller Cheek puffer Lip tightener Dimpler Lip pressor Lip corner depressor Lips part Lower lip depressor Jaw drop Lip puckerer Mouth stretch Lip stretcher Chin raiser Lip funneler p 19_23 27.02.03Sonification of the facial expressions

  20. Preliminary results Facial landmark. Nose Nose wrinkler Upperupper lip raiser Nasolabial deepener p 20_23 27.02.03Sonification of the facial expressions

  21. Prospectives • The sonification system of facial expressions which includes new sound mapping techniques to display facial features and expressions p 21_23 27.02.03Sonification of the facial expressions

  22. references • http://www.icad.org/websiteV2.0/Resources/glossary.html#S • http://www.tsbvi.edu/Education/vmi/tactile_symbols.htm#EMOTIONS • http://www.mcsdtechcenter.org/multimedia/sounds-samp.htm • http://www.matthawkins.co.uk/html/sound_fx_index.htm • http://kin-dza-dza.etel.ru/http://www.soundobject.org/ • http://viswiz.gmd.de/~eckel/publications/eckel98a/eckel98a.html • http://www-2.cs.cmu.edu/afs/cs/project/face/www/facs.htm • http://www.paulekman.com/ • http://www.ifeelpixel.com/download/whatsnew.htm • http://www.symbols.net/emoticons.htm • http://www.fingertek.com/smilies.htm • http://www-unix.mcs.anl.gov/~kaper/Sonification/ • http://www-pablo.cs.uiuc.edu/Project/Pablo/PabloSonification.htm p 22_23 27.02.03Sonification of the facial expressions

  23. Thank you for yourattention! 27.02.03Sonification of the facial expressions

More Related