1 / 11

Human Capabilities on Video-based Facial Expression Recognition

Human Capabilities on Video-based Facial Expression Recognition. Motivation. Facial Expression Recognition goal: human-like man-machine communication six universal facial expressions [Ekman]: anger, disgust, fear, happiness, sadness, surprise minimal muscle activity

harlan
Download Presentation

Human Capabilities on Video-based Facial Expression Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Human Capabilities on Video-based Facial Expression Recognition

  2. Motivation • Facial Expression Recognition • goal: human-like man-machine communication • six universal facial expressions [Ekman]:anger, disgust, fear, happiness, sadness, surprise • minimal muscle activity -> reliable recognition is difficult • recognition rate of state-of-the-art approaches: ~ 70% • Question • How reliable do humans specify facial expressions? -> survey to determine human capabilities Technische Universität München Ursula Zucker

  3. The Facial Expression Database Cohn-Kanade AU-Coded Facial Expression Database • 488 image sequences (containing 4 up to 66 images) • each showing one of the six universal facial expressions • no natural facial expressions (simulated ground truth) • no context information Technische Universität München Ursula Zucker

  4. Description of Our Survey • Execution of the Survey • participants are shown randomly selected sequences • 250 participants • 5413 annotations -> approx. 11 per sequence Technische Universität München Ursula Zucker

  5. Evaluation • Evaluation of the Survey • no ground truth -> comparison of the annotations to one another • annotation rate for each sequence and each facial expression • relative agreement for an expression • confusion between facial expressions • Comparison to algorithms • recognition rate Technische Universität München Ursula Zucker

  6. Annotation Rate for Each Sequence • Explanation: • 488 rows • 1 row = 1 sequence • darker regions denote a higher annotation rate • sorted by similar annotation • Result: • happiness:best annotation rates • surprise and fear: get confused often • fear: difficult to tell apart Technische Universität München Ursula Zucker

  7. Relative Agreement • Explanation: • example: annotating the sequences as happiness ~ 350 sequences annotated as happiness by nobody, ~ 50 sequences annotated as happiness by everybody • well-recognized facial expressions have peaks at “0” and at “1” Technische Universität München Ursula Zucker

  8. Confusion Between Facial Expressions • fear and surprise: high confusion • happiness and disgust: low confusion confusion rate Technische Universität München Ursula Zucker

  9. Comparison: humans vs. algorithms • ground truth: provided by Michel et. al. • Results: • Michel et. al.: worse at recognizing anger • Schweiger et. al.: worse at recognizing disgust, fear, happiness and on the average Technische Universität München Ursula Zucker

  10. Conclusion • Survey applies similar assumptions as algorithms: • consideration of visual information only • no context information • no natural facial expressions • Summary of our results: • poor recognition rate of humans – worse than expected • some facial expressions get confused easily • Conclusion & Outlook: • integration of more sources of information is highly recommended, e. g. audio/language, context, ... Technische Universität München Ursula Zucker

  11. Thank you Technische Universität München Ursula Zucker

More Related