1 / 23

Facial Expressions in Portuguese Sign Language Videos

Facial Expressions in Portuguese Sign Language Videos. André Maia . Index. Motivation Why Facial Expressions? Problem Characterization The Process: Facial Expressions Analysis Face Databases Face Detection Feature Extraction Classification Work plan. Facial Expressions Analysis.

kail
Download Presentation

Facial Expressions in Portuguese Sign Language Videos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Facial Expressions in Portuguese Sign Language Videos André Maia

  2. Index • Motivation • Why Facial Expressions? • Problem Characterization • The Process: • Facial Expressions Analysis • Face Databases • Face Detection • Feature Extraction • Classification • Work plan Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 1

  3. Motivation • How can weallcommunicatethesameway? • How can wefind a solution for thosewhodon’tknowsignlanguage? Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 2

  4. Why Facial Expressions? • Facial and/or body expressivity is fundamental in any sign language. • It is necessary to demonstrate the feeling to the given gesture and sometimes affecting its meaning Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 3

  5. Why Facial Expressions? • Can mean interrogation, exclamation, denial, statement, doubt, shame, fear, sadness, etc.… • There are gestures that can only be distinguished by the facial expression or the vocal sound that accompanies Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 4

  6. Problem Characterization • Given a recognition problem, how can we identify the expression? • How can we compare it? Are we correct? If not, how close are we? • How can one agent (algorithm) try to understand its result? Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 5

  7. Problem Characterization • Facial Expressions Analysis • Face Databases • Face Detection • Feature Extraction • Classification Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 6

  8. The Process – Facial Expressions Analysis • Facial Action Coding System • Prototypic Emotional Expressions Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 7

  9. The Process – Facial Expressions Analysis • Facial Action Coding System • Describes people’s expressions • in terms of action units • Does not include any inferential • labels • Prototypic Emotional Expressions Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 8

  10. The Process – Facial Expressions Analysis • Facial Action Coding System • Prototypic Emotional Expressions • Majority of facial expression recognition system attempt to recognize a small set of emotions • 1 - Disgust, 2 - Fear, 3 - Joy • 4 - Surprise, 5 - Sadness, • 6 – Anger Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 9

  11. The Process – Face Databases • Large number of face databases have been collected Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 10

  12. The Process – Face Databases • It is essential to have test data to check new approaches and to compare them with previous systems Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 11

  13. The Process – Face Databases • Labellingthe media. • Different human coders means different labelling • Reduce subjectivity, using coding systems (FACS) Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 12

  14. The Process – Face Detection • Face detection is still an ongoing area • It is the first step in face recognition • Skin Color, motion, • facial/head shape, facial appearance • Most successful are based • on appearance Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 13

  15. The Process – Face Detection • Eigenface and Template Matching • Viola’s Framework Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 14

  16. The Process – Feature Extraction • Still Image based methods • For both images and videos • Video frames considered independently • Video based methods • Only for video • Motion information considered Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 15

  17. The Process – Feature Extraction • Still Image based methods • Facial feature as graph deformation • Furrow presence detection • Comparison with reference face image • Video based methods Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 16

  18. The Process – Feature Extraction • Still Image based methods • Recognize facial features: • Color information • Edge information • Shape information • Recognize furrows: • Edge information • Texture information • Video based methods Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 17

  19. The Process – Feature Extraction • Still Image based methods • Video based methods • Motion analysis: Optical flow, tracking algorithms • (Kalman, Condensation,…) • Active Apearence Models (AAM) Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 18

  20. The Process – Feature Extraction Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 19

  21. The Process – Classification • Wide range of classifiers used in automatic expression recognition • Bayesian, thresholding the Normalized Euclidian distance, Fisher discrimination analysis, Local Linear Embedding, Higher Order Singular Value Decomposition • Neural Network and Support Vector Machines Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 20

  22. Work Plan • Analyze and decide which Database to use • Cohn-Kanadedatabase, Japanese Female Facial Expression (JAFFE) database, MMI database, and CMU-PIE database • Selection of the facial expression recognition algorithm • Implementation of the face detection algorithms • Proposal of a new method Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 21

  23. Questions? Facial Expressions Analysis Why Facial Expressions? Problem Characterization Face Databases Face Detection Feature Extraction Work Plan Classification Index Motivation 22

More Related