FACIAL EMOTION RECOGNITION BY ADAPTIVE PROCESSING OF TREE STRUCTURES

1. FACIAL EMOTION RECOGNITION BY ADAPTIVE PROCESSING OF TREE STRUCTURES presented by Jia-Jun Wong and Siu-Yeung ChoForensic and Security LabSchool of Computer EngineeringNanyang Technological UniversitySingapore

2. Abstract • Introduction • What are the basic emotions? • Existing systems? • Facial Emotion Tree Structures • What are Tree Structures? • Why use Tree Structures? • How to process Tree Structures? • Performance Evaluation • Under perfect feature location situations • With missing features • Conclusions

3. ANGER SAD FEAR NEUTRAL SURPRISE HAPPY DISGUST Introductions • There are six basic emotions according to psychologists • Emotions are innate and universal • Facial Emotions are revealed faster • FACS by Paul Ekman • 100 hours to train • Computerize methods • Surface texture analysiswith PCA • Facial Motion through optic flow • ICA, etc.

4. Feature Extraction Emotion Recognition Localised Gabor Filter Feature Extraction Gabor Features Image Processing Eyes Detection Facial Emotion Tree Face Cropping Feature Nose Detection Structure and Resizing Locations Transformation Mouth Detection Face Detection Probabilistic Based Facial Emotion Recognised Capture Image Recursive Neural Tree Structure Emotion State Network Representation Facial Emotion Recognition System

5. A C Scene a c B b d f e D Ground House Sky C A B d e a b c f What are Tree Structures? • Traditionally features are stored and used in a flat vector format • Simple to implement and use • This loses feature to feature relationship information • Flat feature vector can be transformed into tree structures • Encodes feature to feature relationship information • More flexibility in recognition

6. L1 L2 L3 L4 L5 Face Emotion Tree Structure (FEETS)

7. F00 F18 F09 F02 F01 Neural Node F01 Neural Node F09 Neural Node Neural Node F00 F02 Neural Node F18 Adaptive Processing of Tree Structures • Step 1: Encode Data into Tree Structure • Step 2: Feed each node into a interconnecting Neural Node

8. Output, y Tree Node Output layer GMM_1 GMM_G Hidden layer Input layer Input attributes, u Children’s output, y Adaptive Processing of Tree Structures • Maximum number of children for a node, which is the branch factor is assumed for a task.

9. Probabilistic Recursive Model • Class likelihood function • Unsupervised Learning • Expectation Step • Maximisation Step • Supervised Learning • Levenberg Marquardt Algorithm

10. Features Used • Localized Gabor Features • Biological relevance and computational properties. • Captures the properties of • spatial localization, • quadrature phase relationship.

11. Feature Locations • Four primary feature locations • the center of the left eye, • center of the right eye, • tip of the nose, • the center of the lips. • 60 Extended Features

12. Performance Evaluations • Database used • Japanese Female Facial Expression (JAFFE) Database • 213 images of 7 facial expressions (including neutral) posed by 10 Japanese female models

13. FEETS vs Quadtree • FEETS are smaller in size • Higher recognition rate

14. FEETS vs Others

15. Missing Features Performance • Perfect Condition • Eyes Missing • Nose & MouthMisssing

16. FEETS vs Others • Database Used CMU Emotion Database

17. FEETS vs Others

18. Conclusions • FacE Emotion Tree Structures (FEETS) has achieve high recognition rates • Robust recognition when there are missing features • Smaller footprint than Quadtrees

19. Thank You