Automatic Face Recognition under Component-Based Manifolds

1. Automatic Face Recognition under Component-Based Manifolds CVGIP 2006 Wen-Sheng Chu (朱文生) and Jenn-Jier James Lien (連震杰) Robotics Lab. CSIE NCKU

2. Motivation • Face recognition is hard due to several image variations: expression pose illumination

3. Person A Person A Objective • Recognize faces using multiple face patterns rather than a single one. Person B Person B Single input pattern Multiple input patterns

4. Feature Point Detection Automatic Acquisition of Facial Components Training Data of Features Rejected Non-face 2-Class SVM Classifiers Detected Features Original Image Cropped Face I Face Detection Face +ve Removal Facial Components Extraction Registration byAffine Warping Band-pass Filtering Normalized Pose IR Normalized Illumination IB Extracted Facial Components P. Viola and M. Jones, “Robust Real-Time Face Detection”, IJCV 2004.

5. Feature Point Detection Automatic Acquisition of Facial Components Training Data of Features Rejected Non-face 2-Class SVM Classifiers Detected Features Original Image Cropped Face I Face Detection Face +ve Removal Facial Components Extraction Registration byAffine Warping Band-pass Filtering Normalized Pose IR Normalized Illumination IB Extracted Facial Components

6. x x x Facial Feature Detector • 2-class SVM with feature vector v: • Reject false positives o o o o

7. Feature Point Detection Automatic Acquisition of Facial Components Training Data of Features Rejected Non-face 2-Class SVM Classifiers Detected Features Original Image Cropped Face I Face Detection Face +ve Removal Facial Components Extraction Registration byAffine Warping Band-pass Filtering Normalized Pose IR Normalized Illumination IB Extracted Facial Components

8. Registration & Illumination Normalization Registration Affine warping Band-pass filtering Illumination Normalization

9. Feature Point Detection Automatic Acquisition of Facial Components Training Data of Features Rejected Non-face 2-Class SVM Classifiers Detected Features Original Image Cropped Face I Face Detection Face +ve Removal Facial Components Extraction Registration byAffine Warping Band-pass Filtering Normalized Pose IR Normalized Illumination IB Extracted Facial Components

10. Facial Components Extraction • Effects of pose and illumination are smaller in each local region compared with those in the holistic face image. T. K. Kim, H. Kim, W. Hwang and J. Kittler, “Independent Component Analysis in A Local Facial Residue Space for Face Recognition”, PR, 2004.

11. Constrained Mutual Subspace Method (CMSM) • Similarity between i and j == θc • Use the variation of dissimilarity between subjects subspace j subspace i θ project project constrainedsubspace θc ic jc K. Fukui and O. Yamaguchi, “Face Recognition Using Multi-viewpoint Patterns for Robot Vision”, ISRR 2003.

12. PCA basis Constrained Subspace Generation • Take nose for explanation: The eigenvectors, w, selected in ascending order, are the basis of the constrained subspace, Snose. Constraint subspace basis

13. Projection onto Constrained Subspace • Projection basis vectors  constrained subspace Snose • Normalization length(projected vector)  1 • Orthogonalization applying Gram-Schmidt process to orthogonalize the normalized vectors Snose

14. Comparison between Normalized Manifolds • The similarity of nose between subject i and subject j: where are defined as the eigenvalues of matrix . • Similarity(i, j) == summing up the five canonical correlations

15. Experiment Setup

16. Typical Samples in 3D Principal Component Space – Holistic Image subject 1 (․) subject 2 (․) subject 3 (․) subject 4 (․)

17. Original v.s. Projected Subspaces – Eye-braw

18. Original v.s. Projected Subspaces – Left Eye

19. Original v.s. Projected Subspaces – Right Eye

20. Original v.s. Projected Subspaces – Nose

21. Original v.s. Projected Subspaces – Mouth

22. Comparison

23. End F&Q and thanks!