Face Recognition Under Varying Illumination

1. Face Recognition Under Varying Illumination Erald VUÇINI Vienna University of Technology Muhittin GÖKMEN Istanbul Technical University Eduard GRÖLLER Vienna University of Technology

2. Database Image Capture Face Detection Feature Projection Face Identification Face Recognition System Erald Vuçini - Vienna University of Technology

3. Face Recognition Approaches • Principal Component Analysis (PCA) • Linear Discriminant Analysis (LDA) • Local Feature Analysis • Active Appearance Model • Hidden Markov Model • Support Vector Machine • … Erald Vuçini - Vienna University of Technology

4. Face Recognition – Problems • The variations of the same face due to • illumination • viewing direction • are almost always larger than image variations due to changes in the face identity Erald Vuçini - Vienna University of Technology

5. Handling Variable Illumination • Extract illumination invariant features • Transform variable illumination to canonical representation • Model 2D illumination variations • Utilize 3D face models whose shapes and albedos are obtained in advance Erald Vuçini - Vienna University of Technology

6. Outline of proposed approach • Dimensionality Reduction - LDA better than PCA regarding illumination • Image Synthesis - Solve the Small Sample Size (SSS) problem • Reconstruction – Restore frontal illumination Erald Vuçini - Vienna University of Technology

7. Dimensionality Reduction • Dimensionality Reduction - LDA better than PCA regarding illumination • Image Synthesis - Solve the Small Sample Size (SSS) problem • Reconstruction – Restore frontal illumination Erald Vuçini - Vienna University of Technology

8. x2 x2 z1 z2 x1 x1 Principal Component Analysis (PCA) • One of the most commonly used methods in Face Recognition • Maximizes the scattering of all projected samples PCA Erald Vuçini - Vienna University of Technology

9. PCA under Varying Illumination • PCA fails with variant illumination • The scatter being maximized is due to • Between-class scatter • Within-class scatter • Discard 3 most significant principal components to reduce lighting variation Erald Vuçini - Vienna University of Technology

10. LDA Interpretation • LDA is a class specific method LDA Erald Vuçini - Vienna University of Technology

11. LDA Problems • LDA maximizes the ratio of Between-class scatter and Within-class Scatter • Within-class Scatter singularity problem • Fisher LDA (FLDA) removes Null Space • FLDA handles best the variation in lighting, having lower error rate than PCA Erald Vuçini - Vienna University of Technology

12. Image Synthesis • Dimensionality Reduction - LDA better than PCA regarding illumination • Image Synthesis - Solve the Small Sample Size (SSS) problem • Reconstruction – Restore frontal illumination Erald Vuçini - Vienna University of Technology

13. Image Synthesis - Motivation • Face Recognition Systems • Performance related with training database • LDA require many samples per class • In many systems only one image per person is provided • Quotient Image makes possible the synthesis of the image space of a given input image Erald Vuçini - Vienna University of Technology

14. Lambertian Objects - Faces • The image space lives in a 3D linear subspace • Three images are sufficient for generating the image space of the object Albedo Surface Normal Light Source Direction Erald Vuçini - Vienna University of Technology

15. Quotient Image (Definitions) • Ideal class of faces • Same shape • Different albedos • Synthesis Problem: • Given 3N images of N faces of the same class, illuminated under 3 lighting conditions • Synthesize image space of new input Erald Vuçini - Vienna University of Technology

16. Quotient Image (Definitions) • Given objects y and a we define quotient image Q by the ratio of their albedos • Q is illumination invariant • Image space of y can be generated with • Quotient Q • 3 images of a • Generalization: Use bootstrap of 3N images Erald Vuçini - Vienna University of Technology

17. Quotient Image - Examples Quotient Quotient Erald Vuçini - Vienna University of Technology

18. Quotient Image – Image Space Synthesis Erald Vuçini - Vienna University of Technology

19. 10 person Bootstrap 5 person Bootstrap 1 person Bootstrap Erald Vuçini - Vienna University of Technology

20. Reconstruction • Dimensionality Reduction - LDA better than PCA regarding illumination • Image Synthesis - Solve the Small Sample Size (SSS) problem • Reconstruction – Restore frontal illumination Erald Vuçini - Vienna University of Technology

21. YaleB Testing Database • Yaleb Database • 450 images of 10 persons • Divided in 4 subsets • Subset1 up to 10˚ • Subset2 up to 25˚ • Subset3 up to 45˚ • Subset4 up to 75˚ • Normalized Erald Vuçini - Vienna University of Technology

22. Histogram Equalization • Histogram equalization(HE) done as preprocessing increases the recognition rate • Adaptive HE(AHE) is used as a preprocessing step in the iterative face recognition approach Erald Vuçini - Vienna University of Technology

23. Illumination Restoration Approach • A face image with arbitrary illumination is restored to having frontal illumination. It has the following advantages: • No need to estimate face surface normals • No need to estimate light source directions and albedos • No need to perform image warping • Face images will be visually natural looking Erald Vuçini - Vienna University of Technology

24. Algorithm Outline • Compute mean face image and eigenspace • Compute initial restored images • Create iteration by replacing Bro with blurred Hio • Continue iteration until stopping criteria satisfied Restored Image Input Image Blurred Reference Image Blurred Input Image Erald Vuçini - Vienna University of Technology

25. Iteration Steps Erald Vuçini - Vienna University of Technology

26. Experimental Results (Subset 3) Restoration Erald Vuçini - Vienna University of Technology

27. Experimental Results (Subset 4) Restoration Erald Vuçini - Vienna University of Technology

28. Results with YaleB Database Erald Vuçini - Vienna University of Technology

29. Thank you for the attention! • Proposed Method • Dimension Reduction • Image Synthesis • Reconstruction Questions? Erald Vuçini - Vienna University of Technology