Mining Discriminative Components With Low-Rank and Sparsity Constraints for Face Recognition

1. Mining Discriminative Components With Low-Rank andSparsity Constraints for Face Recognition Qiang Zhang, Baoxin Li Computer Science and Engineering Arizona State University Tempe, AZ, 85281 qzhang53, baoxin.li@asu.edu

2. Problem Description • In many applications, we may acquire multiple copies of signals from the same source (an ensemble of signals); • Signals in ensemble may be very similar (sharing a common source), but may also have very distinctive differences (e.g., very different acquisition conditions) plus other unique but small variations (e.g., sensor noise).

3. Examples Common sources

4. Examples Different acquisition conditions

5. Examples Sensor noises

6. Examples Signals in ensemble

7. Signals in Ensemble • The decomposition of the signal has several benefits: • Obtaining better compression rate. E.g., distributed compressed sensing [Duarte], joint sparsity model [Duarte 2005]; • Extract more relevant features. E.g., A compressive sensing approach for expression-invariant face recognition [Nagesh 2009];

8. Example: Face Images • Given face images of same subjects

9. Example: Face Images • Can we identify a “clean” image for them?

10. Example: Face Images • And their illumination conditions?

11. Proposed Model • We represent each image of an ensemble as • the image of subject; • the image ensemble; • the common part for Subject ; • a low rank matrix; • a sparse matrix;

12. Solving the Decomposition • The proposed model can be formulated as: Subject to

13. Comparison with Other Models • Distributed Compressive Sensing; • , s.t., ; • The proposed method is related to DCS. However, in DCS, the innovation components are typically assumed to be sparse, which limits its application; • Instead, the proposed model allows the innovation components have more complex structure, e.g., low rank. • Robust Principle Component Analysis;

14. Comparison with Other Models • Distributed Compressive Sensing; • Robust Principle Component Analysis; • , subject to ; • RPCA decompose a set of images into a low rank matrix and sparse matrix; • The core differences of RPCA from the proposed method is that, RPCA represents each image as a vector of a big matrix. Those vectors are linear dependent, in addition to some sparse noise;

15. Decomposition Algorithms • We apply augmented Lagrange Multiplier to the proposed formulations: • and are parameters.

16. Decomposition Algorithms Cont’d • We use augmented Lagrange Multiplier and block coordinate descent to solve the proposed problem: • Solve for each , while and are fixed; • Solve for each , while and are fixed; • Solve for each , while and are fixed; • Update and as ; • Check the convergence. If not converged, repeat the previous steps.

17. Experiment • We use three experiments to evaluate the proposed model and algorithm: • Decomposing the synthetic images; • Decomposing the images from extended YaleB dataset; • Applying decomposed component to classification tasks;

18. Decomposing the synthetic images We create training images by mixing the images with low-rank background images. In addition, we add some sparse- supported noise.

19. Decomposing the synthetic images • The decomposition result. From top to bottom: common components, low-rank components and sparse components.

20. Decomposition: Robustness over Missing Training Instances • We randomly remove 20% training images and test the robustness of decomposition algorithm. • From top to bottom: training images and common component, low-rank component.

21. Decomposing Extended YaleB Dataset • We use all 2432 images of extended YaleB dataset, which includes 38 subjects and 64 illumination conditions; • Common components capture information unique to certain subjects; • Low rank components capture the illumination conditions of the images; • Sparse components capture the sparse-supported noises and shadows;

22. Decomposing Extended YaleB Dataset Left: common components; Right: the low-rank components.

23. Applying Decomposed Component to Face Recognition • According to previous discussions, common components captures the information essential to the subject and low rank components captures the variation of the images sets; • Ideally a image from Subject should lie in a subspace spanned by its common component and the low-rank components .

24. Face Recognition: Subspaces Reconstruct image with common component of Subject 1 and low-rank components: (a) coefficient of the reconstruction, (b) the input image, and (c) the reconstructed image. Subject 1 Subject 2 Reconstruction with incorrect common components

25. Face Recognition: Measuring the Distance of Subspaces • We use principle angles to measure the distance between two subspaces: • Subspace of Subject • Subspace of test image • Principle angle provides information of relative position of subspaces in Euclidean space.

26. Face Recognition: Experiment • We test the algorithm on extended YaleB dataset and Multi-PIE dataset; • Randomly split the set into training sets and testing sets; • To test its robustness over missing training instances, we randomly remove some of the training instances and keep “# train per subject” training instances for each subject; • The performance is compared with SRC, Volterraface and SUN.

27. Face Recognition: Robustness over Poses Variations • we use all the images from 5 near frontal poses (C05, C07, C09, C27, C29), which includes153 conditions for each subject. We randomly pick M=40 illumination conditions for training and the remaining for testing.

28. Variation Recognition • The proposed method can also be used to identify the variations (e.g., illumination conditions) of the images; • For this purpose, we construct two subspaces and measure the distances with principle angle: • Subspace of variation • Subspace of test image

29. Variation Recognition We test the proposed algorithm on AR dataset, which contains 100 subjects and 2 sessions, where each session 13 variations. We use the first session for training and second session for testing. We show the confusion matrix, which presents the result in percentages. variations

30. Conclusions • We proposed a novel decomposition of a set of face images of multiple subjects, each with multiple images; • It facilitates explicit modeling of typical challenges in face recognition, such as illumination conditions and large occlusion; • For future work, we plan to expand the current algorithm by incorporating another step that attempts to estimate a mapping matrix for assigning a condition label to each image, during the optimization iteration.