Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author : X Tan, S Chen, ZH Zhou, F Zhang

1. Recognizing Partially Occluded, Expression Variant Faces from Single Training Image per Person with SOM and soft kNN Ensemble Advisor : Dr. Hsu Presenter : Jia-Hao Yang Author :X Tan, S Chen, ZH Zhou, F Zhang

2. Guide • Motivation • Object • Architecture • Introduction • The propose method • Experiments • Conclusion • Opinion

3. Motivation • In many real-world applications only one training image per person is available. • The test images may be partially occluded or may vary in expressions.

4. Object • This paper using the SOM to learn the subspace that represented each individual. • And then it uses a soft k nearest neighbor (soft k-NN) ensemble method to identify the unlabelled subjects.

5. Architecture • Although template-based methods have become one of the main techniques, a large training data set is not always possible in many real world tasks. • Beside above problem, there exist other problems, such as occlusion and expression.

6. Architecture (cont.) • This paper extends Martinez’s work using SOM and soft kNN and then it achieves high performance. • The procedure is as follows: • Localization • The use of SOM • The Single SOM-face Strategy • The Multiple SOM-face Strategy • Identification

7. Architecture (cont.) • Finally, this paper have conducted various experiments to verify the performance of the proposed method.

8. Introduction • Face Recognition Technology (FRT) has a variety of potential applications in many aspect.

9. Introduction (cont.) • However, the general face recognition problem is still unsolved due to its inherent complexity. • To overcome this problem is to Search one or more face subspaces of the face to lower the influence of the variations.

10. Introduction (cont.) • Most template-based FRT assume that multiple images per person are available for training. • But a large training data set is not always possible in many real world tasks.

11. The Proposed Method • A. Localizing the face image: • the original image is divided into M(=l/d) sub-blocks with equal size, where l and d are the dimensionalities of the whole image and each sub-block. Image Localization Images

12. SOM Projection Images Image Localization Results Soft kNN Ensemble Decision The Proposed Method (cont.) • B. The use of SOM • The SOM is chosen for several reasons as follows: • It is efficient and suitable for high dimensional process • Its algorithm is more robust to initialization than any other • The trained SOM map are similar to input sub-blocks.

13. The Proposed Method (cont.) • The Single SOM-face Strategy • Step1: according to: Partition all the sub-blocks into Voronoi regions • Setp2: average : • Setp3: Smooth : • The multiple SOM-face Strategy • new image be presented to the system, denoted as • Then a separate small SOM map for the face will be trained using the above SOM algorithm.

14. The Proposed Method (cont.) • C. Identification • Given C classes, to decide which class the test face x belongs to, we first divide the test face into M sub-blocks. • and then project those sub-blocks onto the trained SOM maps. • Arranging it in increasing order : • normalization : • Finally, the label can be obtained :

15. Experiments • On the AR database (variations in Facial Expressions) • the neutral expressions images of the 100 individuals were used for training, while the smile, anger and scream images were used for testing.

16. Experiments (cont.)

17. Experiments (cont.)

18. Experiments (cont.) • On the AR database (variations in partially occluded) • Simulated occlusion • The number of the training data is same, while the smiling, angry and screaming images with simulated partial occlusions were used for testing.

20. Experiments (cont.) • We can find that half face occlusion does not harm the performance except the occlusion of upper face (see Fig.8b). • Because the lower half, included the mouth and cheeks, which can be easily affected by most facial expression variation.

21. Experiments (cont.) • Real occlusion • the neutral expression images of the 100 individuals were used for training, while the occluded images were used for testing.

22. Experiments (cont.) • It is interesting to note that the occlusion of the eyes area led to better recognition results because the scarf occluded each face irregularly.

23. Experiments (cont.) • To simulate the occlusion, we randomly localized a square of size pxp (5<p<50) pixels in each of the four testing image.

24. Experiments (cont.) • On the FERET database • Experiment 1 • the performance of the two SOM-face based algorithms on the subset was evaluated and was compared with other two method’s.

25. Experiments (cont.) • Experiment 2 • choosing an appreciate k-value for the soft k-NN classifier.

26. Experiments (cont.) • Experiment 3 • The effect of different sub-block sizes is studied.

27. Experiments (cont.) • Experiment 4 • To investigate the incremental learning capability of the MSOM strategy, experiment was conducted using different gallery sizes.

28. Experiments (cont.) • Experiment 5 • we repeated one of the simulated occlusion experiments done on the AR dataset .

29. Conclusion • This paper introduce the “SOM-face” to address the problem of face recognition with one training image per person and has several advantages over some of the previous methods. • It attributes these advantages to the seamless connection between the three parts of the method. Image SOM

30. Conclusion (cont.) • But the proposed method assumes that occluded is known in advance. • This paper shows that this paradigm works well in the scenario of face recognition with one training image per person.

31. Opinion • Advantage