Face Recognition: A General Talk Emphasized on Still Images

1. Face Recognition: A General Talk Emphasized on Still Images Yang Shi

2. Topics Covered Background and Application Some psychophysics/neuroscience topics in this area Recognition on still images Recognition from image sequences Evaluation on face recognition systems Summary and conclusions

3. Background Strong need for Commercial and Law enforcement applications Not rely on cooperations from the users for security applications Also techniques are more available for face recognition

4. Typical Applications (Chart)

5. Some Commercial Face Recognition Systems (Chart)

6. Generic Face Recognition System

7. Some psychophysics/neuroscience topics in this area Is this process a dedicated process? Is this a result of holistic or feature analysis? Ranking of importance of facial features Viewpoint invariant recognition? Effect of illumiation Movement and Facial Expression

8. Face Recognition from Still Images 3 key tasks for normal recognition problem: • Detection and rough normalization • Feature extraction and accurate normalization • Identification/verification

9. Face Detection and Feature Extraction Segmentation/Detection: Approaches: Template, Skin color, NN Detection: 2 statictics important: True positives and false positives (Train system with false-positives generated by previous trained systems)

10. Face Detection and Feature Extraction Feature Extraction: Based on Lines, edges, curves template based structural

11. Recognition from Intensity Images • Holistic matching methods • Feature based (structual) matching methods • Hybrid

13. Holistic Approaches PCA: Principal Component Analysis Advantages: Reduced redundancy and noise Let’s get some basic knowledges about the well-known method in face recognition field.

14. PCA in Statistic • Normalize a Sample Matrix • Get the covariance matrix of the normalized matrix R = Z.

15. PCA in Statistic 3. Eigendecomposition the covariance matrix, get the Eigenvector and Eigenvalue U: Eigenvalues D: Eigenvectors 4. The Eigenvectors with the highest eigenvalue are selected as the principal components, and project on the matrix of eigenvector 5. Only use the eigenvectors with higher eigenvalues.

16. PCA in image processing • Standarize (subtract means) • Get covariance • Eigendecomposition • Projection on selected Eigenvectors • classification

17. Why covariance and why eigendecomposition • We can get the variance in the covariance matrix, but we still get something else, that is the covariances, how to do with it? • Diagnolization: eigendecomposition • pure variace, no relations between features.

18. An example for PCA Shown on board Regard C as the energy of each vectors, and the diagonal is variance, the bigger the more information. Usually R is not diagonalizable.

19. Ways of identification Given the eigenvectors (Called eigenfaces), get the vector of weights by a inner product operation. • Eucleadian • Probabilistic based method

20. Some other holistic algorithms • linear/Fisher discriminant analysis • SVM • ICA etc...

21. Others • Some early progresses, geometry of local features. • Elastic Bunch Graph Matching (Using Gabor wavelet) And Hybrid methods

22. Face Recognition based on Auto cropping and Feature Points Extraction • Segmentation • Background suppression and auto cropping • Feature detection and T-shape face • Classify

23. Segmentation • Basic idea: Find pixels with similar color and make them in a set.

24. Segmentation • However, I think that it needs a limitation, distance. • All components are getting close to each other, the search needs to be done in a sequence. • Any thoughts here?

25. Background Suppression • First two biggest set of segmentation must be skin and background. • Whichever more close to the border of the picture is background.

26. Auto Cropping • Find A, B, C, D • A: First skin pixel ↓ • B: First skin pixel → • C: First skin pixel ← • D: middle of P1, P2 • P1, P2: ↑ first pixel and left/right is background

27. Algorithm

28. Formation of T-shaped Face • Make all skin portion gray value = 0, others 1 • Find this pair of points (getting eyes): - 1. upper half - 2. difference of numbers of pixels smaller than a threshold - 3. apart by horizontal distance - 4. more or less same vertical position

29. Nose tip and mouth and T-shape • Middle of nostrils, detection of nostrils is almost the same as eyes. • Mouth: below nostrils and getting biggest area. • 2 limbs and vertical nose and mouth.

30. Classification • First resize the T-shaped face size equal. • PCA

31. Noise components • Extra unimportant segments: • See which part and judge by previous algorithm Or revised version Personally I think the algorithm still needs to be more robust.

32. Experiment • Nice detection rate. • Slight improvement on PCA-based recognition methods.

33. Summary • Earlier mathods treat it as a standard pattern recognition problem, later specialized. • For holistic methods, face image can be really small, but for others, not so. And slightly better performance can be seen if the image is small on holistic. • Accurate location is critical for feature based methods.(Normalization) • Each methods has it’s own advantages.

34. Face recognition from image sequences Issues: • Quality of videos is low • Face images are small • Charisteristics of human or face

35. Categorization of Video based Face Rec

36. Evaluation of Face Recognition Systems • The FERET Protocol • The XM2VTS Protocol

37. The FERET Protocol • Database: the only large database generally available. • Neutral (fa) and different (fb) facial expression for 200 individials, then another set for different light and camera (fc). • fa:1196 probes, then at SEp 1996, as a core used to measure these four groups of probe sets:

38. Contributions for FERET • Make it possible to evaluate the progress of algorithms’ development as quantified numbers. • Revealed 3 areas of research: Illumination, pose and recognizing duplicates.

39. The XM2VTS Protocol • Multimodel database contains 4 recordings of 295 subjects in 4 months, each recording contains a speaking headshot and a rotating headshot. • high-quality color images, sound files, video sequences and a 3D model. • http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb/. • A broad range of application

40. Internet Resources and Databases

41. Conclusions • Active area spanning disciplines: image processing, pattern recognition, computer vision, machine learning. • It’s benifitial to follow some phychophysics related documents, but not necessary. • Neumerous methods, tendency of using machine learning methods. • From a sequence is still a challenging problem • 2 benchmarking protocols: FERET and XM2VTS • Robust recognition is still difficult • Is it possible for a machine to build a system mimic to human perception system, thus no need to worry about limitation numbers?

42. References • Turk M A, Pentland A P. Face recognition using eigenfaces[C]//Computer Vision and Pattern Recognition, 1991. Proceedings CVPR'91., IEEE Computer Society Conference on. IEEE, 1991: 586-591. • Zhao W, Chellappa R, Phillips P J, et al. Face recognition: A literature survey[J]. ACM computing surveys (CSUR), 2003, 35(4): 399-458. • Karmakar D, Murthy C A. Face Recognition using Face-Autocropping and Facial Feature Points Extraction[C]//Proceedings of the 2nd International Conference on Perception and Machine Intelligence. ACM, 2015: 116-122. • http://blog.csdn.net/babywong/article/details/50085239 • https://en.wikipedia.org/wiki/Eigendecomposition_of_a_matrix • https://en.wikipedia.org/wiki/Diagonalizable_matrix • http://blog.csdn.net/yutianzuijin/article/details/10823985 • https://www.zhihu.com/question/21874816 • https://en.wikipedia.org/wiki/Principal_component_analysis Thank you!