Seminar: CSE 717 Soft [1] Biometric Traits in Face Recognition System

1. Seminar: CSE 717Soft[1] Biometric Traits in Face Recognition System Problem Description Part Zhi Zhang zhizhang@cse.buffalo.edu 2/21/2004

2. Ideal Characteristics of Biometric Traits • Universality • Distinctiveness • Permanence • Collectability • Performance • Acceptability • Circumvention[2] Regretfully, NONE of the currently using human biometric traits possesses all of the above characteristics. Soft Biometric Traits in Face Recognition System - Zhi Zhang

3. What is Soft[1] Biometric Traits? Traditional (Primary) Biometric Traits[2]: • DNA Sequences • Iris/Retina • Fingerprint • Voice • Face • Signature The above human biometric traits are relatively universal, distinctive, permanent and resistant to circumvent. But they may not be collectable or acceptable to all the people. Soft Biometric Traits in Face Recognition System - Zhi Zhang

4. What is Soft[1] Biometric Traits? - Cont Soft[1] Biometric Traits: • Gender • Ethnicity • Eye/Skin/Hair color • Age • Height • Weight The above human biometric traits are relatively LESS distinctive, permanent and resistant to circumvent. But they provide some evidence about the user identity that could be exploited[1] Soft Biometric Traits in Face Recognition System - Zhi Zhang

5. Why using Soft Biometric Traits? During enrollment, many existing biometric systems actually collected information like: • Gender • Ethnicity • Eye/Skin/Hair color • Age • Height • Weight If the above traits can be automatically extracted and incorporated in the decision making process, the performance of the system can be improved significantly[1]. Soft Biometric Traits in Face Recognition System - Zhi Zhang

6. Necessary Devices • Image or Video Device As a special Face Recognition System, an image or video device is a must for both enrollment and verification/identification. As color is a relatively important characteristic for Soft Biometric Traits, the images collected from the image or video device must be color images. Soft Biometric Traits in Face Recognition System - Zhi Zhang

7. Necessary Devices - Cont • Auxiliary Devices - Optional For those Soft Biometric Traits that can not be extracted directly from the images, some auxiliary devices are needed. If Height trait is expected, an extra height sensor could be installed to extract this information. If Weight trait is expected, an hidden scale could be installed to extract this information. Soft Biometric Traits in Face Recognition System - Zhi Zhang

8. Difficulty Levels of the System • Verification vs. Identification • Controlled vs. Uncontrolled • Database • Location and Segmentation • Feature Definition • Feature Extraction • Feature Combination • Matching/Classification • Decision Making Soft Biometric Traits in Face Recognition System - Zhi Zhang

9. Verification vs. Identification • Verification System • 1-1 Matching • Commercially available[3] • Identification System • 1-n Matching • Still a challenge area Soft Biometric Traits in Face Recognition System - Zhi Zhang

10. Controlled vs. Uncontrolled • Controlled Environment: • Fixed pose • Simple background • Special/Fixed illumination • Uncontrolled Environment: • Free pose • Complex background • Different illumination Soft Biometric Traits in Face Recognition System - Zhi Zhang

11. Database Availability: • FERET[4] • Large, 14051 images • 8-bit greyscale images • Database from other universities or institutes[5] • Variable size • Color images • Not standard • Build our own image database Soft Biometric Traits in Face Recognition System - Zhi Zhang

12. Database - Cont • Selection of the images • Demographical Distribution • Gender Distribution • Age Distribution • Illumination Distribution - Optional • Pose Distribution - Optional • Management of Database • Indexing • Binning Soft Biometric Traits in Face Recognition System - Zhi Zhang

13. Location and Segmentation • Behavior-based Agent Model[6] • Search the skin-like pixels by a number of color-sensitive behavior-based agents, which distributed uniformly in the 2-D image • Mark the face-like region by activating the evolutionary behavior of the agents • Examine the shape information of each face candidate region and determine the face region by fuzzy shape feature analysis • Luminance/Chrominance-Component-based Approach[7] • Detect the face location by exploring the distribution property of the luminance and chrominance components Soft Biometric Traits in Face Recognition System - Zhi Zhang

14. Feature Definition • Feature definition in Traditional (Primary) Biometric Traits • Feature definition in Soft Biometric Traits Soft Biometric Traits in Face Recognition System - Zhi Zhang

15. Feature Definition in TBT[8] • Geometric feature-based method • Economical representation • Insensitivity to variations in illumination and viewpoint • Sensitive to the feature extraction process • Appearance-based method • Eigenfaces • Karhunen-Loeve (KL) Transform or Principal Component Analysis (PCA) • Most Expressive Features (MEFs) Soft Biometric Traits in Face Recognition System - Zhi Zhang

16. Feature Definition in SBT • Gender Classification Features[9] • Feature Selection • Different eigenvectors encode different kind of information • Some of the eigenvectors may be irrelevant to gender classification • Using a Genetic Algorithm (GA) to select a subset of the eigenvectors • Using the selected subset to train a Neural Network (NN), which could be applied to perform gender classification Soft Biometric Traits in Face Recognition System - Zhi Zhang

17. Feature Definition in SBT - Cont • Ethnic Classification Features • A mixture of experts consisting of ensembles of radial basis functions for the classification of gender, ethnic origin, and pose of human faces was proposed[10] • The above work was on FERET database, which means no color information was utilized • We could acquire the skin color information after face location and segmentation process • Feature Selection combined with skin color information, which could be an important feature in ethnic classification Soft Biometric Traits in Face Recognition System - Zhi Zhang

18. Feature Definition in SBT - Cont • Age Estimation Features • Relatively a new topic • A classifier was designed to accept the model-based representation of unseen images and produce an estimate of the age of the person in the image[11] • A wrinkle modeling was proposed and a research about age and gender estimation based on wrinkle texture and color of facial image was introduced[12] • We could see that both texture and color information could be applied to age estimation Soft Biometric Traits in Face Recognition System - Zhi Zhang

19. Feature Extraction • A kernel Principal Component Analysis (PCA) was proposed[13] for feature extraction • A nonlinear extension of PCA • First map the input data into a feature space via a nonlinear mapping, then apply PCA in the above feature space • Feature extraction for Soft Biometric Traits Soft Biometric Traits in Face Recognition System - Zhi Zhang

20. Feature Combination A face verification algorithm based on multiple feature combination and supporting vector machine was proposed[15]. It combines • eigenface • eigenUpper • eigenTzone • edge distribution These features are projected to a new intra-person/extra-person similarity space and are evaluated by a supporting vector machine supervisor Soft Biometric Traits in Face Recognition System - Zhi Zhang

21. Matching/Classification Various matching schemes: • Neural Networks (NN) • Deformable Models • Hidden Markov Models (HMM) • Support Vector Machines (SVM)[14] And a lot of hybrid schemes have been applied in this field Soft Biometric Traits in Face Recognition System - Zhi Zhang

22. Decision Making How to make a reasonable decision out of the following results: • Traditional BT classification result • Soft BT classification results: • gender • ethnic • eye/hair color • age • height • weight Soft Biometric Traits in Face Recognition System - Zhi Zhang

23. Decision Making - Cont Approaches could be used: • Decision Tree • Neural Network • Bayesian approach • Supporting vector machine Soft Biometric Traits in Face Recognition System - Zhi Zhang

24. Decision Making Module Primary Biometric System Face Templates Soft Biometric System Feature Extraction Module Matching Module Soft Biometric Processing Module Feature Extraction Module System Diagram Soft Biometric Traits in Face Recognition System - Zhi Zhang

25. References [1] Anil K. Jain, Sarat Dass and Karthik Nandakumar, “Soft Biometric Traits for Personal Recognition System”. [2] Anil K. Jain, Arun Ross and Salil Prabhakar, “An introduction to biometric Recognition”, IEEE Trans. on Circuits and Systems for Video Technology, Special Issue on Image- and Video-Based Biometrics, Vol. 14, No. 1, Jan. 2004. [3] P. J. Philips, P. Grother, R. J. Micheals, D. M. Blackburn, E. Tabassi, and J. M. Bone, “FRVT 2002: Overview and Summary”, March 2003, Available from: http://www.frvt.org/FRVT2002/documents.htm [4] “The Facial Recognition Technology (FERET) Database”, Available from: http://www.itl.nist.gov/iad/humanid/feret/feret_master.html [5] “Computer Vision Test Images”, Available from: http://www-2.cs.cmu.edu/~cil/v-images.html [6] Jiebo Luo, Chang Wen Chen, Parker, K.J., “Face location in wavelet-based video compression for high perceptual quality videoconferencing”, Circuits and Systems for Video Technology, IEEE Trans. on , Vol. 6 , No. 4 , Aug. 1996, pp 411 – 414. [7] Chai, D., Ngan, K.N., “Automatic Face Location for Videophone Images”, TENCON '96. Proceedings. 1996 IEEE TENCON. Digital Signal Processing Applications , Vol. 1 , 26-29 Nov. 1996, pp.137 - 140 vol.1 Soft Biometric Traits in Face Recognition System - Zhi Zhang

26. Reference - Cont [8] Dugelay, J.-L.; Junqua, J.-C.; Kotropoulos, C.; Kuhn, R.; Perronnin, F.; Pitas, I.; “Recent advances in biometric person authentication”, Acoustics, Speech, and Signal Processing, 2002. Proceedings. (ICASSP '02). IEEE International Conference on , Vol. 4, 13-17 May 2002, pp. IV-4060 - IV-4063 vol.4 [9] Zehang Sun; Xiaojing Yuan; Bebis, G.; Louis, S.J.; “Neural-network-based Gender Classification using Genetic Search for Eigen-feature Selection”, Neural Networks, 2002. IJCNN '02. Proceedings of the 2002 International Joint Conference on , Vol. 3 , 12-17 May 2002, pp. 2433 – 2438 [10] Gutta, S.; Huang, J.R.J.; Jonathon, P.; Wechsler, H.; “Mixture of experts for classification of gender, ethnic origin, and pose of human faces”, Neural Networks, IEEE Trans. on , Vol. 11 , Issue: 4 , July 2000, pp. 948 – 960 [11] Lanitis, A.; Draganova, C.; Christodoulou, C.; “Comparing Different Classifiers for Automatic Age Estimation”, Systems, Man and Cybernetics, Part B, IEEE Trans. on , Vol. 34 , Issue: 1 , Feb. 2004, pp. 621 – 628 [12] Hayashi, J.; Yasumoto, M.; Ito, H.; Koshimizu, H.; “Age and Gender Estimation based on Wrinkle Texture and Color of Facial Images”, Pattern Recognition, 2002. Proceedings. 16th International Conference on , Vol. 1 , 11-15 Aug. 2002, pp. 405 - 408 vol.1 [13] Kwang In Kim; Keechul Jung; Hang Joon Kim; “Face recognition using kernel principal component analysis”, Signal Processing Letters, IEEE , Vol. 9 , Issue: 2 , Feb. 2002, pp. 40 – 42 Soft Biometric Traits in Face Recognition System - Zhi Zhang

27. Reference - Cont [14] G. D. Guo, S. Z. Li, and K. L. Chan, “Face recognition by Support Vector Machines”, in Proc. Int. Conf. Automatic Face and Gesture Recognition, 2000, pp. 196-201. [15] Do-Hyung Kim; Jae-Yeon Lee; Jung Soh; Yun-Koo Chung; “Real-time face verification using multiple feature combination and a support vector machine supervisor”, Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03). 2003 IEEE International Conference on , Vol. 2 , 6-10 April 2003, pp. II - 353-6 vol.2 Soft Biometric Traits in Face Recognition System - Zhi Zhang