1 / 26

Berk G ö kberk Bo ğaziçi University – Perceptual Intelligence Lab Turkey

Principles of 3D Face Recognition. Berk G ö kberk Bo ğaziçi University – Perceptual Intelligence Lab Turkey. Prom ises and motivations. 2D face recognition still requires help Pose, expression, illumination variations Possible solutions: other modalities? Video, infra-red, stereo, 3D

kato
Download Presentation

Berk G ö kberk Bo ğaziçi University – Perceptual Intelligence Lab Turkey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Principles of 3D Face Recognition Berk Gökberk Boğaziçi University – Perceptual Intelligence Lab Turkey

  2. Promises and motivations • 2D face recognition still requires help • Pose, expression, illumination variations • Possible solutions: other modalities? • Video, infra-red, stereo, 3D • Promises of 3D facial recognition • High-security applications • 3D shape information invariance • Pose and illumination problems can be solved • Better facial feature localization

  3. Some scenarios • 3D-to-3D • 2D-to-2D via 3D • 2D-to-3D or 3D-to-2D

  4. How do we get 3D facial data? • Stereo cameras • Quality: low to medium • Speed: fast • Problems: reconstruction • Structured-lights • Quality: medium • Speed: fast • Problems: intrusive • Laser scanners • Quality: high • Speed: slow • Problems: intrusive, • Shape from {shading, motion, video} • N/A

  5. 3D Facial Recognition Pipeline Face Normalization/Alignment 3D Face Detection Pre-proc. Features Pattern Classifier Point Clouds Depth Images Cropping Coarse Alignment Noise Removal Hole Filling Smoothing Landmark Finding Fine Alignment

  6. 3D Face Detection • This problem has not been touched so far! • Simple heuristics such as nose tip • In complex scenes, curvature analysis is generally used Ref: 3D face detection using curvature analysis, Alessandro Colombo, Claudio Cusano, Raimondo Schettini, Pattern Recognition 39 (2006) 444 – 455

  7. Pre-processing • Artifact removal • Noise removal: spikes (filters), clutter (manually), noise (median filter) • Holes filling (Gaussian smoothing, linear interpolation, symmetrical interpolation)

  8. Face Normalization/Alignment • Coarse alignment by • Centre of mass, • Plane fitted to the data • Facial landmarks (eyes, nosetip) • Fine alignment • ICP • Warping • Elastic deformations

  9. 3D Facial Features

  10. -ICP gives the distance -Hausdorff -Too many points PC

  11. -ICP gives the distance -Hausdorff -Too many points PC SN -Enhanced Gaussian Image -Too many normals

  12. -ICP gives the distance -Hausdorff -Too many points PC SN -Enhanced Gaussian Image -Too many normals PRO -Sparse -Easy to compare -Not fully descriptive

  13. -ICP gives the distance -Hausdorff -Too many points PC SN -Enhanced Gaussian Image -Too many normals PRO CURV -Landmark detection -Segmentation -Sensitive to noise and the quality of data Principal directions -Sparse -Easy to compare -Not fully descriptive Mean Shape Index Gaussian

  14. -ICP gives the distance -Hausdorff -Too many points PC PCA DI SN -Benefit from 2D literature -Easy to fuse with texture -Applicable to 2.5D only -Enhanced Gaussian Image -Too many normals PRO CURV -Landmark detection -Segmentation -Sensitive to noise and the quality of data Principal directions -Sparse -Easy to compare -Not fully descriptive Mean Shape Index Gaussian

  15. PCA -ICP gives the distance -Hausdorff -Too many points TEX PC Gabor PCA DI SN -Benefit from 2D literature -Easy to fuse with texture -Applicable to 2.5D only -Enhanced Gaussian Image -Too many normals PRO CURV -Landmark detection -Segmentation -Sensitive to noise and the quality of data Principal directions -Sparse -Easy to compare -Not fully descriptive Mean Shape Index Gaussian

  16. Baseline algorithms • When you design a new system, which algorithms should be selected to show your algorithm’s superiority? Baseline Algorithm 1 PCA Match score for texture Texture & Shape Use weighted sum PCA Match score for shape Baseline Algorithm 2 Face A Perform ICP Output the surface matching error Face B

  17. What are the scenarios tested? • The discriminative power of texture and shape? Taken from: “Three-dimensional face recognition”, Bronstein, A.M., and Bronstein, M.M., and Kimmel, R. International Journal of Computer Vision 2005, Vol.64 No.1 p.5-30

  18. What are the scenarios tested? • The discriminative power of texture, shape, or texture+shape? • Pose variations • No disciplined analysis to compare 2D and 3D under pose variations • Expression variations • Most of the databases do not contain expression variations • No comparison to 2D • Illumination variations • Image relighting • Albedo estimation

  19. Open Issues & Challenges • Uncontrolled acquisition • Non-cooperative • Different lighting conditions • Texture map + shape map inconsistenies • Real-time 3D video data • Computational complexity • Issues related to performance assesment • Publicly available standard face databases • Quality (resolution) of the data? • Artifacts such as eyeglasses Images taken from: A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition, Kevin W. Bowyer, Kyong Chang, Patrick Flynn, Computer Vision and Image Understanding 101 (2006) 1–15

  20. Quick notes on 3D acquisition systems & 3D face databases

  21. 3D Acquisition Systems • Face specific • Biometrics • A4Vision • Geometrix • Modeling • Cyberware • Genex • Inspeck • Medeim • Breuckmann • General sensors • Minolta V-910

  22. 3D Face Databases • UND • 275 subjects, 943 scans • Shape + texture • FRGC • 400 subjects, 4007 scans • Shape + texture • 3D_RMA • 120 subject, 6 scans • Shape only • GavabDB • 61 subjects (9 scans) • Shape only • Pose, expression variations • USF database • 357 scans • 3DFS generator • Custom face databases • 12 persons to ~6000 persons (A4Vision) UND USF GavabDB 3DFS

  23. Conclusions • 3D face recognition systems were proposed to overcome expression, illumination, and pose challenges • Illumination correction is simpler • Facial landmark localization is better • Baseline recognizers • Combining depth maps with texture channel at the decision level • The core algorithm, ICP, has limited capabilities • Not suitable for non-rigid deformations • Lots of research on shape channel representation • Few in combining shape + texture

  24. References • Boğaziçi University • Perceptual Intelligence Lab (PILAB) • Signal and Image Processing Lab (BUSIM) • Relevant Papers: • Comparative analysis of decision-fusion methods for 3D face recognitionB. Gökberk, L. Akarun, submitted for publication. • Exact 2D-3D Facial Landmarking for Registration and RecognitionSalah, A.A., H. Çınar, L. Akarun, B. Sankur, submitted for publication. • 3D Shape-based Face Representation and Feature Extraction for Face RecognitionB. Gökberk, M. O. İrfanoğlu, L. Akarun, Image and Vision Computing (accepted). • 3D Face Recognition by Projection Based MethodsH. Dutağacı, B. Sankur, Y. Yemez, in SPIE Conf. on Electronic Imaging, 2006. • 2D/3D facial feature extractionÇınar Akakın, H., A.A. Salah, L. Akarun, B. Sankur, in SPIE Conf. on Electronic Imaging, 2006. • Selection and Extraction of Patch Descriptors for 3D Face RecognitionB. Gökberk, L. Akarun, ISCIS 2005, LNCS, Vol. 3733 Springer 2005, pp. 718-727. • 3D Face Recognition for Biometric ApplicationsL. Akarun, B. Gökberk, A. A. Salah, EUSIPCO2005, Antalya, Turkey. • Rank-based Decision Fusion for 3D Shape-based Face RecognitionB. Gökberk, A. A. Salah, L. Akarun, AVBPA 2005, LNCS, Vol.3546 p.1019-1028. • 3D Shape-based Face Recognition using Automatically Registered Facial SurfacesM. O. İrfanoğlu, B. Gökberk, L. Akarun, ICPR2004, pp.183-186 • Contact: • Berk Gökberk, e-mail: gokberk@boun.edu.tr

  25. Additional Material

  26. Face Recognition Grand Challenge 3D Shape+Texture 3D Texture Only High Resolution 2D vs 2D 3D Shape Only Reference: Jonathon Phillips, FRGC Workshop, CVPR’05

More Related