Active Appearance Models for Face Detection

1. Active AppearanceModelsforFaceDetection Rocío Cabrera, Guillaume Lemaître, MojdehRastgoo

2. PresentationOutline • Introduction • Database Used • The IMM Face Database • Models • Statistical Shape Models • Statistical Models of Appearance • Active Appearance Models • Implementation • Training Stage • Testing Stage • Conclusions 3D Digitization - Active Appearance Models for Face Detection

3. Introduction • Non-trivial Applications in Machine Vision • “Understand” the presented images • Recover image structure • Know what this structure means • Real applications include complex/variable structures • Faces Detection • Model-based Methods • Prior knowledge of the problem • Expected Shapes of Structures • Their Spatial Relationship • Greylevel Appearance RestrictAutomatedSearchto Plausible Interpretations 3D Digitization - Active Appearance Models for Face Detection

4. Introduction • Generative Models • Are able to generate realistic images of target objects • Deformable Models • Are able to deal with object variability • Two main desired characteristics • General – capable of generating plausible examples of the class they represent • Specific – capable of generating only legal/valid example • Model-based Methods • Top-down Strategy • Prior Model of Expected Class • Find Best Match in Image • “Measure” if the target is actually present 3D Digitization - Active Appearance Models for Face Detection

5. The IMM Face Database • An Annotated Dataset of 240 Face Images • 40 different human subjects (7 females vs. 33 males) • All without glasses or accessories • Manual Annotation of 58 landmarks • Six Different Positions • Full frontal face, neutral/happy expression, diffuse light • Face rotated (30° right/left), neutral expression, diffuse light • Full frontal face, neutral expression, spot light added at the person's left side. • Full frontal face, arbitrary expression, diffuse light. • Mouth • Jaw • Eyebrows • Eyes • Nose 3D Digitization - Active Appearance Models for Face Detection

6. StatisticalShapeModels Shape alignment Modelling Shape Variation 3D Digitization - Active Appearance Models for Face Detection • Procrustes Analysis : Aligning the images onto the same reference axes • Translation, Rotation and Scaling Transformations • Procrustes Analysis minimizes the distance between a reference shape and each shape in the dataset Computation of the mean shape Computation of the scatter (covariance) matrix Sorting the eigenvectors and keeping the first k eigenvectors , based on the largest eigenvalues Eigen decomposition of the shapes where , Value of k is based on

7. Statistical Shape Models • Mean Shape and Largest Deformation 3D Digitization - Active Appearance Models for Face Detection

8. StatisticalShapeModels 3D Digitization - Active Appearance Models for Face Detection

9. StatisticalModels of Appearance • Texture mapping is required to generate the photo realistic synthetic images • Combination of a shape variation model with texture variation model • Configuration of landmarks • Texture is the pattern of intensities or color across the image patch shape model Texture model 3D Digitization - Active Appearance Models for Face Detection

10. Statistical Models of Appearance Training set of label Images Computation of statistical shape models -PCA Statistical Shape models – Mean shapes Computation of Free-Patch Images – Image wrapping Appearance models PCA Applying PCA on Free-Patch Images Statistical texture Models 3D Digitization - Active Appearance Models for Face Detection

11. Statistical Models of Appearance – Image wrapping • Piece Wise Affine • Performing the Delaunay triangulation on each shape model • Affine Transformation which maps the corner of the triangles to their new positions in new Image 3D Digitization - Active Appearance Models for Face Detection

12. Statistical Models of Appearance – Texture modeling • Training set of shape-free normalized image patches • Performing PCA • Model of texture: Set of orthogonal modes of variations Set of gray level parameters Mean normalized gray level 3D Digitization - Active Appearance Models for Face Detection

13. StatisticaltextureModels Texture model Eigen-faces decomposition 3D Digitization - Active Appearance Models for Face Detection

14. Statistical Models of Appearance – Combined Image • Shape parameter vector and texture parameter vector might have correlation • Performing PCA • Appearance Model: Diagonal matrix of weight for each shape parameter Controlling both shape and texture Eigenvectors Shape and texture will be a function of c 3D Digitization - Active Appearance Models for Face Detection

15. StatisticalAppearanceModels Combination of texture model and shape model Difference Image Texture model 3D Digitization - Active Appearance Models for Face Detection

16. Implementation • Consists of two main stages • Training Stage • Multi-scale implementation to obtain an AAM model • N scalesimplementation • Testing Stage • Searches for the object (face) in a test image 3D Digitization - Active Appearance Models for Face Detection

17. Implementation – Training Stage • Load Training Data • for SCALE = 1:N • MakeShapeModel • Align shapes with Procrustes Analysis • Obtain main directions of variations with PCA • Keep the 98% most significant eigenvectors • Grey-levelappearanceModel • Transform face image into mean texture image • Normalize the greyscale, to compensate for illumination • Perform PCA • Keep the 99% most significant eigenvectors • CombinedShape-AppearanceModel • Addition of the shape and appearance models • Perform PCA • Keep only 99% of all eigenvectors • Search Model • Find the object location in a test set • Training done by translation and intensity difference computation (keep position with smallest difference) • Transform the image to a coarser scale • end Make Shape Model Grey-level Appearance Model Combined Shape-Appearance Model Search Model Transform Image to a Coarser Scale 3D Digitization - Active Appearance Models for Face Detection

18. Implementation – Testing Stage • Manual Initialization • For SCALE = 1:N (start in coarserscale) • GetModelforCurrentScale • ImageScaling • SearchIterations • SampleImageIntensities • Compute differencebetweenmodel and real imageintensities • IfErrorold < Errorcurrent • Gotopreviouslocation • Else • UpdateErrorold • End • End • Gotonextfinerscale • End • Show DetectionResults Manual Initialization Search Iterations Show Detection Results 3D Digitization - Active Appearance Models for Face Detection

19. Results Lower Scale Higher Scale 3D Digitization - Active Appearance Models for Face Detection

20. Results Highest Scale Texture map found at this scale 3D Digitization - Active Appearance Models for Face Detection

21. Problems Faced during Implementation • Memoryissuesduringthe training • Problem of thereconstruction of theappearance • Not real-time application 3D Digitization - Active Appearance Models for Face Detection

22. Conclusion • Facedetection and face tracking are non-trivial applications in machine vision • Model-basedmethods • Prior knowledge of the problem • Expected Shapes of Structures • Their Spatial Relationship • Grey-level Appearance • Active Appearance Models • Are built from a set of training examples • Shouldaccountforclassvariabilty • They heavily rely on Principal Component/ Eigenvalue Analysis • Through a search algorithm we seek to interpret a new target image with the optimal model parameters which best describe the target image • TheextensiontoFaceDetectionwasnotyetachievedbutitisexpectedtoworkforthedeliverabledue date • The use of AAM seemlike a promisingmethodtoperformfacedetection and/orrecognition 3D Digitization - Active Appearance Models for Face Detection

23. References [1] Ginneken B. et al. "Active Shape Model Segmentation with Optimal Features", IEEE Transactions on Medical Imaging 2002.  [2] T.F. Cootes, G.J Edwards, and C,J. Taylor "Active Appearance Models", Proc. European Conference on Computer Vision 1998  [3] T.F. Cootes, G.J Edwards, and C,J. Taylor "Active Appearance Models", IEEE Transactions on Pattern Analysis and Machine Intelligence 2001 [4] VazeosIoannis. Active Appearance Models (AAM). MASTER THESIS REPORT. Master of Science in Information Networking. Athens Information Technology. 2004 -2005 [5] T.F. Gootes and C.J. Taylor StatisticalModels of AppearanceforComputerVision. ImagingScience and BiomedicalEngineering, University of Manchester. TechnicalReport. 2004. [6] F. Dornaika and J. Ahlberg . Efficient Active Appearance Model for Real-Time Head and Facial Feature Tracking Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG’03). 2003 [7] AkshayAsthana, Jason Saragih, Michael Wagner and Roland Goecke. Evaluating AAM Fitting Methods for Facial Expression Recognition. 2009 IEEE [8] Mingcai Zhou, Yangsheng Wang, Xiaoyan Wang and XuetaoFeng. A Two-Stage Approach for AAM Fitting. Eighth International Conference on Intelligent Systems Design and Applications. 2008 IEEE [9] Fangqi Tang and Benzai Deng Facial Expression Recognition using AAM and Local Facial Features. Third International Conference on Natural Computation (ICNC 2007). 2007. 3D Digitization - Active Appearance Models for Face Detection