1 / 25

Face Alignment with Part-Based Modeling

Face Alignment with Part-Based Modeling. Vahid Kazemi Josephine Sullivan CVAP KTH Institute of Technology. Objective: Face Alignment. Find the correspondences between landmarks of a template face model and the target face. Annotated images (source: IMM dataset).

stefan
Download Presentation

Face Alignment with Part-Based Modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Face Alignment with Part-Based Modeling Vahid Kazemi Josephine Sullivan CVAP KTH Institute of Technology

  2. Objective: Face Alignment • Find the correspondences between landmarks of a template face model and the target face. Annotated images (source: IMM dataset) Test image (source: YouTube)

  3. Why: Possible Applications • The outcome can be used for: • Motion Capture: by determining head pose and facial expressions. • Face Recognition: by comparing registered facial features with a database. • 3D Reconstruction: by determining camera parameters using correspondences in an image sequence • Etc.

  4. Global Methods • Overview: • Create a constrained generative template model • Start with a rough estimate of face position. • Refine the template to match the target face. • Properties: • Model deformations more precisely • Arbitrary number of landmarks • Examples: • Active Shape Models [Cootes 95] • Active Appearance Model [Cootes 98] • 3D Morphable Models [Blanz 99]

  5. Part-Based Methods • Overview: • Train different classifiers for each part. • Learn constraints on relative positions of parts. • Properties: • More robust to partial occlusion • Better generalization ability • Sparse results • Examples: • Elastic Bunch Graph Matching [Wiskott 97] • Pictorial Structures [Felzenszwalb 2003]

  6. Our approach to face alignment • How can we avoid the draw backs of existing models?

  7. Our approach to face alignment • Find the mapping, q, from appearance to the landmark positions: • But q is complex and non-linear…

  8. Linearizing the model • Use piece-wise linear functions

  9. Linearizing the model • Use a part based model

  10. Linearizing the model • Use a suitable feature descriptor Feature Descriptor

  11. Part Selection Criteria • Detect the parts accurately and reliably • Contain strong features • Ensure a simple (linear) model • Minimum variation • Capture the global appearance • Cover the whole object

  12. Part Selection for the face We chose nose, eyes, and mouth as good candidates Image from IMM dataset

  13. Appearance descriptor • Variation of PHOG descriptor • Divide the patch into 8 sub-regions • Recursively repeat for square regions

  14. Part detection • Build a tree-structured model of the face, with nose at the root, and eyes and mouth as the leafs of the tree.

  15. Part detection • Detect the parts by sliding a patch on image and calculating the Mahalanobis distance of the patch from the mean model

  16. Part detection • Find the optimal solution by minimizing the pictorial structure cost function: • We can solve this efficiently by using generalized distance transform [Felzenszwalb 2003] by limiting the cost function

  17. Regression • Model the mapping between the patch’s appearance feature (f) and its landmark positions (x) as a linear function: • Estimate weights from training set using Ridge regression

  18. Regression • Comparison of different regression methods

  19. Robustify the regression function • Why • Compensate for bad part detection • Deformable parts don’t exactly fit in a box • How • Extend training set by adding noise to part positions

  20. Experiments • Use 240 face images from IMM dataset. • Dataset contains still images from 40 individual subjects with various facial expressions under the same lighting settings • 58 landmarks are used to represent the shape of subjects

  21. Results • Comparison of localization accuracy of our algorithm comparing to some existing methods on IMM dataset. * Mean error is the mean Euclidean distance between predicted and ground truth location of landmarks in pixels

  22. Results • The results of cross validation on IMM dataset PredictedGround truth

  23. Demo More videos: http://www.csc.kth.se/~vahidk/face/

  24. Conclusion and future work • Part-Based models can be used to simplify complicated models • The choice of parts is very important • HOG descriptors are not fully descriptive

  25. Questions?

More Related