1 / 36

Context Constrained Facial Landmark Localization Based on Discontinuous Haar-like Features

Context Constrained Facial Landmark Localization Based on Discontinuous Haar-like Features. Xiaowei Zhao*, Xiujuan Chai*, Zhiheng Niu**, Cherkeng Heng**, Shiguang Shan * *Institute of Computing Technology, CAS **Panasonic Singapore Laboratories Pte Ltd (PSL). Outline. Background Motivation

nell
Download Presentation

Context Constrained Facial Landmark Localization Based on Discontinuous Haar-like Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Context Constrained Facial Landmark Localization Based on Discontinuous Haar-like Features Xiaowei Zhao*, Xiujuan Chai*, Zhiheng Niu**, Cherkeng Heng**,Shiguang Shan* *Institute of Computing Technology, CAS **Panasonic Singapore Laboratories Pte Ltd (PSL)

  2. Outline • Background • Motivation • Our Method • Experiments • Summary

  3. Task • Localize key facial landmarks • e.g. corners of the eyes, corners ofthe mouth Localization of key facial landmarks

  4. Previous Works • Rule-based methods • Skin color-based • R.Hsu et al. 2002 • Integral projection • Z.Zhou et al. 2004

  5. Previous Works • Rule-based methods • Deformable template • Fitting some pre-defined geometric shapes to the input image • A.Yuille et al. 1992

  6. Previous Works • Rule-based methods • Deformable template • Statistical models • Fitting some Statistical shape/appearance models • Active Shape Models (ASM) • T.Cootes et at. 1995 • Active Appearance Models (AAM) • T.Cootes et al. 1998 • Constrained Local Model • D.Cristinacce et al. 2006 • Many variations…

  7. Previous Works Rule-based methods Deformable template Statistical models AdaBoost-based methods Almost the same as face detection AdaBoost + Haar-like

  8. Previous Works • Rule-based methods • Deformable template • Statistical models • AdaBoost-based methods • Context-constrained methods • The positions of one landmark can be “determined” from other landmarks. • T.Kozakaya et al. 2010 • M.Valstar et al. 2010

  9. Previous Works • Boosted Regression and Graph Models [M.Valstar et al. CVPR 2010] • Explore the relationship between a patch L and the target location T. • SVR is used to estimate the direction of the target (left panel) and the distance to the target (right panel) It is good, but might be a little complicated.

  10. Motivation • Problem • How to exploit the context in an easy way, better not to change the AdaBoost framework? • A new type of feature?

  11. Outline • Background • Motivation • Our Method • Experiments • Summary

  12. Basic Idea • Traditionally, we do like this… Target point

  13. Our method • Discontinuous Haar-like Features • Mode #1: subtraction of two rectangles Mode #1

  14. Our method • Discontinuous Haar-like Features • Mode #2: subtraction of three rectangles Mode #2

  15. Our method • Discontinuous Haar-like Features • Mode #2: subtraction of four rectangles Mode #3

  16. Discontinuous Haar-like Features • Principles Behind • Context is modeled naturally in the form of feature co-occurrence

  17. Some Implementation Details

  18. Some Implementation Details • Exploiting both features • Traditional Haar-like features • Within the candidate window • Proposed discontinuous Haar-like features • Exploiting the context

  19. Some Implementation Details • Feature Styles Selection • Mode #2 and #3 are too complicated to control (numerous candidate features)

  20. Some Implementation Details • Even for Mode #1, numerous features • Reduce the number of candidates • Blocks are restricted to be square • The size of the square is fixed • Larger sampling steps Sampling step=2 Sampling step=2

  21. Some Implementation Details • No need to change the Boosting framework • Automatically select the features/weak classifiers (Look-Up Table by B.Wu et al. 2004) • Feature heritage is used to speed up detection. • Use less different features

  22. Outline • Background • Motivation • Our Method • Experiments • Summary

  23. Training and Testing Dataset Training datasets CAS-PEAL, PIE, FGRC v1, FG-Aging Totally about 7000 near-frontal images with labels Testing datasets: BioID, Cohn-Kanade FE database

  24. Are the New Feature Valid? Discontinuous Haar-like features • The top 4 features selected by using Real AdaBoost (for right mouth corner) 1 2 3 4

  25. Are the New Feature Valid? About 1/3 features selected by Boosting are discontinuous Haar-like features! Traditional Haar-like features Discontinuous Haar-like features Percentage of two kinds of features

  26. Are the New Feature Valid? • When both features are used as candidates, fewer total features are used (with similar accuracy)! • So, new feature is more discriminative. Using only traditional Haar-like features as candidates Using both feature as candidates Comparison of selected feature number

  27. How Block Size affects? In case of 105x105 face size Square size = 10 x 10 works better

  28. Comparison with only Haar-like Features Test Results on BioID database Outer corner of left eye Outer corner of right eye

  29. Comparison with only Haar-like Features Test Results on BioID database Inner corner of left eye Inner corner of right eye

  30. Comparison with only Haar-like Features Test Results on BioID database Left mouth corner Right mouth corner

  31. Comparison with Other Methods • Comparison with other methods on BioID *S. Milborrow and F. Nicolls, “Locating facial features with an extended active shape model,” In Proc. ECCV, pp. 504–513, 2008. **K. Kinoshita, Y. Konishi, S. Lao, and M. Kawade, “A fast and robust facial feature detection and 3D head pose estimation based on 3D model fitting,” In Proc. MIRU, 2008.

  32. Comparison with Other Methods • Comparison with other methods on Cohn-kanade database *D. Vukadinovic and M. Pantic, “Fully automatic facial feature point detection using gabor feature based boosted classifiers,” In Proc.Systems, Man and Cybernetics, vol. 2, pp. 1692–1698, 2005.. ***S. Milborrow and F. Nicolls, “Locating facial features with an extended active shape model,” In Proc. ECCV, pp. 504–513, 2008.

  33. Summarize Our contribution To model context, discontinuous Haar-like features are designed. Its effectiveness is Preliminarily validated. Pros Simple Context leads to robustness to variations No need to change the Boosting part Cons Not effective for occlusion Large pool of candidate features

  34. Future Work • Explore Mode #2 & #3 features • More context in each feature • Generalize to other object detection • e.g. face detection (using hair, shoulder as context) • Locate multiple facial landmarks Simultaneously • Co-training • How to obtain occlusion-robustness?

  35. Thanks!

  36. Boosted Regression & Graph Models SVR regression (0,0) • Regression Prediction • Learning the mapping between the appearance of the area surrounding a point and the positions of these points • Support vector regression is adopted • Haar-like feature is adopted to describe appearance • AdaBoost-based feature selection to reduce the dimension of appearance features • Spatial Relations • Preventing unfeasible facial points combinations • Difference with our method • Only context/feature around facial point is used • MRF is used to model the relationship among facial landmarks MRF points model

More Related