1 / 22

AAM based Face Tracking with Temporal Matching and Face Segmentation

CVPR 2010. AAM based Face Tracking with Temporal Matching and Face Segmentation. Mingcai Zhou 1 、 Lin Liang 2 、 Jian Sun 2 、 Yangsheng Wang 1. 1 Institute of Automation Chinese Academy of Sciences, Beijing, China. 2 Microsoft Research Asia Beijing, China. Outline. AAM Introduction

edan-fuller
Download Presentation

AAM based Face Tracking with Temporal Matching and Face Segmentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CVPR 2010 AAM based Face Tracking with Temporal Matching and Face Segmentation Mingcai Zhou1、 Lin Liang2、 Jian Sun2、Yangsheng Wang1 1Institute of Automation Chinese Academy of Sciences, Beijing, China 2Microsoft Research Asia Beijing, China

  2. Outline • AAM Introduction • Related Work • Method and Theory • Experiment

  3. AAM Introduction • A statistical model of shape and grey-level appearance Shape model Appearance model

  4. Shape Model Building :mean shape :shape bases ,shape parameters learn by PCA generate mean shape、 shape bases

  5. Texture Model Building :mean appearance :appearance bases :appearance parameters 灰階值 W(x) Mean shape Shape-free patch

  6. AAM Model Building

  7. AAM Model Search • Find the optimal shape parameters and appearance parameters to minimize the difference between the warped-back appearance and synthesized appearance map every pixel x in the model coordinate to its corresponding image point

  8. Problems- AAM tracker • Difficultly generalize to unseen images • Clutterd backgrounds

  9. How to do? • A temporal matching constraint in AAM fitting -Enforce an inter-frame local appearance constraint between frames • Introduce color-based face segmentation as a soft constraint

  10. Related Work -feature-based (mismatched local feature) Integrating multiple visual cues for robust real-time 3d face tracking,W.-K. Liao, D. Fidaleo, and G. G. Medioni. 2007 -intensity-based (fast illumination changes) Improved face model fitting on video sequences, X. Liu, F. Wheeler, and P. Tu. 2007 temporal matching constraint

  11. Method and Theory • Extend basic AAM to Multi-band AAM • The texture(appearance) is a concatenation of three texture band values • The intensity (b) • X-direction gradient strength (c) • Y-direction gradient strength (d)

  12. Temporal Matching Constraint • Select feature points with salient local appearances at previous frame • I(t−1) to the Model coordinate and get the appearance A(t-1) • Use warping function W(x;pt) maps R(t-1) to a patch R(t) at frame t

  13. Shape parameter Initialization Face Motion Direction ,

  14. Shape parameter Initialization When r reaches the noise level expected in the correspondences, the algorithm stops

  15. Shape parameter Initialization -Comparison Motion direction Previous frame’s shape Feature matching

  16. Face Segmentation Constraint Where are the locations of the selected outline points in the model coordinate

  17. Face Segmentation Constraint -Face Segmentation

  18. Face Segmentation Constraint

  19. Experiments Lost frame num

  20. Experiments

  21. Conclusion ─ Our tracking algorithm accurately localizes the facial components, such as eyes, brows, noses and mouths, under illumination changes as well as large expression and pose variations. ─ Our tracking algorithm runs in real-time. On a Pentium-43.0G computer, the algorithm’s speed is about 50 fps for thevideo with 320 × 240 resolution

  22. Future Work ─ Our tracker cannot robustly track profile views with large angles ─ The tracker’s ability to handle large occlusion also needs to be improved

More Related