1 / 21

Tracking Face Orientation

Tracking Face Orientation. Kentaro Toyama Vision–Based Interaction Group Microsoft Research. Puppeteering of graphical avatars Chat rooms/online games/video conf. Online customer service Performance-driven animation Enhanced accessibility Input for novel UI User monitoring.

dolf
Download Presentation

Tracking Face Orientation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tracking Face Orientation Kentaro Toyama Vision–Based Interaction Group Microsoft Research

  2. Puppeteering of graphical avatars Chat rooms/online games/video conf. Online customer service Performance-driven animation Enhanced accessibility Input for novel UI User monitoring Avatar animation Automated cameraman Hands-free cursor control Gaze-adjusted Video conferencing Active- window control Applications Vision for Graphics

  3. Related Work • Templates/EKF • (Jebara & Pentland, MIT Media Lab) • JET/Elastic bunch graphs • (von der Malsburg et al., Eyematic Interfaces) • Active Appearance Models • (Cootes & Taylor, Univ. of Manchester) Vision for Graphics

  4. + Face Detection Head Position Fine 3D Pose Coarse Orientation Overview Vision for Graphics Attention Detection

  5. Points tracked by multi-scale sum-of-absolute-difference template matching Estimate 6-DOF pose of known 3D points with Levenberg-Marquardt optimization 3D Pose Tracking Vision for Graphics

  6. Assume N=9 known points, in head-centered frame: Track N points in image: Find best-fit 6-DOF pose: Project model points: Algorithm Vision for Graphics

  7. Extract wavelet-based edge density features from known head location Project feature vectors onto trained ellipsoidal model Find maximum- likelihood 3D rotation Coarse Orientation Estimation detected face edge density templates 3D ellipsoidal model feature vectors Vision for Graphics Joint work with Ying Wu

  8. Algorithm Overview: Training Annotated Pose Ellipsoid Model Cropped Input Image Prepro-cessing Feature Extraction Vision for Graphics

  9. CroppedInput Image Grayscale and Resizing Histogram Equalization Masking Preprocessing Vision for Graphics

  10. Feature Extraction Preprocessed Image Feature Kernels Output Feature Vectors Vision for Graphics

  11. Algorithm Overview: Estimation Ellipsoid Model Estimator Predictor Cropped Input Image Prepro-cessing Feature Extraction Final Pose Motion Model Vision for Graphics

  12. Coarse Head Pose Estimation Vision for Graphics

  13. Coarse Head Pose Estimation Vision for Graphics

  14. Coarse Head Pose Estimation Vision for Graphics

  15. Bootstrap Initialization Boostrapped Ellipsoid Model Estimator Generic Ellipsoid Model Cropped Input Image Prepro-cessing Feature Extraction Final Pose Vision for Graphics

  16. Face Detection Head Position Attention Detection + Vision for Graphics

  17. Bayesian fusion of low-level information Observable indicators of component reliability influence weighting Head Position Estimation final estimate skin color edge motion color reliability em reliability rel. indicator rel. indicator Vision for Graphics Joint work with Eric Horvitz

  18. Skin-color blob Ellipse contour tracking Components Tracking algorithm • Reliability indicators - Bounding box aspect ratio - Fraction of pixels classified as skin • Ellipse-tracking residual • Fraction of pixels exhibiting interframe difference Vision for Graphics

  19. Compute edge density and average intensity in predefined regions Graph match with relational template over range of positions and scales Quick & Dirty Face Detection detected face edge density image relational template Vision for Graphics

  20. Bibliography • F. Pighin, J. Hecker, D. Lischinski, D. H. Salesin, and R. Szeliski. Synthesizing realistic facial expressions from photographs. In SIGGRAPH'98 Proceedings, pages 75--84, Orlando, July 1998. • Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen. Rapid modeling of animated faces from video. Technical Report MSR-TR-2000-11, Microsoft Research, February 2000. • B. Guenter et al. Making faces. Proceedings of SIGGRAPH 98, pages 55--66, July 1998. • V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. Proceedings of SIGGRAPH 99, pages 187--194, August 1999. • K. Toyama. Prolegomena for robust face tracking. Technical Report MSR-TR-98-65, Microsoft Research, November 1998. • F. Pighin, D. H. Salesin, and R. Szeliski. Resynthesizing facial animation through 3D model-based tracking. In Seventh International Conference on Computer Vision (ICCV'99), pages 143--150, Kerkyra, Greece, September 1999. Vision for Graphics

  21. Bibliography • I. Buck et al. Performance-driven hand-drawn animation. In Symposium on Non Photorealistic Animation and Rendering, pages 101--108, Annecy, June 2000. ACM SIGGRAPH. • D. A. Rowland and D. I. Perrett. Manipulating facial appearance through shape and color. IEEE Computer Graphics and Applications, 15(5):70--76, September 1995. • M. Turk and A. Pentland. Face recognition using eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'91), pages 586--591, Maui, Hawaii, June 1991. IEEE Computer Society Press. • P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711--720, July 1997. • A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742--756, July 1997. Vision for Graphics

More Related