1 / 24

An isometric model for facial animation and beyond

An isometric model for facial animation and beyond. Michael M. Bronstein. Department of Computer Science Technion – Israel Institute of Technology. Co-authors. Alex Bronstein. Ron Kimmel. Agenda. Single texture mapping onto an animated face. Morphing.

eugene
Download Presentation

An isometric model for facial animation and beyond

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An isometric model for facial animation and beyond Michael M. Bronstein Department of Computer Science Technion – Israel Institute of Technology

  2. Co-authors Alex Bronstein Ron Kimmel

  3. Agenda Single texture mapping onto an animated face Morphing Expression interpolation and extrapolation Beyond…

  4. Isometric model of facial expressions • Face: deformable Riemannian surface with geodesic distances • Facial expression: approximate isometry B2K, IJCV 2005

  5. Virtual makeup Map a single texture image onto a 3D video sequence of animated face in an expression-invariant manner TEXTURE 3 D V I D E O S E Q U E N C E

  6. Approach I: Common parametrization • Parametrize and over a common parametrization domain • by the maps and • Draw the texture in the parametrization domain • Map the texture to and using the maps and

  7. How to find a parametrization? Embed and into the plane by a minimum-distortion map G. Zigelman et al., IEEE TVCG, 2002

  8. Multidimensional scaling Given a sampling the minimum-distortion embedding is found by optimizing over the images and not on itself Alternative, more robust formulation: • Approximately common parametrization • Requires alignment (usually manual, according to some fiducial points) • Difficult to handle different or complicated topologies A. Elad, R. Kimmel, CVPR 2001

  9. Approach II: Correspondence problem • Assume that the texture is drawn on • Find correspondence between and • Transfer the texture by the map • In case of common parametrization,

  10. How to find the correspondence? • Fiducial points-based methods usually give sparse correspondence and • require manual assistance • Optical flow between texture images (Blanz et al.) is not applicable when • only geometric information is given Embed into by a minimum-distortion map

  11. Generalized multidimensional scaling (I) G MDS: MDS: • are computed once using • fast marching • have to be computed at each iteration • Note that are not restricted to • the mesh vertices B2K, PNAS 2006

  12. Generalized multidimensional scaling (II) A weighted least-squares version of the problem • More robust in practice • Weights allow to handle different • topologies (e.g. open mouth) and • missing data (scanner artifacts) • Multiresolution / multigrid schemes • to prevent local convergence B2K, PNAS 2006

  13. Reference Transferred texture

  14. Calculus of faces (I) Interpolation Extrapolation • Abstract manifold of facial articulations • Face animation: trajectory • Minimum-distortion correspondence allows creating a (locally) linear • space, in which faces are represented as vectors

  15. Calculus of faces (II) CORRESPONDENCE Extrinsic geometry Texture • Extrinsic coordinates and texture interpolation

  16. Interpolation I N T E R P O L A T E D F R A M E S 0 0.25 0.5 0.75 1 • Temporal super-resolution: increase frame rate of 3D video by • adding interpolated frames • Interpolation of geometry and texture

  17. Extrapolation NEUTRAL EXPRESSION EXAGGERATED EXPRESSION 0 1 1.5 • Expression exaggeration: synthesize new expressions using a • non-convex combination • Interpolation of geometry and texture

  18. Bronstein2 & Kimmel An isometric model for facial animation and beyond

  19. Morphing SOURCE TARGET 0 0.25 0.5 0.75 1 • Convex combination between two different faces • Morphing of geometry and texture

  20. Bronstein2 & Kimmel An isometric model for facial animation and beyond

  21. Virtual body art Texture mapping on articulated human body, similarly to body art

  22. 22 Bronstein2 & Kimmel An isometric model for facial animation and beyond Reference Transferred texture

  23. 23 Bronstein2 & Kimmel An isometric model for facial animation and beyond Reference Transferred texture

  24. Summary • Isometric model of facial expressions • Automatic dense correspondence based on the minimum-distortion • mapping • Possibility to find correspondence between partially missing or partially • overlapping surfaces (COME TO THE SECOND TALK AT 15:30) • Texture mapping, expression synthesis, morphing, etc… • GMDS - a generic tool that can be applied to different problems

More Related