1 / 29

Realistic Performance-driven Facial Animation

Realistic Performance-driven Facial Animation. M. Sanchez, J. Edge and S. Maddock. Realism in Facial Animation. Traditional concept: Not necessarily true… The human face is a highly complicated system of which we have rather limited information to build a mechanical simulation

Download Presentation

Realistic Performance-driven Facial Animation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Realistic Performance-drivenFacial Animation M. Sanchez, J. Edge and S. Maddock

  2. Realism in Facial Animation • Traditional concept: • Not necessarily true… • The human face is a highly complicated system of which we have rather limited information to build a mechanical simulation • We mainly perceive the visual effects of such processes, not the mechanisms underneath • Physically-based approaches have issues of their own: • Stability is difficult to assess, convergence impossible • Most numerical methods scale up worse than linearly wrt the complexity of the model • The problem is connaturally stiff (linear elasticity >> resistance to bend and shear), making it even more numerically sensitive • Conclusion: it’s questionable whether this concept of realism can lead to more plausible results than mere anatomically-driven geometric systems (eg. Waters)

  3. Capturing Realism • One sensible alternative: • Intrinsic problems: • Only a limited domain of information of actual facial motion is accessible through capture (spatially discrete / timewise discrete) • The captured data is bound to the physiognomy of the actual subject • Geometric deformation techniques can be used to extrapolate spatially discrete motion (eg. markers) to the full extent of facial skin • Can be consistent with large-scale properties of skin deformation • Oversee fine detail aspects of skin mechanics (furrows / creases) • Retargeting of facial motion is possible using purely geometric criteria (without an anatomic model) • Small skin deformation can be modelled as a parallel process to large-scale motion

  4. System overview

  5. System overview

  6. System overview

  7. System overview

  8. System overview

  9. System overview

  10. 1. Large-scale deformation

  11. 1.3. Animating the skin

  12. 1.3. Animating the skin • Large-scale aspects of skin mechanics to be reproduced: • Motion is propagated across the connectivity of the skin • Underlying layers of tissue dampen the extents of deformation • Local smoothness is generally preserved • Conventional techniques for spatial interpolation of motion do not fully satisfy these criteria • Point-based deformations (Williams, Pelachaud, Kshirsagar) • Volumetric techniques (Kalra, Escher) • RBFs (Fidaleo) • Similar approaches could work if interpolation was performed along geodesic curves on the surface, but that would only work for 2-manifolds, and the skin has openings

  13. 1.3. Animating the skin • Through surface-to-surface mappings we can consistently interpolate motion across approximations of the mesh to be deformed • BIDS does precisely that… • Motion propagates naturally across a piecewise surface whose patch topology is compatible with that of the face model • Smoothness and locality of deformation can be asserted through the construction and conditioning of such approximating surface • For further details: • “Realistic Performance-driven Facial Animation using HW Acceleration” (submitted to ToG) • Thesis (soon to come ;)

  14. 1.2. Adapting facial motion

  15. 1.2. Adapting facial motion • Key fact: • Conventional MoCap retargeting cannot be applied to soft body deformation • The relation between source and target motion is highly non-linear, both with relation to its point of application and its extent and direction • We model this by constructing an RBF-based mapping between the embedding of source and target control surfaces… • capable of representing the two kinds of non-linearities • lacking of boundary separation issues due to the application of BIDS • stable under the usual scale of facial motion • For further details: • “Use and re-use of facial MoCap” VVG’03

  16. 1.1. Labelling the target mesh

  17. 1.1. Labelling the target mesh • Basic idea: • Our approach to this model consists on deforming the ref. mask to fit the target mesh, and then using the resulting control surface • this is solved as an iterative process, minimizing the following energy function: • the elastic energy terms correspond to the classic definition by Terzopoulos, and they have analytic solutions for Bezier Triangles • non-analytic terms require using blind methods (eg simplex downhill) or forward differences for their spatial derivatives

  18. 1. Large-scale deformation

  19. 2. Fine tissue detail

  20. 2. Fine tissue detail

  21. 2.a. Fitting a model to photographs

  22. 2.a. Fitting a model to photographs • Motivation: • A model of the face is needed to act as support of the normal fields captured in 2.b • We need to analyse the ratio of deformation for each of the expressions in 2.c • We proceed by retrieving the position of markers on the face of the subject with reverse stereometry from 3 viewpoints… and then deform the reference mask using BIDS:

  23. 2.b. Shape from shading

  24. 2.b. Shape from shading • Consider the simplified BDRF equation: without the specular component, it would be linear… • use complementary light polarizers on lights and camera lenses, or • apply translucent makeup on the subject (cheap option) • With a set of 3 exposures to distinct lighting conditions, N is fully determined for any given pose (Rushmeier) • The variation of the normal field (N) is then encoded in the tangent space of the fitted mesh, producing normal maps

  25. 2.cd. Deriving a model of wrinkling

  26. 2.cd. Deriving a model of wrinkling • The fitted model in every expression not only acts as reference for normal maps, but also allows measuring the ratio of deformation on the face by approximating its strain tensor with that on the control surface • represents both magnitude and direction of infinitesimal strain • compression (negative strain) can be isolated by chopping off negative eigenvalues • Compression is ultimately the cause of facial tissue wrinkling, however… • elastic response is not the same in every region, neither in every direction of strain • linear compression is much more relevant than shearing • We have a sampling representing these properties (normal maps), together with the corresponding compression tensors, so we can build a model relating the two: • it’s a polynomial approximation of N wrt the 3 components of the symmetric form of the tensor • experimentation reveals linear degree suffixes • For any displaced configuration of the control surface, we can evaluate the model and compute the corresponding normal map

  27. Results

  28. Results

  29. Question time… {m.sanchez, j.edge} @dcs.shef.ac.uk

More Related