Download
1 / 29
290 likes | 531 Views
Realistic Performance-driven Facial Animation. M. Sanchez, J. Edge and S. Maddock. Realism in Facial Animation. Traditional concept: Not necessarily true… The human face is a highly complicated system of which we have rather limited information to build a mechanical simulation
E N D
Realistic Performance-drivenFacial Animation M. Sanchez, J. Edge and S. Maddock
Realism in Facial Animation • Traditional concept: • Not necessarily true… • The human face is a highly complicated system of which we have rather limited information to build a mechanical simulation • We mainly perceive the visual effects of such processes, not the mechanisms underneath • Physically-based approaches have issues of their own: • Stability is difficult to assess, convergence impossible • Most numerical methods scale up worse than linearly wrt the complexity of the model • The problem is connaturally stiff (linear elasticity >> resistance to bend and shear), making it even more numerically sensitive • Conclusion: it’s questionable whether this concept of realism can lead to more plausible results than mere anatomically-driven geometric systems (eg. Waters)
Capturing Realism • One sensible alternative: • Intrinsic problems: • Only a limited domain of information of actual facial motion is accessible through capture (spatially discrete / timewise discrete) • The captured data is bound to the physiognomy of the actual subject • Geometric deformation techniques can be used to extrapolate spatially discrete motion (eg. markers) to the full extent of facial skin • Can be consistent with large-scale properties of skin deformation • Oversee fine detail aspects of skin mechanics (furrows / creases) • Retargeting of facial motion is possible using purely geometric criteria (without an anatomic model) • Small skin deformation can be modelled as a parallel process to large-scale motion
1.3. Animating the skin • Large-scale aspects of skin mechanics to be reproduced: • Motion is propagated across the connectivity of the skin • Underlying layers of tissue dampen the extents of deformation • Local smoothness is generally preserved • Conventional techniques for spatial interpolation of motion do not fully satisfy these criteria • Point-based deformations (Williams, Pelachaud, Kshirsagar) • Volumetric techniques (Kalra, Escher) • RBFs (Fidaleo) • Similar approaches could work if interpolation was performed along geodesic curves on the surface, but that would only work for 2-manifolds, and the skin has openings
1.3. Animating the skin • Through surface-to-surface mappings we can consistently interpolate motion across approximations of the mesh to be deformed • BIDS does precisely that… • Motion propagates naturally across a piecewise surface whose patch topology is compatible with that of the face model • Smoothness and locality of deformation can be asserted through the construction and conditioning of such approximating surface • For further details: • “Realistic Performance-driven Facial Animation using HW Acceleration” (submitted to ToG) • Thesis (soon to come ;)
1.2. Adapting facial motion • Key fact: • Conventional MoCap retargeting cannot be applied to soft body deformation • The relation between source and target motion is highly non-linear, both with relation to its point of application and its extent and direction • We model this by constructing an RBF-based mapping between the embedding of source and target control surfaces… • capable of representing the two kinds of non-linearities • lacking of boundary separation issues due to the application of BIDS • stable under the usual scale of facial motion • For further details: • “Use and re-use of facial MoCap” VVG’03
1.1. Labelling the target mesh • Basic idea: • Our approach to this model consists on deforming the ref. mask to fit the target mesh, and then using the resulting control surface • this is solved as an iterative process, minimizing the following energy function: • the elastic energy terms correspond to the classic definition by Terzopoulos, and they have analytic solutions for Bezier Triangles • non-analytic terms require using blind methods (eg simplex downhill) or forward differences for their spatial derivatives
2.a. Fitting a model to photographs • Motivation: • A model of the face is needed to act as support of the normal fields captured in 2.b • We need to analyse the ratio of deformation for each of the expressions in 2.c • We proceed by retrieving the position of markers on the face of the subject with reverse stereometry from 3 viewpoints… and then deform the reference mask using BIDS:
2.b. Shape from shading • Consider the simplified BDRF equation: without the specular component, it would be linear… • use complementary light polarizers on lights and camera lenses, or • apply translucent makeup on the subject (cheap option) • With a set of 3 exposures to distinct lighting conditions, N is fully determined for any given pose (Rushmeier) • The variation of the normal field (N) is then encoded in the tangent space of the fitted mesh, producing normal maps
2.cd. Deriving a model of wrinkling • The fitted model in every expression not only acts as reference for normal maps, but also allows measuring the ratio of deformation on the face by approximating its strain tensor with that on the control surface • represents both magnitude and direction of infinitesimal strain • compression (negative strain) can be isolated by chopping off negative eigenvalues • Compression is ultimately the cause of facial tissue wrinkling, however… • elastic response is not the same in every region, neither in every direction of strain • linear compression is much more relevant than shearing • We have a sampling representing these properties (normal maps), together with the corresponding compression tensors, so we can build a model relating the two: • it’s a polynomial approximation of N wrt the 3 components of the symmetric form of the tensor • experimentation reveals linear degree suffixes • For any displaced configuration of the control surface, we can evaluate the model and compute the corresponding normal map
Question time… {m.sanchez, j.edge} @dcs.shef.ac.uk
