Planar Bones:

1. Planar Bones: A deformation technique for Facial Animation Manuel A. Sánchez, University of Sheffield

2. Overview • Current Facial Animation approaches • Surface-driven deformations: The Planar Bones algorithm • Implementation using Vertex Programs • Considerations of efficiency and quality of representation

3. Academic Fully automated systems, no artistic control input Use of high level control paradigms, anatomically-based Computation time and amount of memory required are not decisive factors Emphasis is on how neat and innovative the system is CGI / Computer Games Animation process is highly dependent on artists Mostly generic geometrical approaches, not FA-specific Real-time editing or rendering is required, sometimes with very limited resources Only results matter 1. Current approaches to Facial Animation(I)

4. Academic Wide variety of methods: Physically-based simulations Muscle functions Displacement of Control Point neighbourhoods Free Form Deformations Radial Basis Functions … CGI / Computer Games Reduced set of methods, intensively used: Blending facial poses (morph targets) Bones 1. Current approaches to Facial Animation(II)

5. 1.1 Blending facial morph targets • Pros: • Simple, light-weight implementation • Allows precise artistic editing of the poses, down to vertex granularity • Cons: • Massive data set: one mesh per pose • The blending has to be performed in the CPU, since storing all the morphs in gfx memory is overkill • Interpolation shortcomings • How to accurately control the animation in-between the morphs?

6. 1.2 Bones for Facial Animation: Pros • Overcome morph target shortcomings: • The only extra storage this method requires is the bone weights per vertex • Can be completely implemented in the GPU, reducing bus usage to a minimum • Integrate easily with full body skeletal character animation • The tools for content generation are provided by any major 3D package, e.g. Maya, Character Studio

7. 1.3 Bones for Facial Animation: Cons • Lack of the fine control of morph target editing • Axial deformation technique, not surface-oriented • Treat regions of the face like part of cylinder-like objects • Artefacts become evident when polygon count increases Planar surface deformed using a grid of (red) bones

8. 2. Surface-driven deformations • Why not use polygonal meshes to control the deformation of surfaces? • Intuitive, LOD-wise, deformation paradigm • Better control provided by the correspondence between control and deformed surfaces • Can be combined with linear bones to provide complete character animation functionality

9. 2.1 Planar Bones: deforming a point S • A triangle is used as the control element, instead of a segment • The triangle defines a frame of reference, although not orthogonal • For every triangle abc: • Consider the deformation induced by a triangle moving from OPQ to opq • The image of any vertex S is given by:

10. 2.2 Planar Bones: weighted sums • A weighted sum combines the effects of several control elements acting on the same vertex • A weight is defined as a function of the distance between the control triangle and the vertex to be deformed • Computing that distance requires a case classification for S An initial partitioning can be obtained from the barycentric coordinates of the vertex projection on OPQ :

11. 2.2 Planar Bones: weighted sums This partitioning is refined for R’1 (and similarly for R’2 and R’3) by projecting the vertex over the correspondent segment: • Then the distance is computed from a point, line or plane, according to the partition S belongs to • A distance limit is applied to define the region of space the control element deforms (affection volume) • To preserve continuity on the deformed surface, the weighting function must vanish when the distance approaches the distance limit

12. 2.3 Planar Bones: extremities problem • The extremities of the affection volume of a given control element may be distorted as a consequence of the deformation, thus not preserving the distance to it • This breaks the correspondence between control and deformed surfaces, causing the deformed surface to “flow away” from control structure • Joints between control elements are especially sensitive, exhibiting constructive distortions Breach of correspondence between control mesh and deformed surface

13. 2.4 Solving the extremities problem • The solution is to give a specific treatment to the geometry encompassed in the flawed regions so that deformation actually preserves the distance to the control element • This results in seven different cases, out of which three are not linear transformations • However, these non-linear expressions can be satisfactorily approximated by their linear counterparts

14. 2.5 Planar Bones: application • Planar Bones produce satisfactory mesh-driven surface deformations • Specific treatment of the extremities corrects the potential problematic distortions

15. 3. Implementation using Vertex Programs • Similar to skeletal animation, isn’t it? • The deformed geometry is given as the weighted sum of the result of several linear transformations, evaluated for every vertex float4 position = float4(0.0, 0.0, 0.0, 1.0); for(int i = 0; i < 4; ++i) { position.xyz += mul(bnMats_T[indices.x], IN.position)*weights.x; indices = indices.yzwx; weights = weights.yzwx; } OUT.position = mul(modelViewProj, position);

16. 3. Implementation using Vertex Programs • Well, not exactly… • The transformations are not isometric, therefore scaling and shearing will be present • To guarantee continuity along the deformation, the affection volumes of the control elements will have frequent and large intersections, thus some vertices will be affected by a significant amount of different control elements

17. 3.1 Transformations are not isometric • Therefore the surface normals have to be transformed by the inverse-transpose of the correspondent matrix • Weights are handled identically • The number of static input registers required per control element doubles, halving the number of them that can be handled simultaneously

18. 3.2 Intense affection volume intersections • For a populated control mesh, the number of groups of deformed faces affected by different sets of control elements can be quite large Control Mesh (106 elements) Initial partitioning (2186 groups)

19. 3.2 Intense affection volume intersections after compacting (179 groups) • To maximize bus efficiency and minimize state changes, the number of groups should be minimal • Groups whose control elements are a subset of those of another group can be compacted with the later • Clustering algorithms can be applied to obtain face macro-groups that build a sub-optimal partitioning of the mesh according to this criteria • The limit in the size of the clusters is set by the number of control elements that can be considered simultaneously after clustering (13 groups)

20. 4. Efficiency / Quality of representation • Several variables can be adjusted to set the most convenient trade-off between quality of representation and available resources: • Number of control elements considered per vertex • Number of control elements allowed per mesh partition • Normal computation model

21. 4.1 Control elements per vertex 4 elements per vertex • Gives the number of iterations per vertex, which is a key aspect both in vertex evaluation time and program size • Determines the number of variable input registers required for storing weights and indices • Affects critically the continuity of the deformation 8 elements per vertex

22. 4.2 Control elements per mesh partition • Inversely proportional to the number of face macro-groups: the less partitioning the better • Limited by the number of static input registers available on the graphics card • GeForce 3 & 4: room for 30 3x4 matrices for skinning • GeForce FX: up to 80 • … 28 elements per face, 13 partitions 80 elements per face, 4 partitions

23. 4.3 Surface normals model • The correct model duplicates requirements in static input registries • Choosing the incorrect one results in flawed lighting, and this can be visually perceptible in the regions where the deformation is not isometric Directly transformed normals Normals computed using inverse-transpose

24. 5. Summary • Planar Bones provide a convenient tool for controlling surface deformation • They define a “well-behaved” correspondence between the control mesh and deformed surface • Facial Animation can be controlled in an intuitive way • They can be implemented in a programmable transformation pipeline architecture • Partitioning of the mesh to be deformed may be required to operate with limited resources • Computing surface normals correctly comes at a high cost

25. Contact info Manuel A. Sánchez, m.sanchez@dcs.shef.ac.uk