360 likes | 707 Views
3D Face Modeling and Animation. CMSC 3D Character Design & Animation. Contents. Motivation 3D facial geometry modeling 3D facial deformation modeling 3D facial animation The iFace system Speech-driven talking heads Text-driven talking heads A glimpse at MP5. Motivation.
E N D
3D Face Modeling and Animation CMSC 3D Character Design & Animation
Contents Motivation 3D facial geometry modeling 3D facial deformation modeling 3D facial animation The iFace system Speech-driven talking heads Text-driven talking heads A glimpse at MP5
Motivation • Avatar-based Human-Computer Interaction • Animation • Teleconference • Recognition: Face recognition, soft biometrics
3D Facial Geometry Modeling Polygonal mesh Vertices Triangles, quadrangles, etc. Normals Texture Each mesh vertex is a sample point of the human facial surface How do we acquire the positions of these sample points?
3D Facial Geometry Acquisition Artist’s designs 3D scanners (active) 3D reconstruction from 2D image(s) (passive)
Active Acquisition Time of flight Examples: DepthSense , PMD Structured light Example: Kinect
Passive Stereo Photogrammetry Photo courtesy: Dimensional imaging
Beeler - Siggraph 2010 http://www.youtube.com/watch?v=JX5stsU6xfE
3D Face Reconstruction Framework Neutral Frontal Face Texture 2D Alignment MPG4 FAT Expression Pose Illumination Models Classifier ?? New Face
3D Facial Deformation Modeling Free-form deformation models Muscle-based deformation models
Free-form Deformation Model The coordinates of the mesh vertices can be deformed in a free-form manner by changing the positions of some control points Control points can either belong to the mesh vertices or not Example: Piecewise Bezier Volume Deformation Model (Tao and Huang, 1998)
Muscle-based Deformation Model Muscles of the face
Muscle-based Deformation Model Build simplified mathematical model that simulates muscle actions on the facial skin Linear Muscle Models
Muscle Based Animation Uses a mass-and-spring model to simulate facial muscles. Muscles are of two types: linear muscles that pull and elliptic muscles that squeeze. Muscle parameters: muscle vector and zone of muscle effect.
Modeling the Primary Facial Expressions Basic facial expressions that are considered to be generic to the human face: Happiness, Anger, Fear, Surprise, Disgust and Sadness.
Synthesized Facial ExpressionsWaters SIGGRAPH ‘87 Neutral face Anger Happiness Surprise Fear Disgust
Facial Action Coding System (FACS) The system was developed by Ekman and Friesen, in 1978 FACS describes facial deformations in terms of “Action Units” (AUs) Some of the AUs correspond directly to actions of facial muscles; others involve things like the movement of the tongue or air filling the cheeks AUs may be combined to describe any facial expressions
MPEG4 Facial Animation Parameters (FAPs) MPEG4 defines 68 FAPs, categorized into 10 groups
Motion Units Learn the basic facial deformations from motion capture data (Hong, Wen, and Huang, 2001)
The Anatomical Model The face can be modeled by two layers and three surfaces Dermal-fatty Layer Muscle Layer Epidermal surface Fascia Surface Skull Surface
The Volume Preservation Forces • The human skin is incompressible • Volume preservation force is needed to simulate the wrinkles • Pressing the node upwards proportionally to the decrement of the volume
Geometry models for other head components Teeth, eyes, and neck are modeled separately These data are difficult to be captured by the scanner
Muscle-based Animation Estimating the muscle activation from the motion capture data First, a precise anatomical model is Produced from Visible Human Motion Dataset Next, the muscles are activated so that the simulated location of the marker overlaps with its real location
3D Facial Animation Key-frame interpolation method Place particular facial deformations at particular time instants (key-frames) Facial deformations in-between key-frames are obtained by a certain interpolation scheme Two basic key-frame types Visemes Expressions
Visemes Representational unit used to classify speech sounds in the visual domain Was introduced based on the interpretation of the phoneme as a basic unit of speech in the acoustic/auditory domain But …Viseme Phoneme
Visemes Describes the particular facial and oral positions and movements that occur alongside the voicing of phonemes The analogous term for the acoustic reflection of a phoneme would be "audieme", but this is not in use
Visemes Phonemes and visemes do not always share a one-to-one correspondence Often, several phonemes share the same viseme
Visemes Conversely, some sounds which are hard to distinguish acoustically are clearly distinguished by the face For example, acoustically speaking English /l/ and /r/ could be quite similar (especially in clusters, such as 'grass' vs. 'glass') Yet visual information can show a clear contrast This is demonstrated by the more frequent mishearing of words on the telephone than in person