1 / 35

3D Face Modeling and Animation

3D Face Modeling and Animation. CMSC 3D Character Design & Animation. Contents. Motivation 3D facial geometry modeling 3D facial deformation modeling 3D facial animation The iFace system Speech-driven talking heads Text-driven talking heads A glimpse at MP5. Motivation.

ulla
Download Presentation

3D Face Modeling and Animation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Face Modeling and Animation CMSC 3D Character Design & Animation

  2. Contents Motivation 3D facial geometry modeling 3D facial deformation modeling 3D facial animation The iFace system Speech-driven talking heads Text-driven talking heads A glimpse at MP5

  3. Motivation • Avatar-based Human-Computer Interaction • Animation • Teleconference • Recognition: Face recognition, soft biometrics

  4. 3D Facial Geometry Modeling Polygonal mesh Vertices Triangles, quadrangles, etc. Normals Texture Each mesh vertex is a sample point of the human facial surface  How do we acquire the positions of these sample points?

  5. 3D Facial Geometry Acquisition Artist’s designs 3D scanners (active) 3D reconstruction from 2D image(s) (passive)

  6. Laser Scans

  7. Active Acquisition Time of flight Examples: DepthSense , PMD Structured light Example: Kinect

  8. Passive Stereo Photogrammetry Photo courtesy: Dimensional imaging

  9. Beeler - Siggraph 2010 http://www.youtube.com/watch?v=JX5stsU6xfE

  10. 3D Face Reconstruction Framework Neutral Frontal Face Texture 2D Alignment MPG4 FAT Expression Pose Illumination Models Classifier ?? New Face

  11. 3D Facial Deformation Modeling Free-form deformation models Muscle-based deformation models

  12. Free-form Deformation Model The coordinates of the mesh vertices can be deformed in a free-form manner by changing the positions of some control points Control points can either belong to the mesh vertices or not Example: Piecewise Bezier Volume Deformation Model (Tao and Huang, 1998)

  13. Muscle-based Deformation Model Muscles of the face

  14. Muscle-based Deformation Model Build simplified mathematical model that simulates muscle actions on the facial skin Linear Muscle Models

  15. Muscle Based Animation Uses a mass-and-spring model to simulate facial muscles. Muscles are of two types: linear muscles that pull and elliptic muscles that squeeze. Muscle parameters: muscle vector and zone of muscle effect.

  16. Modeling the Primary Facial Expressions Basic facial expressions that are considered to be generic to the human face: Happiness, Anger, Fear, Surprise, Disgust and Sadness.

  17. Synthesized Facial ExpressionsWaters SIGGRAPH ‘87 Neutral face Anger Happiness Surprise Fear Disgust

  18. Facial Action Coding System (FACS) The system was developed by Ekman and Friesen, in 1978 FACS describes facial deformations in terms of “Action Units” (AUs) Some of the AUs correspond directly to actions of facial muscles; others involve things like the movement of the tongue or air filling the cheeks AUs may be combined to describe any facial expressions

  19. Facial Action Coding System (FACS)

  20. MPEG4 Facial Animation Parameters (FAPs) MPEG4 defines 68 FAPs, categorized into 10 groups

  21. Motion Units Learn the basic facial deformations from motion capture data (Hong, Wen, and Huang, 2001)

  22. The Anatomical Model The face can be modeled by two layers and three surfaces Dermal-fatty Layer Muscle Layer Epidermal surface Fascia Surface Skull Surface

  23. The Volume Preservation Forces • The human skin is incompressible • Volume preservation force is needed to simulate the wrinkles • Pressing the node upwards proportionally to the decrement of the volume

  24. Geometry models for other head components Teeth, eyes, and neck are modeled separately These data are difficult to be captured by the scanner

  25. Muscle-based Animation Estimating the muscle activation from the motion capture data First, a precise anatomical model is Produced from Visible Human Motion Dataset Next, the muscles are activated so that the simulated location of the marker overlaps with its real location

  26. 3D Facial Animation Key-frame interpolation method Place particular facial deformations at particular time instants (key-frames) Facial deformations in-between key-frames are obtained by a certain interpolation scheme Two basic key-frame types Visemes Expressions

  27. Visemes Representational unit used to classify speech sounds in the visual domain Was introduced based on the interpretation of the phoneme as a basic unit of speech in the acoustic/auditory domain But …Viseme  Phoneme

  28. Visemes Describes the particular facial and oral positions and movements that occur alongside the voicing of phonemes The analogous term for the acoustic reflection of a phoneme would be "audieme", but this is not in use

  29. Visemes Phonemes and visemes do not always share a one-to-one correspondence Often, several phonemes share the same viseme

  30. Visemes Conversely, some sounds which are hard to distinguish acoustically are clearly distinguished by the face For example, acoustically speaking English /l/ and /r/ could be quite similar (especially in clusters, such as 'grass' vs. 'glass') Yet visual information can show a clear contrast This is demonstrated by the more frequent mishearing of words on the telephone than in person

  31. Visemes

  32. Visemes

  33. Facial expressions

  34. Speech-driven talking heads

  35. Text-driven talking heads

More Related