1 / 17

Facial Type, Expression, and Viseme Generation

Facial Type, Expression, and Viseme Generation . Josh McCoy, James Skorupski, and Jerry Yee. Introduction. Virtual Human Faces Hard to generate Easy to criticize Motivation Movies Games Problems Hand-made models take time Physically-based models look weird. Contribution.

bud
Download Presentation

Facial Type, Expression, and Viseme Generation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Facial Type, Expression, and Viseme Generation Josh McCoy, James Skorupski, and Jerry Yee

  2. Introduction • Virtual Human Faces • Hard to generate • Easy to criticize • Motivation • Movies • Games • Problems • Hand-made models take time • Physically-based models look weird

  3. Contribution • Data-driven facial face generation • User-guided categorization • Real-time pose generation from data

  4. Related Work: Face Retargeting • V. Blanz, C. Basso, and T. Vetter • Reanimating Faces in Images and Video. • Use a morphable model to synthesize a 3D face of the 2D image. • Capture 35 scans of static face poses (expressions and visemes in neutral expression) from a source actor. • Find dense point-to-point correspondences • Retarget facial movements to the 3D face. • Render the 3D face back into the 2D image.

  5. Related Work: Face Retargeting • Problems • Does not generate new expressions that are not in the source data set. • Does not combine and retarget expressions and visemes together.

  6. Related Work: Bilinear Model • E. Chuang and C. Bregler • Mood Swings: Expressive Speech Animation • Capture a video of an actor reading script under three different expressions (happy, angry, neutral) • Create a bilinear model, factoring expressions and visemes into two separate components. • Synthesize new facial movements with any expression and viseme.

  7. Related Work: Bilinear Model • Problems • Requires a full Cartesian product of facial expressions and visemes. • Does not generate new expressions that are not in the source data set. • Does not change the facial characteristics (identity). • Pres Videos\Jerry\moodswings.mov

  8. Related Work: Multilinear Model • D. Vlasic, M. Brand, H. Pfister, & J. Popovic • Face Transfer with Multilinear Models • Capture videos of 16 actors, each performing 5 visemes under 5 different expressions. • Create a multilinear model, factoring expressions, visemes, and identity into three separate components. • Synthesize new facial movements with any expression, viseme, and identity

  9. Related Work: Multilinear Model • Problems • Requires a full Cartesian product of facial expressions, visemes, and identity. • Limitations in the missing data imputation process. • Does not generate new expressions that are not in the source data set. • Pres Videos\Jerry\vlasic-2005-ftm-sing.mp4

  10. Methods • Acquire and Categorize • Learn • Generate

  11. Vertex Correspondence User “rates” attributes of each face Video Acquire and Categorize • Three data sets are needed to fill the model space • Set of many neutral faces • Set of one face in many poses • Set of Visemes with reference face

  12. Acquire and Categorize

  13. Expression deformation Type deformation Learn • Analyze each triangle and transform type separately Reference Face Viseme deformation

  14. Low-dimensional subspace (PCA) polygons individuals Learn • Compare each pose to reference face • Principle Component Analysis (PCA) • Apply to each axis of variation • Analyze transformation of every face in mesh • Infer variation of single attribute from combination of many

  15. Generate • Same sliders as categorization UI • Generate any combination of attributes • Runs in real-time

  16. Results

  17. Conclusion • Realistic face poses from real-world basis data • Arbitrary faces from sparse data set • Future Work • Use high res data to drive low res morphing • Incorporate more biologically accurate face model

More Related