230 likes | 237 Views
This paper explores a unified approach for analyzing and synthesizing images of faces, using example-based modeling and optimization techniques. It discusses various methodologies for modeling facial attributes, learning from labeled examples, and optimizing face parameters. The paper also presents future challenges in the field of computer vision and computer graphics.
E N D
Learning the Appearance of Faces:A Unifying Approach for the Analysis and Synthesis of Images. Thomas Vetter University of Freiburg Germany http://graphics.informatik.uni-freiburg.de
Computer Vision & Computer Graphics Vision ( image) parameters image Graphics ( parameters ) -1 G( image ) Parameters | G(p) - I |2 = minParameters Computer Graphics can help to solve Computer Vision!
Analysis by Synthesis model parameter Analysis Image Model Synthesis Image 3D World Image Description
Synthesis of Faces Database Morphable Face Model Face Analyzer 3D Head Modeler Result Input Image
Approach: Example based modeling of faces 2D Image 3D Face Models = w1 * + w2 * + w3 * + w4 * +. . .
Cylindrical Coordinates h f h f red(h,f) green(h,f) blue(h,f) radius(h,f)
Morphing 3D Faces 1 __ 2 3D Blend 3D Morph 1 __ = + 2
Correspondence: A two step process! 2nd Example Example Reference • Correspondence between • two examples ( Optical Flow like algorithms). • many examples ( Morphable Model )
Vector space of 3D faces. • A Morphable Model can generate new faces. a1 * + a2 * + a3 * + a4 * +. . . = b1 * + b2 * + b3 * + b4 * +. . .
Manipulation of Faces Modeler
Modelling in Face Space Caricatur Original Average
Modelling the Appearance of Faces A face is represented as a point in face space. • Which directions code for specific attributes ?
Learning from Labeled Example Faces Fitting a (linear) regression function
Facial Attributes Subjective Attractiveness Weight Original
Transfer of Facial Expressions - = Smile Novel Face: + Smile = Originals:
Facial Expressions Original
3D Shape from Images Face Analyzer Input Image 3D Head
Matching a Morphable 3D-Face-Model Optimization problem! a1 * + a2 * + a3 * + a4 * +. . = R b1 * + b2 * + b3 * + b4 * +. .
Error Function • Image difference • Plausible parameters • Minimize
Optimization Strategies • Difference Decomposition • Stochastic Gradient Decent
Future Challenges • Which Object Classes are linear ? • How to built them automatically?