1 / 51

3D Face Reconstruction from Monocular or Stereo Images.

3D Face Reconstruction from Monocular or Stereo Images. . Thomas Vetter. Universit y of Basel. Switzerland . http://gravis.cs.uni bas.ch. Change Your Image . Analysis by Synthesis. model parameter. Analysis. Image Model. Synthesis. Image. 3D World.

Angelica
Download Presentation

3D Face Reconstruction from Monocular or Stereo Images.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Face Reconstruction from Monocular or Stereo Images. Thomas Vetter University ofBasel Switzerland http://gravis.cs.unibas.ch

  2. Change Your Image ...

  3. Analysis by Synthesis model parameter Analysis Image Model Synthesis Image 3D World Image Description

  4. Approach: Example based modeling of faces 2D Image 3D Face Models 2D Image 2D Face Examples = w1 * + w2 * + w3 * + w4 * +. . .

  5. Morphing 3D Faces 1 __ 2 3D Blend 3D Morph 1 __ = + 2

  6. Shape and Texture Vectors Reference Head Example i

  7. Surface registration: Which representation?

  8. Registration in different representations • Curvature Guided Level Set Registration using Adaptive Finite ElementAndreas Dedner, Marcel Lüthi, Thomas Albrecht and Thomas Vetter IN: Proceedings DAGM'07: Heidelberg 2007 • Optimal Step Nonrigid ICP Algorithms for Surface RegistrationBrian Amberg, Sami Romdhani and Thomas Vetter IN: Proceedings, CVPR'07, Minneapolis, USA 2007. • A Morphable Model for the Synthesis of 3D Faces. Volker Blanz and Thomas VetterIN: SIGGRAPH'99 Conference Proceedings, 187-194 • Implicit: • Triangulated: • Parameterized:

  9. Database of 3D Faces

  10. Vector space of 3D faces. • A Morphable Model can generate new faces. a1 * + a2 * + a3 * + a4 * +. . . = b1 * + b2 * + b3 * + b4 * +. . .

  11. Manipulation of Faces Modeler

  12. Continuous Modeling in Face Space Caricature Original Average Anti Face

  13. Modelling the Appearance of Faces A face is represented as a point in face space. • Which directions code for specific attributes ?

  14. Learning from Labeled Example Faces Fitting a regression function

  15. Facial Attributes Weight Subjective Attractiveness Gender Original

  16. 3D Shape from Images Face Analyzer Input Image 3D Head

  17. Matching a Morphable 3D-Face-Model • R = Rendering Function • = Parameters for Pose, Illumination, ... Find optimal a, b, r !

  18. Automated Parameter Estimation Ambient: intensity, color Parallel: intensity, color,direction Color: contrast, gains, offsets • Face Parameters • 150 shape coefficients ai • 150 texture coefficients bi head position head orientation focal length • 3D Geometry • Light and Color

  19. Image Formation: at each Vertex k • Rigid Transformation • Normals • Phong Illumination • Perspective Projection • Color Transformation • bi • ai

  20. Error Function • Image difference (pixel intensity cost function) • Plausible parameters • Minimize

  21. animation by Volker Blanz.

  22. Using Multiple Features

  23. Which Feature to use? someEdge detector

  24. Edge Feature • Rigid Transformation • Normals • Phong Illumination • Perspective Projection • Color Transformation • bi • ai

  25. Edge Fitting Results

  26. Multi-Features Fitting Algorithm

  27. Multi-Features Fitting Algorithm 1 2 3 4 5 At stage 4:

  28. Recognition from Images Complex Changes in Appearance Images: CMU-PIE database.

  29. 3D Computer Graphics

  30. Correct Identification “1 out of 68” (%) • 99.5 • 83.0 • 97.8 • 86.2 • 79.5 • 85.7 • 92.3 • 95.0 • 89.0 • gallery • front • side • profile • probe • front • 99.8 • side • 99.9 • profile • 98.3 • total CMU-PIE database: 4488 images of 68 individuals 3 poses x 22 illuminations = 66 images per individual

  31. Reanimation of Images V. Blanz, C. Basso, T. Poggio & T. Vetter Reanimating Faces in images and Video Proc. of Eurographics 2003

  32. Expression Transfer Fitting Fitting Rendering

  33. Analysis by Synthesis model parameter • Image Processing • Edges • Highlights • Segmentation • …… Image Model some ║ ║X Analysis Synthesis 3D World Image Description Image

  34. Segmenting hair a general requirement ? No outlier detection with outlier mask

  35. Skin segmentation • We need to mask out non-skin regions / outliers • 3DMM is not sufficient

  36. Shading Problem • Skin regions contain strong intensity gradients that make a segmentation difficult!

  37. Illumination Compensation

  38. Illumination Compensation • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007 Local fitting

  39. Segmentation Results GrabCut • Skin Detail Analysis for Face RecognitionJean Sebastian Pierrard , Thomas Vetter CVPR 2007 Thresholding

  40. Try New Hairstyles 3D Angle, Position Illumination, Foreground, Background 3D Shape and Texture

  41. More Hairstyles 3D Shape and Texture 3D Angle, Position Illumination, Foreground, Background

  42. Using more than a single image ? Reconstructing High Quality Face-Surfaces using Model Based Stereo Brian Amberg, Andrew Blake, Andrew Fitzgibbon, Sami Romdhani and Thomas Vetter  IN: Proceedings ICCV 2007 Rio de Janeiro, Brazil

  43. Model Based Stereo

  44. Model Based Stereo

  45. Silhouette Term

  46. Colour Difference Term

  47. Results

  48. Results

  49. Results on Flash Data Ground Truth Monocular Stereo

More Related