1 / 21

Robust Stereo Fitting with Occlusion Handling: A Model-Based Approach

Explore robust stereo fitting using 3D morphable models (3DMMs) with occlusion handling for improved accuracy. Leveraging joint shape and texture stereo fitting, this approach enhances results under various occlusion scenarios. The methodology involves an optimization procedure and multiple experiments to demonstrate its efficacy.

robinsonray
Download Presentation

Robust Stereo Fitting with Occlusion Handling: A Model-Based Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Model-Based Stereo with Occlusions Fabiano Romeiro and Todd Zickler TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAAAAA

  2. Introduction Varying illumination Varying pose Occlusions Varying expressions

  3. Introduction Past Work: Image-based For example: Eigenfaces [Turk and Pentland, 1991] Fisherfaces [Belhumeur et al, 1997]

  4. 3D Morphable Models (3DMMs) 2D+3D AAMs 2D AAMs [Cootes et al, 1998; Baker et al, 2004; Mathews et al, 2004; Gross et al, 2006] [Xiao et al, 2004] [Blanz et al, 1999; Blanz et al, 2003; Blanz et al, 2005; Smet et al, 2006] Introduction Past Work: Model-based

  5. 3D Morphable Models (3DMMs) Pros • - Self-occlusion handled by model itself • - Allows direct modeling of illumination Cons - Difficult and expensive fitting process [Blanz et al, 1999; Blanz et al, 2003; Blanz et al, 2005; Smet et al, 2006] Introduction Past Work: 3DMMs

  6. - Stereo based cost [Fransens et al, 2005] • Stereo fitting with both shape and texture • →Robust to foreign-body occlusions Introduction Past Work: Stereo 3DMMs - Texture model not needed Our work • → Improved Accuracy

  7. Outline • 3DMM Background • Joint Shape and Texture Stereo Fitting • Handling Occlusions • Conclusions

  8. Basis for shape - Basis for texture Background 3DMMs Vectorization of laser scans: PCA performed: [Blanz and Vetter, 1999]

  9. [Blanz and Vetter, 1999] Background 3DMMs Representation of face shape and texture: Prior probabilities on the coefficients:

  10. Stereo Match

  11. Texture Match

  12. Joint Shape and Texture Stereo Fitting of 3DMMs

  13. Robust Stereo Fitting of 3DMMs

  14. Optimization Procedure Initial fit - Fit Shape, Pose to minimize reprojection error on selected feature points - Rough initial estimates of Shape and Pose Optimization procedure 4 experiments Stereo and texture Stereo Robust Stereo and texture Robust Stereo

  15. Results First 2 experiments: Stereo and Texture vs. Stereo 480 recovered shape models (60 individuals, 8 poses) K.U. Leuven Stereo face database [Fransens et al, 2005]

  16. Results Qualitative Results Stereo and Texture Stereo

  17. Results Qualitative Results Stereo+texture Stereo

  18. Results Quantitative Results

  19. Results Under Occlusions Half-Occlusion Full Occlusion near Full-Occlusion far

  20. Robust Stereo Robust S+T Results Under Occlusions Input Robust Stereo Robust S+T Shape Estimate Occlusion Map

  21. Conclusions Robust stereo fitting of 3DMMs - Uses both stereo constraint, texture information • Increased accuracy of fit - Ability to handle occlusions Future Work - More sophisticated stereo matching term - Different feature spaces - Break model into segments respecting occlusion boundaries

More Related