1 / 30

Shape from Contours and Multiple Stereo A Hierarchical, Mesh-Based Approach

Shape from Contours and Multiple Stereo A Hierarchical, Mesh-Based Approach. Hendrik K ü ck, Wolfgang Heidrich, Christian Vogelgsang. The goal. The goal. Our approach. Perform reconstruction using Color Object’s silhouettes Create initial approximation based on silhouettes (Visual Hull)

tammy
Download Presentation

Shape from Contours and Multiple Stereo A Hierarchical, Mesh-Based Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Shape from Contours and Multiple StereoA Hierarchical, Mesh-Based Approach Hendrik Kück, Wolfgang Heidrich, Christian Vogelgsang

  2. The goal

  3. The goal

  4. Our approach • Perform reconstruction using • Color • Object’s silhouettes • Create initial approximation based on silhouettes (Visual Hull) • Improve mesh using color information within an optimization approach

  5. Talk outline • Image data & preprocessing • Visual Hull • Definition • Computation as a triangle mesh • Image based mesh optimization • Results

  6. Calibrated images Foreground/ background segmentation Image data & preprocessing

  7. Silhouette + camera information: Silhouette Cone Completely contains real object Shape from Silhouettes

  8. Silhouette + camera information: Silhouette Cone Completely contains real object Silhouettes can have holes Shape from Silhouettes

  9. Visual Hull • Definition: Largest volume that produces the same silhouettes as the object • Construction:Intersection of the silhouette cones

  10. Visual Hull

  11. Computing the Visual Hull • Extract Visual Hull using Extended Marching Cubes algorithm(Kobbelt, Botsch, Schwanecke, Seidel, 2001) • Vertices lie exactly on isosurface • Can preserve sharp discontinuities • Requires signed directed distance functionD(VH,x,d) • Distance from x to VH surface along direction d • Positive, if x outside VH, negative if inside

  12. Computing the Visual Hull • D(VH,x,d) = D(  SCi ,x,d) = maxi D(SCi ,x,d) • D(SCi ,x,d)can be efficiently computed in image space

  13. Computing the Visual Hull 14000 triangles 1400 triangles

  14. The Optimization Stage • Evolve triangle mesh into a shape that is • Consistent with color in the images • Consistent with silhouettes in images • Free from self-intersections • Smooth (low curvature) • Composed of well shaped triangles

  15. Optimization • Only geometry is optimized, not topology • 3 Nv degrees of freedom • Global optimization hopeless • Use local per-vertex optimizations • Locally minimize energy function E(vi) using 3D Simplex Method • Iterate over vertices

  16. Per Vertex Energy Function color triangle shape self penetration silhouette local curvature

  17. Color consistency • Assumption: Lambertian reflectance • Surface points appear the same from all viewing directions • Points on the real surface will project onto pixels of the same color in all images that see them • Projecting images onto the mesh • If surface is consistent, the color from different images will match • Color cost term  color mismatch (L2 norm)

  18. Use OpenGL for color projection Projective texture mapping And for determining visibility Shadow mapping Color consistency v v v v

  19. Set up orthographic view of triangle fan around vertex v Choose scale according to sampling rate in the images Render fan to get samples of color and occlusion, once for each (relevant) image Color consistency

  20. Silhouette consistency • No part of the geometry may project outside any silhouette (must stay inside the visual hull) • Strongly penalize distance outside Visual Hull¼ maxi ( distance outside silhouettes in image i ) • Geometry may be smaller than Visual Hull • Where color does not provide enough information, use Visual Hull as fallback solution • Slightly penalize distance inside Visual Hull ¼mini ( distance inside silhouettes in image i )

  21. Silhouette consistency • Encode distance from silhouette in alpha channel of OpenGL textures • Project onto triangle fan along with color & visibility

  22. Multi-Resolution Optimization • Local minima are a problem,especially when • Triangle size is small compared to geometric error • Texture frequencies are high compared to geometric error • Solution: Perform optimization at multiple resolutions

  23. Start with low resolution Visual Hull mesh Multi-Resolution Optimization

  24. Start with low resolution Visual Hull mesh Optimize until convergence Multi-Resolution Optimization

  25. Start with low resolution Visual Hull mesh Optimize until convergence Subdivide & optimize more Multi-Resolution Optimization

  26. Start with low resolution Visual Hull mesh Optimize until convergence Subdivide & optimize more Do it again, … Multi-Resolution Optimization

  27. Final Results before

  28. Final Results before after

  29. Final Results before after

  30. Thank you

More Related