1 / 27

Accurate, Dense and Robust Multi-View Stereopsis

Accurate, Dense and Robust Multi-View Stereopsis. Yasutaka Furukawa and Jean Ponce. Presented by Rahul Garg and Ryan Kaminsky. Agenda. Problem Statement Multi-view Stereo Taxonomy Algorithm Results Comparison to other works Questions. Problem Statement. Multi-view Stereo

varsha
Download Presentation

Accurate, Dense and Robust Multi-View Stereopsis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Accurate, Dense and Robust Multi-View Stereopsis Yasutaka Furukawa and Jean Ponce Presented by Rahul Garg and Ryan Kaminsky

  2. Agenda • Problem Statement • Multi-view Stereo Taxonomy • Algorithm • Results • Comparison to other works • Questions

  3. Problem Statement • Multi-view Stereo • Dense shape reconstruction from multiple views = + + +

  4. Multi-View Stereo Taxonomy S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski • Scene Representation • Photoconsistency Measure • Visibility Model • Shape Prior • Reconstruction algorithm • Initialization

  5. Scene Representation • Geometry on 3D grid • Voxels, Level sets • Polygon Mesh • Set of planar facets • Depth Map • Image that stores depth per pixel

  6. Photoconsistency Measure • Definition: Measures visual compatibility of reconstruction with input images • Scene Space • Project part of reconstruction into images, measure closeness • Measures: Variance , sum of squared distances, normalized cross-correlation • Image Space • Use scene geometry to transform image to different view, measure error of predicted vs. actual (prediction error)

  7. Visibility Model • Definition: Views to consider when evaluating photo consistency • Geometric • Explicitly model geometry of the scene • Quasi-Geometric • Approximate geometric reasoning • Outlier based approaches • Treat occlusions as outliers

  8. Shape Prior • Definition: Additional constraints or assumptions about reconstruction • Minimal Surfaces • Level sets, Min-cut • Maximal Surfaces • Voxel coloring, space carving • Local Measures • Assume local smoothness on nearby pixels

  9. Reconstruction Algorithm • Optimize cost function • Voxels, graph cut, level sets, meshes • A set of consistent depth maps • Feature extraction, matching, surface fitting

  10. Initialization • Definition: Constraints on scene geometry • Bounding box or volume • Visual hull • Range of disparity

  11. Overview of Algorithm input imagedetected reconstructed final patches polygonal surface features patches after after expansion from reconstructed the initial and filtering patches matching

  12. Algorithm Block Diagram Initialization Expansion Filter Reconstruction Patch Model Feature Detection

  13. Init • Detect features using Harris Corner and DoG • Feature matching to generate sparse set of patches

  14. Patch Models • R(p): Most closely associated image with p • S(p): Images where p should be visible • T(p): Images where p is truly visible

  15. c(p): from triangulation n(p): Direction of optical ray from c(p) to O β pixels Epipolar line

  16. Normalized Cross Correlation (NCC) where   is the mean of the feature and      is the mean of f(x,y) in the region under the feature. Optimization step: Maximizing the average NCC score

  17. Patch Expansion • Expand patches along tangential planes into empty areas. • Optimize for normal and center and add if photometric constraints are satisfied in at least k images.

  18. Filtering • Analyzing visibility consistency

  19. Filtering (Contd.) • Local smoothness constraint : Remove patches for which proportion of neighboring patches with tangential plane “nearly” parallel is less than ε

  20. Polygonal Surface Reconstruction • Initialize using convex hull of patches • Iteratively deform/snap to the patch model using two kinds of forces • Smoothness term • Photometric Consistency term S : Current surface S* : True surface n(v) : Normal at v Π(v) : Set of patches compatible with v d(v) : Distance between S and S*

  21. Algorithm Taxonomy Categorization • Scene Representation • Depth Map + Mesh • Photoconsistency Measure • NCC • Shape Prior • Assume local smoothness • Reconstruction • Feature extraction , depth maps, optimization over patches • Initialization • None

  22. Results Patch Model Polygonal Surface Model

  23. Results (Contd.)

  24. Results (Contd.) • Evaluation on vision.middlebury.edu Accuracy Measure: Distance d in mm which brings 90% of the reconstruction within ground truth * Old Results

  25. Results (Contd.) • Handle occlusions/obstacles

  26. Similar Approaches • Setup similar to Goesele et al. (ICCV’07) – initialize patches, expand and optimize for position and normal

  27. Questions • Pose the problem as an optimization problem simultaneously accounting for local smoothness, photo consistency, occlusion • Convergence of Expand/Filter – do more iterations lead to better reconstructions? • Occlusion/Outlier handling – results on more datasets • Advantages of patch model – Adaptive Resolution, generalizes to large number of object classes

More Related