1 / 16

Exploiting Segmentation for Robust 3D Object Matching

Exploiting Segmentation for Robust 3D Object Matching . Michael Krainin, Kurt Konolige , and Dieter Fox. Shape-based Pose Quality Metrics. Given an object model (triangle mesh) and a depth map, Should not assume any object texture Should be robust to clutter and occlusion. Outline.

oria
Download Presentation

Exploiting Segmentation for Robust 3D Object Matching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exploiting Segmentation for Robust 3D Object Matching Michael Krainin, Kurt Konolige, and Dieter Fox

  2. Shape-based Pose Quality Metrics • Given an object model (triangle mesh) and a depth map, • Should not assume any object texture • Should be robust to clutter and occlusion

  3. Outline • Progression of quality metrics • Iterative Closest Point • Beam-based Sensor Models • Segmentation Sensor Model • Key Contributions • Gradient-based optimization for beam-based models • Improves on ICP by reasoning about free-space • Novel sensor model framework • Uses segmentation to relax beam-independence assumptions • Explicitly reasons about surface extents

  4. ICP Error Metric Mean squared distance for pairs of closest points [Besl& McKay‘92] No notion of amount of model or scene explained by the correspondences Discards viewpoint/free-space information

  5. Beam-Based Sensor Models x x x x x p(pixel | model, pose) x x l 0 MeasuredDepth – RenderedDepth Maximize data likelihood: Typically assume pixel independence:

  6. Beam Model Sensor Models • Beam-based sensor model properties: • Consider viewpoint • Prefer to explain more pixels using the model • Use in pose estimation: • 2D particle-based localization [Thrun et al. ‘05] • Coarse to fine grid search for body tracking [Ganapathi et al. ‘10] • Annealed particle filters for vehicle detection and tracking [Petrovskaya & Thrun ‘09] • Has not been used with gradient-based optimization

  7. Beam Model Optimization observed rendered ICP Beam Optimization • We propose a gradient-based optimization • Levenberg-Marquardt using the g2o graph optimization framework [Kümmerle et al. ‘11] • Error function evaluations by re-rendering • OpenGL rendering, CUDA sensor model evaluation • 1 ms per evaluation on a mid-range graphics card • ~500 evaluations per g2o optimization

  8. Surface Extents Can still match to the wrong surface Idea: Use size of surfaces to rule out matches like these

  9. Segmentation (Over-)Segmentation as an estimate of surface extents Connected components using depths and normals

  10. Segmentation Sensor Model Usual beam-based sensor model: Segmentation model: Allows consistent classification of entire segment as being generated from the model or not

  11. Segmentation Sensor Model Let S be a partition (segmentation) of D. Then given model M and pose T, Let mi be an indicator for whether Si was generated from M Segment pixels conditionally independent given classification

  12. Error Functions x Standard Beam-Based Model Segmentation Model

  13. Test Data Recorded 46 cluttered scenes with ground truth poses

  14. Pose Estimation Results Optimization algorithms: Random restart evaluation functions (upper bounded by 93% from previous result):

  15. Robustness to Segmentation Granularity Spam 1 of 5 0 degrees x x x (Fixed to 8 degrees in other experiments) 12 degrees 8 degrees

  16. Conclusions • Contributions • Novel formulation for beam-based sensor models • Demonstration of gradient-based approach for optimizing beam-based sensor models • All code and test data freely available • Future Work • Detecting failed segmentations • Extensions such as color and normal direction • Directly optimizing the segmentation model

More Related