1 / 49

Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot

Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot. Yinxiao Li and Stanley T. Birchfield Department of Electrical and Computer Engineering Clemson University { yinxial, stb }@clemson.edu. Motivation. Goal: Segment the floor in a single corridor image. Why?

josephpruitt
Download Presentation

Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot Yinxiao Li and Stanley T. Birchfield Department of Electrical and Computer Engineering Clemson University { yinxial, stb }@clemson.edu

  2. Motivation Goal: Segment the floor in a single corridor image • Why? • obstacle avoidance • mapping • autonomous exploration and navigation

  3. Motivation Goal: Segment the floor in a single corridor image • Why? • obstacle avoidance • mapping • autonomous exploration and navigation

  4. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  5. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  6. Previous Work Free space detection Floor detection Apply planar homographies to optical flow vectors (Kim 2009, Zhou 2006) Stereo homographies (Fazl-Ersi 2009) Geometric reasoning (Lee 2009) • Combination of color and histogram (Lorigo 1997) • Optical flow (Stoffler 2000, Santos-Victor 1995) • Stereo matching (Sabe 2004) • Limitations • Multiple images for motion • Multiple cameras for stereo • Require different colors for floor and wall • Computational efficiency • Calibrated cameras • Assume ceiling visible Our contribution: Segment the floor in real time using a single image captured by a low-height mobile robot

  7. Challenging problem: Reflections • Also: • variety of poses (vanishing point, ceiling not always visible) • sometimes wall and floor color are nearly the same

  8. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  9. Douglas-Peucker Algorithm • Purpose: Reduce the number of points in a curve (polyline) Algorithm: • Connect farthest endpoints • Repeat • Find maximum distance between the original curve and the simplified curve • Split curve at this point Until max distance is less than threshold David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  10. Douglas-Peucker Algorithm • Purpose: Reduce the number of points in a curve (polyline) Algorithm: • Connect farthest endpoints • Repeat • Find maximum distance between the original curve and the simplified curve • Split curve at this point Until max distance is less than threshold

  11. Douglas-Peucker Algorithm • Purpose: Reduce the number of points in a curve (polyline) Algorithm: • Connect farthest endpoints • Repeat • Find maximum distance between the original curve and the simplified curve • Split curve at this point Until max distance is less than threshold

  12. Douglas-Peucker Algorithm • Purpose: Reduce the number of points in a curve (polyline) Algorithm: • Connect farthest endpoints • Repeat • Find maximum distance between the original curve and the simplified curve • Split curve at this point Until max distance is less than threshold

  13. Douglas-Peucker Algorithm • Purpose: Reduce the number of points in a curve (polyline) Algorithm: • Connect farthest endpoints • Repeat • Find maximum distance between the original curve and the simplified curve • Split curve at this point Until max distance is less than threshold

  14. Douglas-Peucker Algorithm • Purpose: Reduce the number of points in a curve (polyline) Algorithm: • Connect farthest endpoints • Repeat • Find maximum distance between the original curve and the simplified curve • Split curve at this point Until max distance is less than threshold

  15. dallowed Modified Douglas-Peucker Algorithm original algorithm: dallowed is constantmodified algorithm: dallowed is given by half-sigmoid function original modified

  16. Detecting Line Segments (LS)

  17. Detecting Line Segments (LS) Compute Canny edges

  18. dallowed Detecting Line Segments (LS) • Compute Canny edges • Modified Douglas-Peucker algorithm to detect line segments • Vertical LS: slope within ±5° of vertical direction • Horizontal LS: Slope within ±45° of horizontal direction

  19. dallowed Detecting Line Segments (LS) • Compute Canny edges • Modified Douglas-Peucker algorithm to detect line segments • Vertical LS: slope within ±5° of vertical direction • Horizontal LS: Slope within ±45° of horizontal direction • 3. Pruning line segments (320x240) • Vertical LS: minimum length 60 pixels • Horizontal LS: minimum length 15 pixels • Vanishing point (height selection):The vanishing point is computed as the mean of the intersection of pairs of non-vertical lines. It is used for throwing away horizontal line segments coming from windows, ceiling lights, etc.

  20. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  21. Score Model Weights horizontal line Bottom Score Structure Score Homogeneous Score • is assigned to each horizontal line

  22. Score Model – Structure Given a typical corridor image |I(x,y)| > T1(How to set threshold T1?) |I(x,y)| > T1

  23. Score Model – Structure Given a typical corridor image |I(x,y)| > T1(gradient magnitude)(How to set threshold T1?) I(x,y) < T2(image intensity) (How to set threshold T2?) |I(x,y)| > T1 and I(x,y) < T2

  24. Score Model – Structure Given a typical corridor image |I(x,y)| > T1(gradient magnitude)(How to set threshold T1?) I(x,y) < T2(image intensity) (How to set threshold T2?) We applied SVM to 800 points in 200 images: |I(x,y)| > T1 and I(x,y) < T2

  25. Score Model – Structure Given a typical corridor image |I(x,y)| > T1(gradient magnitude)(How to set threshold T1?) I(x,y) < T2(image intensity) (How to set threshold T2?) Approximate using two thresholds: |I(x,y)| > T1 and I(x,y) < T2

  26. Score Model – Structure Given a typical corridor image |I(x,y)| > T1(gradient magnitude) I(x,y) < T2(image intensity) Now threshold original image using TLC (average gray level of pixels satisfying #1 and #2) I(x,y) < TLC

  27. Score Model – Structure Given a typical corridor image |I(x,y)| > T1(gradient magnitude) I(x,y) < T2(image intensity) Now threshold original image using TLC (average gray level of pixels satisfying #1 and #2) Compute the chamfer distance between the line segments and structure blocks

  28. Score Model – Structure Comparison of different threshold methods Ridler-Calvard Otsu Our method is able to remove the spurious pixels on the floor caused by reflections or shadows Our method

  29. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  30. Score Model – Bottom Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary

  31. Score Model – Bottom Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary

  32. Score Model – Bottom Connect the bottom points of consecutive vertical line segments to create “bottom” wall-floor boundary Compute the distance of each horizontal line segment to the “bottom” wall-floor boundary (red circled horizontal line segments indicates a positive contribution)

  33. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  34. Score Model – Homogeneous Idea: The floor tends to have larger regions (due to decorations, posters, windows on the wall). Algorithm: Color-based segmentation of the image (using Felzenszwalb-Huttenlocher’s minimum spanning tree algorithm) For each horizontal line segment, compute size of region just below segment size of largest segment

  35. Score Model – Homogeneous Idea: The floor tends to have larger regions (due to decorations, posters, windows on the wall). Algorithm: Color-based segmentation of the image (using Felzenszwalb-Huttenlocher’s minimum spanning tree algorithm) For each horizontal line segment, compute size of region just below segment size of largest segment

  36. Segmenting the floor Normalize the scores and sum Threshold the final score: Connect the remaining line segments and extend the endpoints to the edges of the image

  37. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  38. Evaluation Criterion • Wall- floor Boundary • Green line: Detected wall-floor boundary • Red line: Ground truth

  39. Evaluation Criterion • Wall- floor Boundary • Green line: Detected wall-floor boundary • Red line: Ground truth • Area • Blue shade area: misclassified area • Red shade area: ground truth floor area • Error Rate • Segmentation is considered successful if rerr < 10%

  40. Sample Results 89.1% success on database of 426 images:

  41. Sample Results Images downloaded from the internet:

  42. Sample Results Lee’s Original Image Ours Some successful results on failure examples from Lee et al. 2008 D. C. Lee, M. Hebert, and T. Kanade. “Geometric Reasoning for Single Image Structure Recovery”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.

  43. Sample Results Textured wall Checkered floor Dark image Bright lights Failure examples

  44. Sample Results Video clip 1

  45. Sample Results Video clip 2

  46. Outline • Previous Work • Detecting Line Segments • Score Model for Evaluating Line Segments • Structure Score • Bottom Score • Homogeneous Score • Experimental Results • Conclusion

  47. Conclusion • Summary • Edge-based approach to segmenting floors in corridors • Correctly handles specular reflections on floor • Nearly 90% of the corridor images in our database can be correctly detected. • Speed: approximately 7 frames / sec • Future work • Speed up the current algorithm • Improving score model by add more visual cues(highly textured floors, low resolution, or dark environment) • Use floor segmentation for mapping

  48. Acknowledgement • Clemson Computer Vision Group reviewers • Zhichao Chen • Vidya Murali • Anonymous reviewers

  49. Thanks! Questions? Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot

More Related