1 / 44

Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot

Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot. Yinxiao Li and Stanley T. Birchfield The Holcombe Department of Electrical and Computer Engineering Clemson University { yinxial, stb }@ clemson.edu. Typical Corridor Images. Low contrast. Common Reflections.

Download Presentation

Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Image-Based Segmentation of Indoor Corridor Floors for a Mobile Robot Yinxiao Li and Stanley T. Birchfield The Holcombe Department of Electrical and Computer Engineering Clemson University { yinxial, stb }@clemson.edu

  2. Typical Corridor Images Low contrast Common Reflections Specular Reflections

  3. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  4. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  5. Previous Work Free space detection Floor detection Previous approaches Apply planar homographies to optical flow vectors (Kim 2009, Zhou 2006) Stereo homographies (Fazl-Ersi 2009) Geometric reasoning (Lee 2009) Limitations Assume ceiling visible Computational efficiency Calibrated cameras Previous approaches • Stereo Matching (Sabe 2004) • Optical Flow (Stoffler 2000, Santos-Victor 1995 • Combination of color and histogram (Lorigo 1997) Limitations • Need motion • require different colors for floor and wall • Computational efficiency • Calibrated cameras

  6. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  7. Douglas-Peucker Algorithm • An algorithm that: • Reduces the number of points in a curve that is approximated by a series of points • Focus on maximum distance between the original curve and the simplified curve • Finds a similar curve with fewer points http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  8. Douglas-Peucker Algorithm • An algorithm that: • Reduces the number of points in a curve that is approximated by a series of points • Focus on maximum distance between the original curve and the simplified curve • Finds a similar curve with fewer points http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  9. Douglas-Peucker Algorithm • An algorithm that: • Reduces the number of points in a curve that is approximated by a series of points • Focus on maximum distance between the original curve and the simplified curve • Finds a similar curve with fewer points http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  10. Douglas-Peucker Algorithm • An algorithm that: • Reduces the number of points in a curve that is approximated by a series of points • Focus on maximum distance between the original curve and the simplified curve • Finds a similar curve with fewer points http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  11. Douglas-Peucker Algorithm • An algorithm that: • Reduces the number of points in a curve that is approximated by a series of points • Focus on maximum distance between the original curve and the simplified curve • Finds a similar curve with fewer points http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  12. Douglas-Peucker Algorithm • An algorithm that: • Reduces the number of points in a curve that is approximated by a series of points • Focus on maximum distance between the original curve and the simplified curve • Finds a similar curve with fewer points http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm David Douglas & Thomas Peucker, "Algorithms for the reduction of the number of points required to represent a digitized line or its caricature", The Canadian Cartographer 10(2), 112–122 (1973)

  13. dallowed Modified Douglas-Peucker Algorithm original algorithm: dallowed is constantmodified algorithm: dallowed is given by half-sigmoid function original modified

  14. Detecting Line Segments (LS)

  15. Detecting Line Segments (LS) Compute Canny edges

  16. dallowed Detecting Line Segments (LS) • Compute Canny edges • Modified Douglas-Peucker algorithm to detect line segments • Vertical LS: slope within ±5° of vertical direction • Horizontal LS: Slope within ±45° of horizontal direction

  17. dallowed Detecting Line Segments (LS) • Compute Canny edges • Modified Douglas-Peucker algorithm to detect line segments • Vertical LS: slope within ±5° of vertical direction • Horizontal LS: Slope within ±45° of horizontal direction • 3. Pruning line segments (320x240) • Vertical LS: length 60 pixels • Horizontal LS: length 15 pixels • Vanishing point (height selection):The vanishing point is computed as the mean of the intersection of pairs of non-vertical lines

  18. dallowed Detecting Line Segments (LS) • Compute Canny edges • Modified Douglas-Peucker algorithm to detect line segments • Vertical LS: slope within ±5° of vertical direction • Horizontal LS: Slope within ±45° of horizontal direction • 3. Pruning line segments (320x240) • Vertical LS: length 60 pixels • Horizontal LS: length 15 pixels • Vanishing point (height selection):The vanishing point is computed as the mean of the intersection of pairs of non-vertical lines

  19. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  20. Score Model Weights • is assigned to each horizontal line segment and we use a threshold to select those ones that beyond the Structure Score Homogeneous Score Bottom Score

  21. Score Model – Structure Structure =>Threshold=? Given a typical corridor image

  22. Score Model – Structure Structure =>Threshold=? Given a typical corridor image Find gradient magnitude greater than a threshold T1

  23. Score Model – Structure Structure =>Threshold=? • Given a typical corridor image • Find gradient magnitude greater than a threshold T1 • Find gradient magnitude greater than a threshold T1 and intensity less than a threshold T2. The figure shows the mask pixels image. • We have a training data over 200 images to determine T1 and T2 by SVM over 800 points.

  24. Score Model – Structure Structure =>Threshold= TLC Given a typical corridor image Find gradient magnitude greater than a threshold T1 Find gradient magnitude greater than a threshold T1 and intensity less than a threshold T2. The figure shows the mask pixels image. Compute the average intensity of the make pixels image over original image and get a threshold value TLC

  25. Score Model – Structure Structure =>Threshold= TLC Given a typical corridor image Find gradient magnitude greater than a threshold T1 Find gradient magnitude greater than a threshold T1 and intensity less than a threshold T2. The figure shows the mask pixels image Compute the average intensity of the make pixels image over original image and get a threshold value TLC Compute the distance between the line segments and structure blocks

  26. Score Model – Structure Comparison of different threshold methods Ridler-Calvard Otsu Our method is able to get rid of the spurious pixels on the floor caused by reflections or shadows Our method

  27. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  28. Score Model – Bottom Using vertical and horizontal line segments Connect each bottom point of the vertical line segments as a “bottom” wall-floor boundary

  29. Score Model – Bottom Using vertical and horizontal line segments Connect each bottom point of the vertical line segments as a “bottom” wall-floor boundary Compute the distance of each horizontal lines segments to the “bottom” wall-floor boundary (red circled horizontal line segments indicates a positive contribution)

  30. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  31. Score Model – Homogeneous Using efficient graph-based segmentation ( Felzenszwalb [5]), we may get a bigger segmented area in floor area, due to decorations, posters, windows etc. on the wall. Compute the ratio of area of the segmented region below a horizontal line segment over the biggest segmented region in the image. The ratio is assigned to each horizontal line segment as a homogenous score.

  32. Detecting wall-floor boundary Given a threshold to the sum of each score Connect the remaining line segments and extend the endpoints to the edges of the image Future work: A larger database and AdaBoost algorithm to generate weights for each score

  33. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  34. Evaluation Criterion • Wall- floor Boundary • Green line: Detected wall-floor boundary • Red line: Ground truth

  35. Evaluation Criterion • Wall- floor Boundary • Green line: Detected wall-floor boundary • Red line: Ground truth • Area • Blue shade area: misclassified area • Red shade area: ground truth floor area • Error Rate

  36. Sample Results Corridor images from our database

  37. Sample Results Images download from internet.

  38. Sample Results Lee’s Original Image Ours Some successful results of failure examples from Lee[1].

  39. Sample Results Texture wall Checked floor Overly dark Bright lights Failure examples

  40. Sample Results Video clip 1

  41. Sample Results Video clip 2

  42. Outline • Previous work • Detecting Line Segments • Score model for Evaluating Line Segments • Structure Score • Bottom Score • Homogenous Score • Experimental Results • Conclusion

  43. Conclusion • Summaries • Over 90% of the corridor images in our database can be correctly detected. • Speed: around 7 frame/sec. • Weakness: not robust to highly textured floor, low resolution and dark environment. • Future work • Speed up the current algorithm • Improving score model by add more visual cues • Mapping the whole corridor structure for robot navigation • …

  44. Acknowledgement • Clemson Computer Vision Group reviewers • Zhichao Chen • Vidya Murali • Anonymous reviewers

More Related