Combining Visual and Spatial Appearance for Loop Closure Detection in SLAM

1. Combining Visual and Spatial Appearance for Loop Closure Detection in SLAM Kin Leong Ho, Paul Newman Oxford University Robotics Research Group

2. Motivation • Loop Closing – the task of deciding whether a vehicle has returned to a previously visited area • Popular approaches – nearest neighbour statistical gate, joint compatibility test

3. Image Loop Closure • Closing loops with visually salient features to avoid dependence on global position estimate

4. Closing the loop

5. Image Feature Extraction Process MSER detector Saliency detector

6. Demonstration of wide-baseline stability of visually salient features under perspective distortion and variation in illumination conditions

7. Matching Performance Query Image Tentative Match Similar posters found in the environment. [Newman,Ho ICRA2005] Tentative Match Tentative Match

8. Results from Image Retrieval System

9. Limitations of Image Matching TentativeMatch Query Image Tentative Match • - Repetitive visual artifacts in urban environments such as posters, signs and wall pattern • False triggering of loop closure event based solely on image matching

10. Incorporating Spatial Information • Spatial information can be used to disambiguate visually confusing locations

11. Spatial Descriptors • Reduced a laser scan patch into a set of descriptor • Describe curvature of shape • Describe complexity of shape • Describe spatial configuration of laser scan

12. Segmentation • Laser scan is divided into smaller but sizeable segments • Segments are formed due to break in boundary or occlusions Original Laser Scan Set of Descriptors

13. Cumulative Angular Function • A plot of the cumulative change in turning angle versus the arc length of the segment • Invariant to rotation and translation Turning Angle Arc length of Segment

14. Entropy of CAF • A measure of complexity of segment • Weight descriptors to prefer between complex versus simple shapes CAF Histogram of Turning Angle

15. Inter-Segment Descriptors • Extract critical points: Critical points are points along a segment where there are sharp changes in cumulative angular function • Distances and relative orientations between critical points form links between segments

16. Descriptor Comparison 1 • Angular function disparity – minimum error between two cumulative angular functions

17. Descriptor Comparison 2 • entropy disparity – Kullback-Leiber distance

18. Edge Comparison • Matching of links • Links that are matched are coloured in black • Links that are not matched are coloured in blue

19. Spatial Similarity Score • Shape similarity metric comprises of two parts: shape similarity and spatial similarity

20. Results from Spatial Retrieval System

21. More Results

22. MSER Detector Query Laser Scan Query Image Selected Regions Saliency Detector Segmentation SIFT Descriptor Laser Descriptor Laser Scan Database Image Database Combined Similarity Scores Similarity Measure Similarity Measure

23. Visual Similarity Matrix

24. Spatial Similarity Matrix

25. Combined Similarity Matrix

26. Demonstration

27. Issues • Setting of threshold values • Principled way of combining similarity scores • At present limited to planar environments Current Extensions • Removal of repetitive images by spectral decomposition • Successful Application to 3D laser mapping and SLAM

28. Questions Thank you!