1 / 46

Ground Truth Free Evaluation of Segment Based Maps

This study explores ground truth free evaluation methods for geometric maps, specifically segment-based mapping for robot maps. It compares pose-based and grid-based approaches, proposing a new structure-based evaluation method. By clustering segments and evaluating intra-cluster distances, the study aims to achieve more accurate and consistent geometric representation. The use of mid-level geometric primitives like line segments enhances the mapping process, enabling faster and memory-efficient operations. The study argues for the precision and efficacy of segment-based mapping, emphasizing its ability to capture structural information and improve processing efficiency. It also discusses the importance of carefully designing segment similarity measures and utilizing hierarchical clustering techniques. Overall, the research offers valuable insights into optimizing geometric map evaluation methodologies.

richardperkins
Download Presentation

Ground Truth Free Evaluation of Segment Based Maps

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rolf Lakaemper Temple University, Philadelphia,PA,USA Ground Truth Free Evaluation of Segment Based Maps

  2. An example for evaluation of a robot map based on mid level geometry (and an advertisement for segment based maps) • Examination if evaluation can benefit from different data representation • Ground truth free: we evaluate map consistency

  3. Overview

  4. We are evaluating GEOMETRIC MAPS (not semantic or topological maps). • We are evaluating with respect to geometric consistency (not completeness) • Classically there are two approaches: • Pose based • Grid based

  5. Pose based • Raw data grouped during data acquisition (single scan) • Consistency is measured by correspondence of point groups (single scans) • Groups are determined during data acquisition by visibility from similar poses • Problem: grouping assumption can be violated (ground scans etc)

  6. Grid based • Consistency is measured by distribution of single points • Data representation driven • Problem: does not measure structural consistency – points belonging to certain structures in single scan can end in grid cells of different structures in global map (and hence might change the quality value)

  7. Proposal: • Structure Based Evaluation • Data is grouped using underlying structure of single scans (similar to pose based evaluation, but a more natural grouping principle) • Consistency is measured by distribution of groups (similar to grid based evaluation, but adjusted to groups in environment)

  8. Mid Level Geometric representation is a (small) step towards semantic representation. • Assuming an underlying 'form from function' paradigm, raw data points are grouped by form to clusters of similar function, i.e. groups are assumed to represent single functional entities.

  9. Basic idea: • Global map is a superimposition of single scans, consisting of segments. Single features of the environment are represented by multiple segments. • We Cluster segments; single cluster for single feature. • Evaluate intra cluster distance.

  10. cluster

  11. -- Mid Level Geometry Based Mapping -- • Mid Level Geometry: e.g. geometric primitives, containing more geometric information than simple points. • Line segments, Circles, Rectangles • Splines (of all kinds) In this example, we useline segments

  12. Extraction of primitives is the first step after obtaining the raw single scan. • Further processing is entirely performed on primitives.

  13. Note: We are not limited to evaluation of segment based mapping algorithms. • Every mapping method using single scan matching can be evaluated, if output are corrected poses of single scans.

  14. -- Why Segments? – • 1)The segment based approach captures structural information, which goes significantly further than the information of object presence and location, contained in raw point data. Line segments combine multiple points to a single feature. This creates the properties of segment direction and feature membership.

  15. -- Why Segments? – • 2)The segment based approach is fast. • A typical scan consists of <20 segments, while the number of data points is typically one to two orders of magnitude higher. • This becomes important, when processing algorithms have complexity of >O(n)

  16. -- Why Segments? – • 3)The segment based approach is memory efficient, each line segment consists of two endpoints.

  17. -- Why Segments? – • 4)The segment based approach is precise. • Segment endpoints don't have to be adjusted to a resolution parameter, hence there are no quantization errors. Resolution parameters are introduced in point based algorithms e.g. when occupancy grids are used.

  18. Similarity Measure

  19. To process segments (or any other primitives), we need a segment-distance measure, or segment-similarity measure. • These measures must be carefully designed!

  20. Examples:

  21. note: usually, these similarity measures do not obey the metric axioms, an example case of our measure: • D(s1,s2)=0 (although s1 != s2) • Such a measure does not create a metric space, which limits the selection of processing algorithms.

  22. Clustering

  23. For the clustering, agglomerative hierarchical clustering in ’single’ mode is utilized. This method seeks to build a bottom up hierarchy of clusters, starting with each segment being a single cluster, ending in a single cluster containing all segments. • Pairs of clusters are merged as one moves up the hierarchy. • Cluster distances are determined by the segment similarity measure.

  24. Hierarchical clustering is a well understood ‘classic’ algorithm. • Does not need a metric space (in contrast to e.g. k-means) • Problem: where to stop (where to cut links in the dendrogram)?

  25. Confidence Measure

  26. Confidence measure evaluates intra cluster consistency • A standard measure for intra cluster consistency was already used as stop criterion for clustering • Stop criterion for clustering is fast, but has certain drawbacks with respect to outliers • We re-evaluate clusters using a geometrically motivated measure

  27. Angular (Ca) and translational (Ct) confidence are computed separately • Cluster confidence is C = min(Ca,Ct)

  28. The resulting confidence C has same tendency as intra-cluster consistency, but is in stronger accord with human perception

  29. Some results of the single cluster confidence

  30. !!!

  31. Confidence M of entire map is defined by weighted average of cluster confidence

  32. Results

  33. Random Distortion

  34. Iterative Map Alignment

  35. Iterative Map Alignment

  36. Map Comparison (M=0.2769)

  37. Map Comparison (M=0.4355)

  38. Map Comparison (M=0.6554)

  39. FFS (0.3386) • FFS & Virtual Scans (0.3876)

  40. Outlook: Map Cleaning & Merging

  41. Outlook: Map Cleaning & Merging

  42. Conclusion: • Segment based representation and segment similarity measure allows for off the shelf clustering • Clusters detect structural consistencies, advantage over grid approaches • Intra cluster consistency defines consistency of feature representation • Results are in accord with perception

  43. Thanks!

More Related