330 likes | 455 Views
PixelLaser : Range scans from image segmentation. Kenny Lei ’15. Steve Matsumoto ’12. Nicole Lesperance ’11. Michael Leece ’11. Max Korbel ’13. Zach Dodds ‘62. REU. Inspiration. Saxena (rccar) ’05. Horswill (polly) ’94. Scans ?. platform. "omnicam" images. errors.
E N D
PixelLaser: Range scans from image segmentation Kenny Lei ’15 Steve Matsumoto ’12 Nicole Lesperance’11 Michael Leece’11 Max Korbel’13 Zach Dodds ‘62 REU
Inspiration Saxena (rccar) ’05 Horswill (polly) ’94
Scans ? platform "omnicam" images errors... C. Plagemann et al., ICRA 2008
Motivation Cheap! Expensive! More information: 640 columns of pixels, each of which we can get a range from Less information: 180 rays of distances But it’s harder (for a computer) to extract data from an image than from a laser range scan. Which is where we come in!
Overall Strategy Maps! Classify Segment Scans (and localization) obstacle ground (in pictorial form!)
Training and Classifying 194 191 211 3.2 25.6 4.1 ... 138 87 53 -1.14 8.6 1.4 ... hand-segmented image features, in kd-trees filters
Nearest-neighbor classification RGB alone doesn’t work well…
Nearest-neighbors RGB + texture produces better segmentations
Classifier Variations • Many attempts were made, but the overall accuracy for reasonable versions hovered around 95% • 3-tree Classifier: used above, below, and line categories • Genetic Algorithm: attempted to find the optimal weight values, but optimized for the set of images used to evolve the classifier. The weight-set did not generalize to other environments • Different Environments: the classifier was tested in environments varying in difficulty
Overall Accuracy Results • Regardless of the environment, the Classifier performs consistently
Hallway: Overall Accuracy 95.40 % Library Downstairs: Overall Accuracy 94.01 %
FLANN (Fast Library for Approximate Nearest Neighbors) • C++ knn implementation with variable precision levels • Decreased kd-tree lookup time by 2 orders of magnitude • Allows for more complex real-time segmentation algorithms • Allows the robot to autonomously wander in real time using only the segmenter Marius Muja: Our hero
Now Arriving at…Segmentation! • Uses the information from the classifier to draw the line between the ground and obstacles • Sends the information to the scanner to make laser range scans Classify → Segment → Scan → Map
Initial approaches bottom-up patch pairs multiresolution
Transition Segmentation • Uses transition strength rather than classification • Assigns transition strength based on the certainty that the below patch is traversable and the top patch is not • Takes the difference of the “above strength” of the top patch and the “below strength” of the bottom one • Issues • Misclassifications can result in incorrectly strong transitions
Transitions using a Line Tree • The strongest transitions are found and the area closest matching the line category is chosen
Seam Carving • Start at one end of the image and find the strongest transition • Assign a cost to deviating from the line height in the previous column
Smoothing Certainty Locally • Calculate certainties for all patches in the image • Apply averaging filter on the certainties to suppress rogue misclassifications • Look for the strongest transition in each column and draw line
Edge Detection • Global image edge detection is not useful, but “snapping” to the nearest edge in the strongest transition area reduces segmentation error
Results Average Pixel Error per Column Average Distance Error per Column (inches)
Results - Moving Average Pixel Error per Column Average Distance Error per Column (inches)
From segmentation to scan segmentation row-to-range map scan pixel row number range
Scan Matching • Match two consecutive laser scans and find the corresponding transformation • Can be used to correct poor odometry
SLAM Map Building • Combine all laser scans to buildmaps using CoreSLAM (simultaneous localization and mapping)
Monte Carlo Localization • Create random list of possible locations of robot • Monte Carlo Localization Cycle: • Move: Move all points based on change in odometry • Sense: Compare the laser scan from segmented image with laser scans at possible locations • Resample: Redistribute points based on probability at each location
Point and Click Navigation • Given a known map, we want to be able to click on a location in the map and have the robot navigate there • The current implementation works only if there is a straight, uninterrupted path between the current location and the destination (haven’t implemented a path planning algorithm) • Things to think about include navigation around obstacles and odometry correction
Future Work • Segmenting with a projected laser level line – possibly to be used for automated training as well • Stabilizing the camera – keeping the horizon constant while moving is almost impossible • Speeding up the wandering speed – a more stable camera would help with this • Autonomous mapping
Segmentation classification + confidence segmentations
194 191 211 3.2 25.6 4.1 138 87 53 -1.14 8.6 1.4 ../TrainingImages/Playspacepswo13Patches/00029/randomBelow/0009.png 194.2575 191.4525 211.4775 195.0 192.0 212.0 8.67070895314 8.76885076564 7.29105573631 211.4775 191.4525 194.2575 212.0 192.0 195.0 7.29105573631 8.76885076564 8.67070895314 3113.18473022 2918.61969259 194.707808243 -8.2951355554e-09 -2.99999999989 3.48871383089 -0.1821058002 1.05861940504 0.798119246226 0.142200402323 1.37643522494 1.51710154734 -3.21108530262 25.6053670274 4.08979546594 0.20963020652 3.42407524192 7.98287776033 0.655198704147 1.40172476457 1.2862398764 -0.068793854887 7.97682628048 3.32687477389 -0.83219461218 2.58844742173 1.83432937157 -0.239667915222 1.22096756236 3.56803311807 0.0253655120168 0.415881369811 7.11907142389 -0.829352050768 1.14166676605 4.1432245279 -0.33452013644 -0.0121915477333 6.30267114206 -0.714448599523 0.711384510758 3.38445323448 -0.566103623094 -0.00278409428233 5.92992221434 -0.932328715083 1.17018711051 ../TrainingImages/Playspacepswo13Patches/00026/randomabove/0007.png 138.1075 87.9525 53.4725 138.0 88.0 53.0 3.65184114523 3.44532200962 5.05512054752 53.4725 87.9525 138.1075 53.0 88.0 138.0 5.05512054752 3.44532200962 3.65184114523 1584.29001984 1485.27385943 99.0473083038 2.67691915514e-09 -2.99999999998 0.839568062542 -0.0913685033632 5.68745353063 0.334375851147 -0.0494906892805 5.40416976808 0.617058294109 -1.14642893693 8.69695694778 1.38147297404 -0.87936321568 4.10330225187 1.59491339286 -0.69926257399 0.679234808869 0.526708837532 -0.253691320988 7.92226252642 0.619482921372 -0.155302887182 1.51598656224 0.455410613623 0.896134364593 4.90251778762 1.44298148203 -1.32247289798 1.54652876227 1.82381743704 1.10446810418 0.850926709651 1.23981013329 -0.346508022218 -0.252958061091 1.53636914298 1.26906228604 1.09496630534 1.45689240135 1.4011501228 1.85701734954 1.68840166501 1.0053643485 0.471348780351