1 / 38

Real-time Dense Visual Odometry for Quadrocopters

Real-time Dense Visual Odometry for Quadrocopters. Christian Kerl. Outline. Motivation Hardware & Software Approach Problems Ideas. Motivation. Quadrocopters need sensors to fly in unknown environments Motion Position Obstacles Restricted on-board sensors IMU

milla
Download Presentation

Real-time Dense Visual Odometry for Quadrocopters

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real-time Dense Visual OdometryforQuadrocopters Christian Kerl

  2. Outline • Motivation • Hardware & Software • Approach • Problems • Ideas

  3. Motivation • Quadrocopters need sensors to fly in unknown environments • Motion • Position • Obstacles • Restricted on-board sensors • IMU • Visual navigation (no GPS) • Restricted computing resources  Autonomous system

  4. Motivation • Standard approach to visual odometry: • Sparse feature tracking in intensity / color images • Examples: Jakob, ETH Zurich, TU Graz, MIT • On-board frame rates 10 Hz • Our approach: • Using full RGB-D image information • No feature tracking

  5. Hardware – AsctecPelican

  6. Hardware – AsctecPelican IMU AutoPilotBoard Atom Board

  7. Hardware – AsctecPelican • IMU • 3 axis magnetometer, gyroscope, accelerometer • AutoPilot Board • Highlevel + Lowlevel Processor (ARM) • Atom Board • Intel Atom Z530 1.6 GHz • 1 GB RAM • 7 Mini-USB Ports • WirelessLAN • 600 g payload

  8. Software – AsctecPelican • ROS drivers for Asctec Pelican from ETH Zurich • Nonlinear dynamic inversion for position control • Luenberger Observer for data fusion • Updated version using Extended Kalman Filter to be presented on ICRA 2012 • Needs absolute position input from external source • Allows to command accelerations, velocities or positions

  9. Hardware – AsusXtion Pro Live • 24 bit RGB image • 16 bit depth image • 640x480 @ 30 Hz • 150 g + On-camera RGB and depth image registration + Time synchronized depth and RGB image - Rolling shutter - Auto exposure

  10. Approach • Estimate transformation minimizing squared intensity error (energy minimization)

  11. Approach • Linearization • with • minimize • => solve normal equations

  12. Analysis • Estimate transformation minimizing squared intensity error (energy minimization) X translation Y translation

  13. Image Data from Quadrocopter • Hovering

  14. Image Data from Quadrocopter • Trajectory along camera z-axis

  15. Problems • Motion blur • Auto exposure • Dynamic objects (humans)

  16. Problems – Motion Blur

  17. Problems – Motion Blur

  18. Problems – Motion Blur

  19. Problems – Motion Blur

  20. Problems – Motion Blur

  21. Problems – Motion Blur

  22. Problems – Motion Blur

  23. Problems – Auto Exposure

  24. Problems – Auto Exposure

  25. Problems – Dynamic Objects

  26. Ideas • Weighted Least Squares • Initial motion estimate between 2 consecutive frames from IMU data fusion • Multiple iterations per level, convergence checks • Regularization term to minimize / constrain least squares solution • Minimization of intensity and depth error

  27. Ideas – Weighted Least Squares • Assign smaller weight to residual outliers • => • Weight calculation

  28. Ideas – Weighted Least Squares • Influence function • Tukey weight • Huber weight

  29. Ideas – Weighted Least Squares • Weighted error

  30. Ideas – Weighted Least Squares • Influence on energy function X translation w/o weights X translation w/ Huber weights

  31. Ideas – Weighted Least Squares • Influence on energy function Y translation w/o weights Y translation w/ Huber weights

  32. Ideas – Weighted Least Squares

  33. Ideas – Weighted Least Squares • Robustification with respect to dynamic objects • Slightly degrades tracking performance • How to choose parameter b?

  34. Ideas – Initialization from IMU • Use transformation from IMU data fusion as initial estimate

  35. Ideas – Initialization from IMU • Use transformation from IMU data fusion as initial estimate

  36. Ideas – Initialization from IMU • Use transformation from IMU data fusion as initial estimate

  37. Ideas – Multiple Iterations • Perform multiple optimization steps per image pyramid level • Stop when increment below threshold • Bad frames / divergingresults can be recognized and skipped

  38. Summary/Discussion • Weighted Least Squares needs more work (especially weight calculation) • Initialization from IMU promising • Multiple Iterations for increased accuracy and divergence detection promising, but computationally expensive • Jumps in trajectory are really problematic! => Ideas welcome!

More Related