G. Casalino, E. Zereik , E. Simetti, A. Turetta, S. Torelli and A. Sperindè

1. Planetary Rover Navigation via Visual Odometry:Performance Improvement Using Additional ImageProcessing and Multi-sensor Integration G. Casalino, E. Zereik, E. Simetti, A. Turetta, S. Torelli and A. Sperindè EUCASS 2011 – 4-8 July, St. Petersburg, Russia

2. Agenda • Introduction • Visual Odometry • Additional Measurements • State Estimators • Sequence Estimators • Multi-Sensor Integration • Discussion and Conclusion EUCASS 2011 – 4-8 July, St. Petersburg, Russia

3. Introduction • Planetary Robot • accurate localization and motion estimation • Different techniques • WO • IMU • VO • Improve VO • additional image processing • Extended/Iterated Kalman Filters • Sequence Estimators • Integration scheme • Final multi-sensor scheme EUCASS 2011 – 4-8 July, St. Petersburg, Russia

4. Visual Odometry EUCASS 2011 – 4-8 July, St. Petersburg, Russia

5. Visual Odometry EUCASS 2011 – 4-8 July, St. Petersburg, Russia

6. Visual Odometry • At eachstep , the roverposition and orientation are • computedwithrespectto the previousstep • Sequence of positions-orientations truly attained by the vehicle EUCASS 2011 – 4-8 July, St. Petersburg, Russia

7. Estimations independent white noise sequences affecting position and orientation measurements position and orientation covariance matrices measured rover position and orientation EUCASS 2011 – 4-8 July, St. Petersburg, Russia

8. Estimations • At eachstep, VO provides the rover relativeposition and orientation • Absolute rover position and orientation: • where EUCASS 2011 – 4-8 July, St. Petersburg, Russia

9. Estimations • Linear open chain of frames, with independentpositioning • Errorprogressivelyincreasingwith the numberofstages • No furtherconstraints • Improvements: • Additionalmeasurements • At eachstepprovidemeasurementsof the occurred • motionbetweenframes and EUCASS 2011 – 4-8 July, St. Petersburg, Russia

10. Additional Measurements • Assumption: • stereo camera can recognize in the current frame a sufficient number of features belonging to images and EUCASS 2011 – 4-8 July, St. Petersburg, Russia

11. State Space Model orientation position EUCASS 2011 – 4-8 July, St. Petersburg, Russia

12. State Space Model EUCASS 2011 – 4-8 July, St. Petersburg, Russia

13. State Estimators • System evolution estimated via standard Kalman filters • Extended (EKF) • Iterated (IKF) • Gaussianityhypothesis for the filter • Gaussianity is not suitable due to system non-linearity • such an approximation leads to suboptimality • These are recursivefilters • linearly increasing errors have still to be expected for increasing • number of stages • Incoming acquisitions to better all the past state estimations EUCASS 2011 – 4-8 July, St. Petersburg, Russia

14. Sequence Estimators state sequence measurement sequence • The problem becomes: EUCASS 2011 – 4-8 July, St. Petersburg, Russia

15. Sequence Estimators • Renew, at each stage, the entire sequence , without relying on the previous one • Error generally characterizing IKF and EKF should result strongly reduced • BUT • Increasing dimensionalitywith increasing number of stages • linearand quite acceptable with a reasonably high maximum number of • stages • after this freezeand restartthe procedure EUCASS 2011 – 4-8 July, St. Petersburg, Russia

16. Sequence Estimators • Solve the followingcascadeofsub-problems: • Maintain a manageableimplementative form that otherwise cannot be • guaranteed, considering the general problem • Minimization of the Gaussian p.d.f. exponents • Bayesformula EUCASS 2011 – 4-8 July, St. Petersburg, Russia

17. Sequence Estimators • At each stage a linearparametrizationisobtained • Itisthe constraint for the previous stage • Back Substitution scheme • Dynamic Programming strategy EUCASS 2011 – 4-8 July, St. Petersburg, Russia

18. Comments • Waituntil stage • Renew the interpolated sequence in correspondence of each new stage • Backward Phase: computational effort increasing with the number of stages • Restart the procedure from the last stage considered as the new initial one • Smaller drifting errors • Accepted suboptimality vs. joint estimation of both sequences EUCASS 2011 – 4-8 July, St. Petersburg, Russia

19. Integration Scheme • Exploit different sensors • Augmented sensor with better performances • Different integration schemes: EUCASS 2011 – 4-8 July, St. Petersburg, Russia

20. Integration Scheme • First sequential scheme • SQE fed with smaller variance measurements data from STE • Feedback loop used to re-initialize the STE module • Totally useless scheme EUCASS 2011 – 4-8 July, St. Petersburg, Russia

21. Integration Scheme • Without further data, things can be bettered with a parallelintegration scheme EUCASS 2011 – 4-8 July, St. Petersburg, Russia

22. Multi-Sensor Integration • IMU: measurements about the angular velocity vector and the linear • acceleration vector • WO integration EUCASS 2011 – 4-8 July, St. Petersburg, Russia

23. Discussion and Conclusion • Previously developed VO module • via software • CUDA implementation: • SURF extraction and descriptors via CUDA • matching and tracking via software with SAD • pose estimation via software • Sequence Estimator • Sensor data integration EUCASS 2011 – 4-8 July, St. Petersburg, Russia

24. Discussion and Conclusion • Open issues: • still under consideration • many simulations and experimental tests are to be carried out • not only planar motion • mange the non-holonomic constraints for the rover • “Visual Odometry Centric” scheme • different state space model starting from a different sensor • integration scheme with possibly different characteristics • worth a comparison EUCASS 2011 – 4-8 July, St. Petersburg, Russia

25. Questions? Thank you! EUCASS 2011 – 4-8 July, St. Petersburg, Russia