1 / 31

Fovea-Based Robot Control for Anticipation Studies in Various Scenarios

This study explores the use of fovea-based robot control for anticipation in different scenarios. The software framework and integration of simulated 3D scenarios are discussed. The simulated vision and the fovea simulation are also explained, along with the software framework and the robot's capabilities.

gbrumbaugh
Download Presentation

Fovea-Based Robot Control for Anticipation Studies in Various Scenarios

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fovea-Based Robot Control for Anticipation Studies in Various Scenarios Alexander Förster, Daan Wierstra, Jürgen Schmidhuber IDSIA - Lugano - Switzerland MindRACES, First Review Meeting, Lund, 11/01/2006

  2. Overview • Real world scenarios • Fovea • Software Framework • Robertino Robot • Simulated 3D scenarios • Integration and Future Work MindRACES, First Review Meeting, Lund, 11/01/2006

  3. Instances of scenario 1: find an object Robot Lab Still Image 3D Environments 2D Environments Office Room Movie (LUCS) MindRACES, First Review Meeting, Lund, 11/01/2006

  4. Fovea Centralis Simulation Three regions with increasing resolutions MindRACES, First Review Meeting, Lund, 11/01/2006

  5. Fovea Centralis Simulation Subsample each region to 13x11 pixels Three regions with increasing resolutions MindRACES, First Review Meeting, Lund, 11/01/2006

  6. Fovea Centralis Simulation Subsample each region to 13x11 pixels Combine the subsampled image Three regions with increasing resolutions MindRACES, First Review Meeting, Lund, 11/01/2006

  7. ? ? ? ? ? ? ? ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 0 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  8. ? ? ? ? ? ? ? ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 1 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  9. ? ? ? ? ? ? ? ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 2 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  10. ? ? ? ? ? ? ? ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 3 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  11. ? ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 4 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  12. ? ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 5 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  13. ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 6 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  14. ? ? ? ? ? Simulated Vision: 1-D Fovea Simulation State 7 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  15. Simulated Vision: 1-D Fovea Simulation State 8 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  16. Simulated Vision: 1-D Fovea Simulation State 9 Task: Find the sad smiley Bins Objects 1-D world Transformation Fovea Memory MindRACES, First Review Meeting, Lund, 11/01/2006

  17. Simulated Vision: abstract 2-D Simulation • Extended 1-D Simulation • Only the centered sensor can differentiate between objects MindRACES, First Review Meeting, Lund, 11/01/2006

  18. Simulated Vision: 2-D Simulation Original image with 2 objects MindRACES, First Review Meeting, Lund, 11/01/2006

  19. Center of the fovea Simulated Vision: 2-D Simulation Original image with 2 objects Simulated fovea images Step 1 MindRACES, First Review Meeting, Lund, 11/01/2006

  20. Center of the fovea Simulated Vision: 2-D Simulation Original image with 2 objects Simulated fovea images Step 1 Step n Detected!!! MindRACES, First Review Meeting, Lund, 11/01/2006

  21. Software Framework Client Server • Robertino • Debian/GNU Linux operating system • CAN bus interface • FireWire (IEEE 1394) interface • Video4Linux library • Fovea simulation • Robomon • Linux or Windows • Experiment management • Interface for learning algorithms • Full remote control of the robot/simulation TCP/IP • Robosim • Linux or Windows • Ogre framework • Fovea simulation • Simple physical and collision detection system • Same interface as the real robot Previously recorded data MindRACES, First Review Meeting, Lund, 11/01/2006

  22. Robertino – Client Software Manual control of the robot Fovea image Learning control interface MindRACES, First Review Meeting, Lund, 11/01/2006

  23. Robertino - Overview • Diameter: 40 cm • Height: 43 cm • Weight: 6.5 kg. • Holonomic three wheeled drive • PC-103 (industry standard) with a 500MHz Intel Mobile-Pentium II processor on-board • WLAN (IEEE 802.11a) • 2 cameras, used to simulate the fovea • Actuators: the three wheels and the simulated fovea www.openrobertino.org MindRACES, First Review Meeting, Lund, 11/01/2006

  24. Omnidirectional Camera Image of an office environment, as seen by the robot Omnidirectional mirror Web cam MindRACES, First Review Meeting, Lund, 11/01/2006

  25. Image Transformation • Used only for human convenience for navigation • Simple transformation algorithm MindRACES, First Review Meeting, Lund, 11/01/2006

  26. High Quality Camera Image of an office environment, as seen by the robot Imaging Source DFK21 AF04 MindRACES, First Review Meeting, Lund, 11/01/2006

  27. Robot Lab Environment • Robot lab 550 cm x 280 cm • Robot can navigate through the world and move the fovea • Robot can be observed from a top view camera and local camera MindRACES, First Review Meeting, Lund, 11/01/2006

  28. Robot Lab Environment No video for his presentation! Camera image Fovea data Composition MindRACES, First Review Meeting, Lund, 11/01/2006

  29. 3D Simulation of the Robot • Environment design in 3D Studio MAX or G-MAX. • Simulation in Ogre3D • Same objects as in the real world • Shadows (optional) MindRACES, First Review Meeting, Lund, 11/01/2006

  30. Simulated vs. Real Vision (Fovea) • Fovea transformation brings both world more together • Artificial noise simulates camera noise (optional) Simulation Real Robot MindRACES, First Review Meeting, Lund, 11/01/2006

  31. Integration and Future Work • Develop and evaluate anticipatory learning algorithms in the simulated environments (UW, ÖFAI); compare them also to non-anticipatory ones • Share scenarios for attentive vision (LUCS, NBU) • Real robot with simulation as anticipation for surprise studies (CNR) • Simulation-based learning of anticipation (UW) • Share fovea simulation (IST) • Evaluate transfer of learned behavior to the real robot • Systematically increase the complexity of simulated and real environments • Possibly use a movable camera and zoom lens mounted on the robot MindRACES, First Review Meeting, Lund, 11/01/2006

More Related