A Software Architecture and Tools for Autonomous Robots that Learn on Mission

1. A Software Architecture and Tools for Autonomous Robots that Learn on Mission K. Kawamura, M. Wilkes, A. Peters, D. Gaines* Vanderbilt UniversityCenter for Intelligent Systems * Jet Propulsion Laboratory http://shogun.vuse.vanderbilt.edu/cis/DARPA/ February 2002 MARS PI Meeting

2. Objective • Develop a multi-agent based robot control architecture for humanoid and mobile robots that can: • accept high-level commands from a human • learn from experience to modify existing behaviors, and • share knowledge with other robots

3. Accomplishments • Multi-agent based robot control architectures for humanoid and mobile robots have been developed • Agent-based Human-Robot Interfaces have been developed for humanoid and mobile robots • SES (Sensory EgoSphere), a short-term robot memory, was developed and transferred to NASA/JSC Robonaut group • SES- & LES (Landmark EgoSphere)- based navigation algorithm was developed and tested • SES knowledge sharing among mobile robots was developed and tested • SAN-RL (Spreading Activation Network - Reinforcement Learning) method was applied to mobile robots for dynamic path planning

4. Presentation / Demo • Multi-agent based Robot Control Architecture • Humanoid • Mobile robots • Agent-based Human Robot Interfaces • Humanoid (face-to-face) • Mobile robots (GUI) • Sensory EgoSphere (SES) • Humanoid • Mobile robots • SES– and LES– based Navigation • SES and LES Knowledge Sharing • Dynamic Path Planing through SAN-RL

5. Multi-Agent Based Robot Control Architecture for Humanoids Sensory EgoSphere Human Agent SES Manager Self Agent A A A Atomic Agents DBAM Manager A A Human Database A DataBase Associative Memory Novel Approach: Distributed architecture that expressly represents human and humanoid internally Publication[1,2]

6. Multi-Agent Based Robot Control Architecture for Mobile Robots SES LES Peer Agent Egosphere Manager Commander Interface Agent Self Agent A A A Atomic Agents A A A DBAM Manager Path Planning Peer Agent DataBase Associative Memory Novel Approach: Distributed, agent-based architecture to gather mission relevant information from robots Publication [7]

7. Agent-based Human-Robot Interfaces for Humanoids • Self Agent (SA) • monitors humanoid’s activity and performance for self-awareness and reporting to human • determines the humanoid’s intention and response and reports to human • Human Agent (HA) • observes and monitors the communications and actions of people • extracts person’s intention for interaction • communicates with people Novel Approach: Modeling the human’s and humanoid’s intent for interaction Publication [3,4,5]

8. Agent-based Human-Robot Interface for Mobile Robots Camera UIC Sonar UIC Novel Approach: Interface that adapts to the current context of the mission in addition to user preferences by using User Interface Components (UIC) and an agent-based architecture Publication [7]

9. Sensory EgoSphere (SES) for Humanoids • Objects in ISAC’s immediate environment are detected • Objects are registered onto the SES at the interface nodes closest to the objects’ perceived locations • Information about a sensory object is stored in a database with the node location and other index Publication [2]

10. Sensory EgoSphere Display for Humanoids Provides a tool for person to visualize what ISAC has detected

11. Sensory EgoSphere (SES) for Mobile Robots • The SES can be used to enhance a graphical user interface and to increase situational awareness • In a GUI, the SES translates mobile robot sensory data from the sensing level to the perception level in a compact form • The SES is also used for perception-based navigation with a Landmark EgoSphere • The can be also used for supervisory control of mobile robots • Perceptual and sensory information is mapped on a geodesically tessellated sphere • Distance information is not explicitly represented on SES • A sequence of SES’s will be stored in the database SES 2d EgoCentric view Top view Publication [6]

12. SES- and LES-Based Navigation LES SES • Navigation based on EgoCentric representations • SES represents the current perception of the robot • LES represents the expected state of the world • Comparison of these provide the best estimate direction towards a desired region • more Future Work Novel Approach: Range-free perception-based navigation Publication [8]

13. SES and LES Knowledge Sharing ? ? ? ? ? ? ? Object Found ? SES data Target LES LES Information Via LES #1 Via LES #2 • Skeeter creates SES • Skeeter finds the object • Skeeter shares SES data with Scooter • Scooter calculates heading to the object • Scooter finds the object • Scooter has the map of the environment • Scooter generates via LES’s • Scooter shares LES data with Skeeter • Skeeter navigates to the target using PBN Novel Approach: A team of robots that share SES and LES knowledge Future Work Publication [9]

14. DB Dynamic Path Planning through SAN-RL(Spreading Activation Network - Reinforcement Learning) High level command with multiple goals • Behavior Priority : • Using the shortest time • Avoid enemy • Equal priority • More… Get initial data from learning mode SAN-RL After finish training send data back to DB activate/deactivate robot’s behaviors Scooter Atomic Agents Novel Approach: Action selection with learning for the mobile robot Publication [10]

15. Publications • K. Kawamura, R.A. Peters II, D.M. Wilkes, W.A. Alford, and T.E. Rogers, "ISAC: Foundations in Human-Humanoid Interaction", IEEE Intelligent Systems, July/August 2000. • K. Kawamura, A. Alford, K. Hambuchen, and M. Wilkes, "Towards a Unified Framework for Human-Humanoid Interaction", Proceedings of the First IEEE-RAS International Conference on Humanoid Robots, September 2000. • K. Kawamura, T.E. Rogers and X. Ao, “Development of a Human Agent for a Multi-Agent Based Human-Robot Interaction,” Submitted to First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2002), Bologna, Italy, July 15-19, 2002. • T. Rogers, and M. Wilkes, "The Human Agent: a work in progress toward human-humanoid interaction" Proceedings 2000 IEEE International Conference on Systems, Man and Cybernetics, Nashville, October, 2000. • A. Alford, M. Wilkes, and K. Kawamura, "System Status Evaluation: Monitoring the state of agents in a humanoid system”, Proceedings 2000 IEEE International Conference on Systems, Man and Cybernetics, Nashville, October, 2000. • K. Kawamura, R. A. Peters II, C. Johnson, P. Nilas, S. Thongchai, “Supervisory Control of Mobile Robots Using Sensory EgoSphere”, IEEE International Symposium on Computational Intelligence in Robotics and Automation, Banff, Canada, July 2001. • K. Kawamura, D.M. Wilkes, S. Suksakulchai, A. Bijayendrayodhin, and K. Kusumalnukool, “Agent-Based Control and Communication of a Robot Convoy,” Proceedings of the 5th International Conference on Mechatronics Technology, Singapore, June 2001. • K. Kawamura, R.A. Peters II, D.M. Wilkes, A.B. Koku and A. Sekman, “Towards Perception-Based Navigation using EgoSphere”, Proceedings of the International Society of Optical Engineering Conference (SPIE), October 28-20, 2001. • K. Kawamura, D.M. Wilkes, A.B. Koku, T. Keskinpala, “Perception-Based Navigation for Mobile Robots”, accepted Proceedings of Multi-Robot System Workshop, Washington, DC, March 18-20, 2002. • D.M. Gaines, M. Wilkes, K. Kusumalnukool, S. Thongchai, K. Kawamura and J. White, “SAN-RL: Combining Spreading Activation Networks with Reinforcement Learning to Learn Configurable Behaviors,” Proceedings of the International Society of Optical Engineering Conference (SPIE), October 28-20, 2001.

16. Acknowledgements This work has been partially sponsored under the DARPA – MARS Grant # DASG60-99-1-0005 and from the NASA/JSC - UH/RICIS Subcontract # NCC9-309-HQ Additionally, we would like to thank the following CIS students: Mobile Robot Group: Bugra Koku, Carlotta Johnson, Turker Keskinpala, Anak Bijayendrayodhin, Kanok Kusumalnukool, Jian Peng Humanoid Robotic Group:Tamara Rogers, Kim Hambuchen, Christina Campbell