Tennessee State University College of Engineering ENGINEERING RESEARCH INSTITUTE (ERI) Interdisciplinary Research in Ro

1. Tennessee State University College of Engineering ENGINEERING RESEARCH INSTITUTE (ERI) Interdisciplinary Research in Robotics Intelligent Tactical Mobility Research Laboratory (ITMRL) Intelligent Control Systems Laboratory (ICS) Center for Neural Engineering (CNE) Computer and information systems Laboratory (CISE) Mohan J. Malkani, Ph.D. (Director) (615) 963-5400 Fax: (615) 963-5397 mmalkani@tnstate.edu

2. Research Projects in Robotics: Past and Present • Tele-Robotics jointly with Caltech funded by NSF (1997-2000) • Originally Funded by US Army TACOM, Warren, MI, under two research grant contracts: 1. Development of an Integrated High-level Mobility Controller for Virtual Tandem Robotic Vehicles, DAAE07-98-C-0029, (1997-2000) 2. Deliberative, Reactive, and Adaptive Task Planning of Intelligent Cooperative Mobile Robots, DAAE07-01-C-L-065, (2001-2002) • Embodiment of intelligent behaviors on mobile robots using fuzzy-genetic algorithms, funded by NASA/Ames Research Center (2000-2004) • Funded by DARPA Through Penn State Applied Research “Sensor surveillance” under MURI-ESP Research Project, DAAD19-01-1-0504, (2002-2003) • Funded by NASA/JPL, FAR Investigator Program, “Visual Telerobotic Task Planning of Cooperative Mobile Robots”, (2003-2006)

3. Research Focus Areas • Development of Advanced Control schemes that enable tactical team cooperation of Intelligent Autonomous robots effectively and efficiently. • Test and evaluate performance of advanced control schemes under different operational conditions and different sensory data modality experimentally using high-fidelity computer generated simulation and physical robotic test beds. • Technical Competency Areas Included: • Behavior-based Distributed control of Cooperative Mobile Robots. • Sensory data and image processing and fusion for fault tolerance control of intelligent robots. • Advanced control schemes based Soft Computing techniques, (Neural Networks, Fuzzy Logic, Genetic Algorithms, …). • High-fidelity world perception modeling of robotic systems. • Man-machine development for Visual Teleoperation and Telerobotic control of Cooperative Robots.

4. Theoretical and Experimental Research Capabilities • Developed various behavior-based schemes for intelligent deliberative, reactive, and adaptive task planning of cooperative robots. • Developed various image processing techniques for visual localization and target tracking of robots. • Applied different soft computing methods for target pattern recognition and classification. • Developed FMCell comprehensive robotic simulation software for the purpose of man-machine interface development. • State-of-the-art physical robotic test bed consisting of twelve heterogeneous robots. • Embodiment of intelligent behaviors on mobile robots using fuzzy-genetic algorithms

5. Intelligent Man-Machine Interface • Interactive Component Based Architecture for rapid task deployment of cooperative robots. • Image and sensory data processing and analysis capability for intelligent control of autonomous robots. • Soft computing capability for deliberative, reactive, and adaptive development of behavior-based robot tactical schemes. • 3D modeling and simulation tools for world perception modeling and visualization of cooperative mobile robots. • Built-in TCP/IP wireless communication protocols for distributed client/server-based control of remotely operating robots. • Experimental human-robot interaction

6. Robotics Research Interoperability Software Architectures Multi-Robot Cooperation Human-Robot Interaction Intelligent Control Systems Telerobotics Intelligent User Interfaces

7. Human-Robot Interaction Over the Internet; Via PDAs; Via Speech Via cellular phones (speech integrated) Human detection, recognition, and localization Social behavior modeling Interoperability for Robotics Programming language and operating system independent software architecture Intelligent User Interface Design Adaptive - mission aware Multiple users – multiple robots Heterogeneous Multi-Robot Cooperation Behavior-based approach Robotics Research

8. Human Agent Human Detection Agent (motion) Monitoring Agent Interaction Agent Observer Agent Social Agent Human Detection Agent (sound) Human Affect Agent Affect Estimation Agent Human Intention Agent Identification Agent Human Identification Agent (face) Human Identification Agent (voice) Human Database The Human Agent System • The Human Agent is a virtual agent that serves as an internal active representation of people in the robot’s environment. To Self Agent • As a representation, it is able to detect, represent and monitor people. The description active is used, much as in describing active perception vision systems [Bajcsy 1987], to indicate that the system can take action to make its representation richer.

9. Peters redefined the Sensory EgoSphere as a sparse spatiotemporally indexed short term memory (STM). Structure: a variable density geodesic dome. Nodes: links to data structures and files. Indexed by azimuth, elevation and time. Searchable by location and content. Sensory EgoSphere (SES) for Mobile Robots images sonar laser Peters, R. A. II, K. E. Hambuchen, K. Kawamura, and D. M. Wilkes, “The Sensory Ego-Sphere as a Short-Term Memory for Humanoids”, Proc. IEEE-RAS Int’l. Conf. on Humanoid Robots, pp. 451-459, Waseda University, Tokyo, Japan, 22-24 Nov. 2001.

10. Experimental Design • 2 training tasks with the original and enhanced interface • 2 teleoperation tasks with the enhanced and orignal interface.

11. Telepresence Software Architecture (Over Internet) USER TCP/IP (Internet) Hardware Robot Control Programs Internet Control (ServerSide) Internet Control (Client Side) BeeSoft API SERVERS

12. Research Motivations (Consumer Tele-presence) Team of Cooperative Intelligent Robots Human Commander Soldiers TCP/IP Internet Audio Commands Speech Recognition Robot Commander

13. Final Decision (Fuzzy Logic) Fuzzy Decision-2 Fuzzy Decision-3 Fuzzy Decision-1 MANAGER Grab Grab Grab Robot-1 Grabs an Image Robot-2 Grabs an Image Robot-3 Grabs an Image Image Image Image Process (NN-Fuzzy) Process (NN-Fuzzy) Process (NN-Fuzzy) System Architecture

14. Research Motivations (Development of Robot Behaviors, NASA, Phase-I) FIRBA Implementation SONAR CLASS ODOMETER CLASS TARGET CLASS PATH CLASS SONAR HANDLING ODOMETER HANDLING TARGET HANDLING PATH HANDLING SONAR PERCEPTIONS TARGET PERCEPTIONS MOTION PRIMITI-VES LEVEL 0 BEHAVIORS LEVEL 1 BEHAVIORS Overall Software Architecture. • Abstracts beeSoft: Complex API protocols are hidden • Object Oriented: Abstraction, reuse by inheritance. • Perception Sharing: Common perceptions can be shared • Action Suggestions: Arbitration through MAL, fuzzy inference and De-fuzzification. • Independent Behaviors

15. SENSORS Pre-Perception Processing Perception Capabilities Behaviors. Action Capabilities Action Execution ACTUATORS Research Motivations (Development of Robot Behaviors, NASA, Phase-I) • FIRBA – Robot Control System • Complexity - • Robustness • Multiple Sensors • Multiple Methods • Integration • Incremental Development • Software • This complexity is handled by system decomposition in terms of : • -- functional units • -- behavioral units The FIRBA architecture.