Networked Robotic Systems

1. Networked Robotic Systems An Overview Dr. Thad Roppel

2. CRR Lab - A Brief History • Outgrowth of work started at Eglin AFB in 1992 • Infrared / Millimeter-Wave Radar Sensor Fusion • Follow-on funding  DARPA  e-NOSE • Best sensor platform? Robots • Many robots are better than one robot..

3. Problem Complexity: Human vs. Machine • Object recognition • Linguistics • Extraction of Relevant Features from Sensor Arrays • Judging HARD Maximum Potential Benefit MACHINE • Thresholding • Tallying • Arithmetic • Logic EASY HARD EASY HUMAN SENSOR FUSION LABORATORY

4. IR / MMW DATA FUSION Support: AFOSR 1992-93 Project Goal: Improved identification of military vehicles from aerial scenes. T-62 Tank M-113 Armored Personnel Carrier (APC) LANCE Missile Launcher

5. APPROACH: IR SCENE PIXELS APC NEURAL NETWORK TANK LAUNCHER MMW RADAR DATA IR / MMW Fusion, cont’d OVERALL RESULT: 14 % improvement with sensor fusion PERFORMANCE ASSESSMENT: • Multiple permutations • Confusion matrix • Average result

6. COMMAND STATION PLUME RF LINK VEHICLE WIND SENSORS ROAD Chemical Sensor Arrays Support: DARPA 1997-99 PROJECT GOAL: Improved identification and detection of chemical plumes in non-laboratory conditions.

7. Canine Training at IBDS Auburn is world-renowned for training of detection dogs at the Institute for Biological Detection Systems.

8. 5 4.5 4 3.5 3 Sensor Voltage 2.5 2 1.5 1 0.5 0 0 100 200 300 400 500 Timestep Odor Sensor Array Sensor Outputs Chemical Sensor Arrays, cont’d Sensor Array Dynamic Response

9. Below Threshold Above Threshold 5 4.5 2 4 4 3.5 6 3 Sensor Voltage Sensor Number 2.5 8 2 10 1.5 Sensors 1-15 12 1 0.5 14 0 0 100 200 300 400 500 10 20 30 40 50 Timestep Timestep Thresholded Binary Output Raw Output Chemical Sensor Arrays, cont’d Preprocessing

10. Sample 1 Sample 2 Sample 3 1 1 ace 20 20 amm dal g87 g89 g93 oil pth xyl 5 10 15 5 10 15 5 10 15 Sensor # Sensor # Sensor # Chemical Sensor Arrays, cont’d

11. Time Evolution of Confusion Matrix: Forward Sequence Trained for 20 timesteps 1 timestep 5 timesteps 10 timesteps ace amm 1 dal network response g87 0.9 g89 0.8 g93 oil 0.7 pth 0.6 xyl 0.5 input categories 0.4 0.3 20 timesteps 50 timesteps Ideal Response 0.2 ace 0.1 amm dal network response 0 g87 g89 g93 oil pth xyl Chemical Sensor Arrays, cont’d

12. Time Evolution of Confusion Matrix: Random Sequence Trained for 20 timesteps 1 timestep 5 timesteps 10 timesteps ace amm 1 dal network response g87 0.9 g89 0.8 g93 0.7 oil pth 0.6 xyl 0.5 input categories 0.4 20 timesteps 50 timesteps Ideal Response 0.3 ace 0.2 amm 0.1 dal network response 0 g87 g89 g93 oil pth xyl Chemical Sensor Arrays, cont’d

13. BIOMIMETICS Support: Under discussion with AF Advanced Guidance Division, Munitions Directorate at Eglin AFB PROJECT GOAL: Learn sensor fusion from animals. Apply this to flying a drone to target using onboard video. Flies land accurately Bats catch evading insects in flight Bees find flowers

14. CRR Lab – History, Cont’d • Feb. 2006: Invited Joe Albree – Math Prof. at AUM - to speak to HKN about history of the engineering profession in USA. • I didn’t know he co-authored a book about the history of West Point with… • Gen. Chris Arney, ARO program in Multi-Agent Systems, who was organizing… • LIMES 2006 at West Point. Language for Intelligent Machines.

20. Cooperative Autonomous Robots for Reconnaissance White Paper for Chris Arney, ARO Prepared 8/29/2005 by Thad Roppel, ECE Dept., Auburn University Contact: roppeth@auburn.edu, (334) 844-1814

21. Hardware Testbed for Collaborative Robotics using Wireless Communication Chris Wilson – MS Dec 2009

23. Mounted optical mice and batteries

24. Wifistix (top card) and Gumstix (bottom card).

28. Eric Hildebrand ELEC 5530 HW 4 November 10, 2010 Dominion The year was 2143, and humanity was at the will of a single man. Known only as “Roppeth”, an evil mastermind had created an army that defeated everything humanity threw at it. No one knew where this army came from, but it could only be assumed that Roppeth created the first generation, and each new generation was spawned by the previous. What made the army so overpowering was the fact that they were autonomous robots controlled by the will of their leader but could act and behave independently from his control. These robots were bipeds, slightly larger than an average human, but completely overpowering to any human counterpart. …

29. Good News… • How to Survive a Robot Uprising

30. ASIMO • Highly functional biped • The future…? • Video • But for now, cooperation is more like this….

31. Oct. 2008 – Robotics and Autonomous Systems - Special Issue on Network Robot Systems • A probabilistic framework for entire WSN localization using a mobile robot • Action evaluation for mobile robot global localization in cooperative environments • Autonomous functional configuration of a network robot system • Framework and service allocation for network robot platform and execution of interdependent services • Robots in the kitchen: Exploiting ubiquitous sensing and actuation • Human behavior recognition using unconscious cameras and a visible robot in a network robot system • End-to-end congestion control protocols for remote programming of robots, using heterogeneous networks: A comparative analysis

33. NRS Definition • The IEEE Society of Robotics and Automation Technical Committee on Networked Robots provides the following definition of Networked Robots • Physical embodiment: Any NRS has to have at least a physical robot which incorporates hardware and software capabilities • Autonomous capabilities: A physical robot must have autonomous capabilities to be considered as a basic element of a NRS. • Network-based cooperation: The robots, environment sensors and humans must communicate and cooperate through a network. • Environment sensors and actuators: Besides the sensors of the robots, the environment must include other sensors, such as vision cameras and laser range finders, and other actuators, such as speakers and switches • Human-robot interaction: In order to consider a system as NRS, the system must have a human-robot related activity.

34. NRS Definition Expanded Two subclasses of Networked Robots: (1) Tele-operated *human supervisors send commands and receive feedback via the network. -Medicine, education, search & rescue,… (2) Autonomous, *robots and sensors exchange data via the network. *sensor network extends the effective sensing range of the robots *allows them to communicate with each other over long distances to coordinate their activity. *The robots in turn can deploy, repair, and maintain the sensor network to increase its longevity, and utility. *Broad challenge: develop a science base that couples communication to control to enable such new capabilities

35. Network Robot Types Three types of network robots: • Visible - can be seen • humanoid, pet, stuffed animal, etc. • Virtual - acts in a cyber space and makes use of information available on Internet. • avatar agent on a mobile phone or info kiosk • Unconscious - users are not aware of the presence of the robot • camera or a sensor embedded in infrastructure

36. Ubiquitous robotics network system for urban settings (URUS)

37. Physically Embedded IntelligentSystems (PEIS)

38. Japan NRS • Japan NRS consists of four major Japanese companies: • NTT - telecommunications; • Toshiba - home appliances; • Mitsubishi Heavy Industries - industrial robots • ATR- telecommunication and social robotics R&D

39. NRS in USA • NetBot Lab at TAMU (Prof. Dezhen Song) • Ghostrider video clip • DARPA, JPL • Georgia Tech

40. WSN Localization

41. WSN Localization, cont’d Scheme of the approach. The signal strength is used to estimate the position of the nodes of the network. The mobile robot computes centrally an initial estimation employing a separate Particle Filter for each node. In the second step, a decentralized Information Filter integrates information received from neighbor nodes and the robot, at each node. (b): An example, a ground robot (Romeo) driving through the network.

42. WSN Localization, cont’d

43. Robots in the Kitchen

44. ROS (Willow Garage) • Willow Garage

45. Conclusion Robots everywhere!!