Tom Wilson University of Tennessee Department of Electrical and Computer Engineering

1. Formulation of Sensor Brick Concept as a Modular Robotic System Architecture IRIS Laboratory Final Presentation August 5, 2005 Tom Wilson University of Tennessee Department of Electrical and Computer Engineering Imaging, Robotics, & Intelligent Systems Laboratory

2. Presentation Overview 1. Contributions of Research 2. Introduction: Sensor Brick System as a Modular System Architecture 3. Robotic Architectural Overview 4. Formulation of the System Architecture 5. Introduction to the Real-time Control System (RCS) 6. Implementing RCS – An Example 7. Introduction to JAUS 8. Conceptual Application of the Real-time Control System (RCS) into the Sensor Brick System 9. Conclusion

3. Research Contributions • Definition of the Sensor Brick • Formulation of the Sensor Brick Concept as the System Architecture • Conceptual Application of the Real-time Control System as the Operational Architecture of the SBS

4. Introduction • Why formulate a mobile robotic system using a modularsystem architecture? • Currently, commercial robotic system development depends on methods that do not use a strictly modular approach. This results in systems designs that are based on specific types of application. • These systems have limited usefulness because of the specialized but limited ranges of application, a lack of inter-operable control capability, and an inability to easily interchange sub-systems due to unique and proprietary design.

5. Introduction • The Sensor Brick System: • Based on the Concept of Independent Sensor Brick units • Strictly Modular Design Approach Overcomes the Weaknesses that are Inherent to Commercial Systems • Can Be Adequately Applied to a Wide Scope of Issues • Modular Design Facilitates an Inter-Operable Control Capability • Interchanging Sub-Systems Easily Accomplished

6. Introduction – Mission Statement Purpose of Research: • To Validate the Efficacy of the Modular Sensor Brick Concept as a Intelligent Mobile Robotic System Architecture. • Although other methods of development are prevalent in commercial robotic systems, those methods present issues that are undesirable or problematic.

7. Introduction • Sensor Brick Concept • Modular Design Methodology Simplifies and Facilitates Robotic System Development • Identifies Essential Sub-Systems as Modules • Modules are Bundled as Standardized, Commercial Technology • Modules Characterize Notion of Operational Independence • Sensor Brick System is Adaptive to aWide Variety of Applications

8. Introduction Sensor Brick System Intelligent Systems Mobility Platforms Sensors Power Power

9. Introduction • Sensor Brick System as the System Architecture 1. “Blueprint” for Robotic Systems Development 2. Wider Range of Applications - Adaptive 3. Facilitates Evolution to Greater Autonomous Operation - works well with the Operational Architecture (Real-time Control System)

10. Introduction Remotec Andros iRobot Packbot Allen-Vanguard Vanguard MKII TARDEC ODIS Foster-Miller Talon Commercial Mobile Robotic Systems

11. Introduction • Unmanned Systems • Reduce Exposure of Personnel to Harmful Environments • Perform Tasks Not Possible for Humans • Provide Cost Effective Solutions to Repetitive Tasks. • Large Numbers of Unmanned System Products are being introduced to the market. • Many of the Systems can be Characterized as: • Task Dependent – Can Not Be Applied to a Wide Variety of Circumstances • Lack an Interoperable Control Capability • Can Not Interchange Sub-Systems

12. Introduction • To Resolve These Issues, Mobile Robotic Systems Must Incorporate: • Standard,Open ArchitecturesDesigned to Support the Rapid and Cost-Effective Development of Unmanned Systems

13. Introduction • Open Architectures: • Interoperable Control: Inter-System Control is Addressed by Aspects of the Technical Architecture - Joint Architecture for Unmanned Systems (JAUS) • Interchange of Sub-Systems and System Control: Intra-System Control is Addressed by Aspects of the Operational Architecture • The Formulation of a Physically Demonstrable System, the System Architecture

14. Introduction • What Factors Drive the Holistic Development of the Sensor Brick System? • - The Continuing Development of the System (Operational Architecture) • Interoperable Control Issues (Technical Architecture) • Interchangeability of Sub-System Issues (System Architecture)

15. Robotic Architectural Overview • Three (3) types of architectures for mobile robotic systems are: • Operational Architecture – OA • Technical Architecture – TA • System Architecture - SA

16. Robotic Architectural Overview Generic Organization of Mobile Robotic Architecture Operational Architecture Technical Architecture System Architecture

17. Robotic Architectural Overview • Operational Architecture - (OA) • Tasks • Operational Units • Information Flows

18. Robotic Architectural Overview • Technical Architecture (TA) - Should Contain a Set of Rules Governing the: • Organization • Interaction • Interdependence of the System Components • This is to facilitate interoperability when the system’s or system-of-system’s components conform to the specification • Largely addressed by JAUS, IEEE 802.x, etc.

19. Robotic Architectural Overview • System Architecture - (SA) • Systems Architecture Describes Physical SystemComponents and Interconnections that Integrate for Particular Applications • The Systems Architecture is Constructed to Satisfy Operational Architecture Requirements per Standards Defined in the Technical Architecture • The System Architecture Developed in the IRIS Laboratory uses a Modular Design Approach

20. Robotic Architectural Overview Operational Architecture Technical Architecture RCS JAUS, IEEE 802.x, etc. Sensor Brick System System Architecture

21. Formulation of the System Architecture • Question: • How can Mobile Robotic Systems Be “Improved” vs. Present Commercial Systems? • Answer: • Utilize the Proper System Architecture

22. Formulation of the System Architecture • Commercial Robotic Systems: • Commercial Systems have Many Similar Capabilities with Marginal Variations • Robust with respect to the Intended Application • Limited Range of Application • Lack of Intelligent Systems • Interoperable Control • Interchange of Sub-Systems

23. Formulation of the System Architecture • Therefore, the System Architectureshould: • Rectify the Weaknesses of Commercial Systems • Be strongly commensurate with the Operational Architecture

24. Formulation of the System Architecture • The Proper Collaboration between the System Architectureand the Operational Architecture results in: • Robust Operational Capability in a Large Variationof Applications • Optimal Use of Intelligent Systems Resources • A Control Structure that Enables Interoperable Control and the Interchange of Sub-Systems

25. Introduction to the Real-time Control System • Real-time Control System (RCS) • A methodology for conceptualizing, designing, engineering, integrating, and testing intelligent systems software for vehicle systems with any degree of autonomy.

26. Introduction to RCS • RCS integrates the functional elements, knowledge representations, and flow of information so that intelligent systems can analyze the past, perceive the present and plan for the future. • It enables systems to assess the cost, risk, and benefit of past events and future plans, and make intelligent choices among alternative courses of action.

27. Plan Plan Act Sense Sense Sense Plan Act Plan Sense Act Introduction to RCS A Hybrid Deliberative-Reactive Architecture Highest Level Lowest Level

28. Introduction to RCS Objects Most Abstract Surfaces SP BG WM Lines Points Least Abstract Sensors and Actuators

29. Introduction to RCS Objects Most Abstract Surfaces SP BG WM Lines Points Least Abstract Sensors and Actuators

30. Value Judgment Sensory Perception Introduction to RCS Sense Plan Changes and Events Simulated Plans Task Goals perception, focus of attention World Modeling Behavior Generation plans, state of action Knowledge Database Act Commanded Actions Observed Input A schematic representation of the RCS Reference Architecture

31. Introduction to RCS Sensory Output Commanded Task (Goal) Status RCS Node VJ Value Judgment Operator Interface Perceived Object & Events Peer Input Output Plan Evaluation SP WM BG Update Plan Sensory Processing World Modeling Behavior Generation Predicted Input State Knowledge Database KD Status Commanded Actions (Sub-goals) Sensory Input Observed Input

32. Introduction to RCS Sensory Output Status to Superior Command from Superior Outputs to Peers Operator Input RCS Node Inputs from Peers Status to Operator Sensory Inputs Status from Subordinates Commands to Subordinates

33. Introduction to RCS Plans for the next 30 seconds – task to be done on one object Objects MACHINE SP WM BG Surfaces E-MOVE 3 second plans – Subtask on object part Obstacle-free paths Inspection Communication Part Handling Tool Motion BG BG SP BG SP SP SP BG WM WM WM WM Lines PRIMITIVE 0.3 second plans – Tool Trajectory SP SP SP BG BG BG SP BG WM WM WM WM SERVO Points 0.03 second plans – Actuator Output Sensors and Actuators

34. Operator Interface Introduction to RCS Orders SP BG SHOP WM Plans for the next day Batches SP BG CELL WM Plans for the next hour Trays of parts and tools Plans for the next 5 minutes – tasks to be done on tray of parts SP BG WORKSTATION WM Plans for the next 30 seconds – task to be done on one object Objects MACHINE SP BG WM E-MOVE PRIMITIVE SERVO Sensors and Actuators

35. Operator Interface Implementing RCS – DARPA XUV III Battalion Formation SP BG Surrogate Battalion WM Plans for the next 24 hours Platoon Formation SP BG Surrogate Platoon WM Plans for the next 2 hours Section Formation Plans for the next 10 minutes – tasks to be done on tray of parts SP BG Surrogate Section WM Plans for the next 30 seconds – task to be done on one object Objects of Attention Vehicle SP BG WM Subsystem Primitive Servo Sensors and Actuators

36. Implementing RCS – DARPA XUV III Plans for the next 30 seconds – task to be done on objects of attention Objects MACHINE SP WM BG Surfaces E-MOVE 5 second plans – Subtask on object surface Obstacle-free paths RSTA Communication Mission Package Locomotion BG BG SP BG SP SP SP BG WM WM WM WM Lines PRIMITIVE 0.5 second plans – Steering, Velocity SP SP SP BG BG BG SP BG WM WM WM WM SERVO Points 0.05 second plans – Actuator Output Sensors and Actuators

37. Conclusion - RCS • RCS defines interfaces to the conceptual and semantic level. • But, not to the syntactic, message, and transport levels (JAUS). • RCS is a highly detailed hybrid-hierarchical architecture that does not specify how messages are passed or which communication protocols must be used.

38. Introduction to Joint Architecture for Unmanned Systems • JAUS – Joint Architecture for Unmanned Systems • JAUS is a technical architecture that is concerned with the data structure of unmanned systems which are comprised of software elements, the externally visible properties of those elements, and the relationships among them

39. Introduction to JAUS • JAUSis a common language consisting ofwell-defined messages, enabling internal and external communication between unmanned systems

40. Introduction to JAUS JAUS System Topology

41. Introduction to JAUS • A Component is the Lowest Level of Decomposition in the JAUS Hierarchy. • A Component is a Cohesive Software Unit that Provides a Well-Defined Service or Set of services • Generally, a Component is an Executable Task or Process.

42. Introduction to JAUS • One of the Principal Goals of JAUS is to Provide a Level of Interoperability Between Intelligent Systems that has been missing in the past • Towards this end, JAUS Defines Functional Components with Supporting Messages, but does not impose regulations on the systems that govern configuration

43. Introduction to JAUS • To Achieve the Desired Level of Interoperability Between Intelligent Computing Entities, All Messages that Pass Between JAUS Defined Components (over networks or via airwaves) Shall Be JAUS Compatible

44. Introduction to JAUS GOA Stack Application SW Application SW Class 4L Class 4L Class 4D -> System Services Operating System Services XOS Services Other Nodes or Operational Subsystems Class 3L Class 3X Class 3D -> Resource Access Services Resource Access Services Class 2L Class 2L Class 2D -> Physical Resources Physical Resources Class 1L Class 1L Class 1D -> External Environment Interface

45. Introduction to JAUS GOA Stack JAUS defines messages and data formats for application layer components, as well as XOS services for message handling Application SW Application SW Class 4L Class 4L Class 4D -> System Services Operating System Services XOS Services Other Nodes or Operational Subsystems Class 3L Class 3X Class 3D -> Resource Access Services Resource Access Services Where other organizations define standards, JAUS does not interfere (e.g., JTA, IEEE, SAE, etc.) Class 2L Class 2L Class 2D -> Physical Resources Physical Resources Class 1L Class 1L Class 1D -> External Environment Interface

46. Introduction to JAUS Subsystem Remote Controller Mobility Platform Sensor Brick Node JAUS Compliant Message Format Wireless System Wireless System Component

47. Introduction to JAUS Subsystem Remote Controller Mobility Platform Sensor Brick Node ? Wireless System Wireless System Component

48. Introduction to JAUS Subsystem Remote Controller Mobility Platform Sensor Brick Node Wireless System Wireless System Component JAUS Compliant Message Format Concept of Inter-Operability Alternative Controller

49. Conclusion - JAUS • Support all Classes of Unmanned Systems - JAUS should ensure platform independence • Rapid Technology Insertion - JAUS should not impose a specific technical approach • Interoperable Operator Control Units (OCU) - JAUS should allow for the interoperability of operator control units • Interchangeable/Interoperable Payloads - JAUS should allow technical advancements while not imposing specific hardware or software implementations • Interoperable Unmanned Systems - JAUS should allow for communication between unmanned systems independent of platform type

50. Level 0 - RCS Vehicle Level Safebot + Laser Range Sensor Brick Subsystem Level Mechanical / Power Computing / IS Communications Primitive Level Driver Servo Level Left Motor Left Motor Sensor / Actuators