Spring 2005 Research Summary IRIS Laboratory Final Presentation May 3, 2005

1. Spring 2005 Research Summary IRIS Laboratory Final Presentation May 3, 2005 Tom Wilson University of Tennessee Department of Electrical and Computer Engineering Imaging, Robotics, & Intelligent Systems Laboratory

2. Presentation Outline • State of Spring 2005 Research Task 1: Re-organization of Internal Reports (5) Task 2: Report on Control Architectures for AI Robotics Task 3: Compilation of a Compendium of Reports Task 4: Draft of Thesis – Modular Robotic System B. Research Focus Task 2: Report on Control Architectures for AI Robotics Task 4: Thesis – Modular Robotic System C. Summary & Conclusion

3. State of Research - Task 1 • Re-Format, Edit and Revise: Internal Reports Completed: January 28 Objective: Edit and revise relevant internal reports pertaining to the development of a thesis on the IRIS Modular Robotic System.

4. State of Research - Task 1 A. Mobility and Drive – Robert E. B. Processing and Fusion – Prashad C. Radar – Venkat D. Path Planning – Gaurav E. Integration – Rick K.

5. State of Research - Task 1 Purpose: To arrange the reports into a useable format that could be easily included into the MRS Thesis. Method: Revise the reports by adding continuity, style, formatting, fixing grammatical errors & syntax, etc. so the reports could be a useable format that could be easily included into the MRS Thesis. Continuance: from Fall 2004 Tasks.

6. State of Research - Task 2 • Generate a ten (10) page report on: Robotic Architectures Completed: March 1 (60 pages) Objective: Develop a report using seminal texts in the field of Robotic Architecture. This report is intended to be used as the first chapter in the MRS Thesis.

7. State of Research - Task 2 Purpose: To learn enough about robotic systems architectures to understand the fundamentals, engineering principles, thought processes and methods that must ultimately apply to the development Intelligent Mobile Robotic systems.

8. State of Research - Task 3 • Not explicitly assigned Compilation of a Compendium of Internal Reports relating to the MRS: Completed: March 28 (400+ pages) Objective: Develop a compilation of all completed and edited internal reports that pertain to the development of the Modular Robotic System.

9. State of Research - Task 3 Compendium (~400 pages) Chapter 1: Robotic Architectures Chapter 2: Develop Operational Concept with Mission-Driven Modeling and Simulation A. Scenarios B. Modeling C. Simulation D. Performance Metrics Chapter 3:Define System Requirements, Conduct System Studies to Refine Existing Modular Design and System Configuration A. Mobility and Drive - Robert E. B. Processing and Fusion C. Communications and Networking - Chung-Hao D. Power Sources and Management Chapter 4:Identify, Procure, and Integrate Sensors into Modular Robotics Platform A. Visible / LLTV - Anjana B. Imaging Infrared - Nikhil C. Gamma and Neutron Radiological Sensing D. Laser Range - Santosh Chapter 5:System Management A. Communication and Networking (Bandwidth Management) - Chung-Hao B. Power Management C. Sensor Resource Allocation and Management Chapter 6: Navigation and Control A. Mobility Bricks B. Path Planning and Obstacle Avoidance & Negotiation - Chang Chapter 7: Prototyping and Evaluation A. Mechanical B. Electrical C. Software D. Integration - Rick K. E. Testing and Benchmarking Chapter 8: Documentation, Reporting, and Transition Planning

10. State of Research - Task 4 • Complete a Draft Version of: Modular Robotic System Thesis: Final Format Assigned: April 1 Draft Version Completed: May 3 (~70 pages) Objective: Develop and complete a draft version of a thesis on the present and future state of the Modular Robotic System.

11. State of Research - Task 4 A. Overview of AI Robotic Architectures • Introduction • Hierarchical • Reactive • Hybrid (including reference architectures for each section) B. Modular Robotic Brick Concept • Modular Philosophy, System Composition • Description of the Sensor Bricks C. Illustration of System Concept • Example of the MRS in an Application • Future Direction of System (Architecture & Systems) • Conclusion

12. State of Research - Task 4 Purpose: The thesis shows the processes elements, architectural frameworks and possible future directions of the MRS.

13. State of Research - Task 4 1. Introduction 2. Hierarchical Paradigm Overview Attributes Advantages Reference Architecture Summary 3. Reactive Paradigm Overview Attributes Advantages Reference Architecture Summary 4. Hybrid Paradigm Overview Attributes Advantages Reference Architecture Summary Highly Completed

14. State of Research - Task 4 5. Modular Robotic System Modular Philosophy Brick Components Sensor Block Components 6. Description of System Bricks Visual Thermal Laser Range Radiological Completed, to be additionally revised Completed, to be additionally revised

15. State of Research - Task 4 7. Application of the MRS Introduction System and Deployment Issues The Inspection Scenario 8. Future Direction of the MRS Hardware Architecture 9. Conclusion Largely Complete, to be additionally revised Concept Model Drafted, to be additionally developed & revised

16. Research Focus - Task 2 Robotic Architectures

17. Research Focus - Task 2 • Some Impeding Issues with Robotic Architectures: • Programmatic Vision:Architecture and infrastructure are often not recognized as exciting/necessary areas of work. • Not Invented Here:Fear of unknown performance, quality, and support. • Flexibility:Each system/engineer has specific needs. Seen as an unnecessary imposition by some. • Overhead:Resource constrained tasks lack time to import from, or export to a larger software system. Must build everything ourselves from scratch mentality. • Critical Mass:Might be useful once in place, but no one wants to buy in early. • Learning Curve: Resistance to transition from old method (Laissez-Faire). • Technical Vision:Only meaningful if you're primarily a system architect; otherwise typically falls in one of the above categories. • (Courtesy of Jet Propulsion Laboratory)

18. Research Focus - Task 2 • So, why do we need architecture? • Duplicative efforts prevent attainment of critical mass: • Parallel Duplication: At least 4 major JPL rover efforts are using 4 different software environments (Rocky 7/8, Athena-Rover and MusesCN, FIDO/SRR, TMR). Similar story for Ames, CMU, etc. • Serial Duplication: New starts of projects often wipe the slate clean to eliminate old system problems and lack of familiarity or trust with previous product. Typically, software with legacy is due solely to a single individual, not the community. Follow software community lead: (Modularization) • Open source movement: The value of shared software is illustrated byLinux, GNU, and newly announced Intel Computer Vision Library effort. • Object oriented design: It dominates software development, especially in industry, but is under-utilized in robotics. Leveraging complimentary efforts: • MDS and X2000: JPL has large scale efforts in spacecraft hardware (X2000) and software (Mission Data System) which promise an infrastructure to be leveraged and expanded. • Software sharing: Across related tasks within NASA is often arduous and rare. (Courtesy Jet Propulsion Laboratory)

19. Research Focus - Task 2 Robotic Architectures • History => (to what kind of system does this apply) • Basic Forms => (architectural paradigms) • Applicational Forms => (reference architectures)

20. Research Focus - Task 2 When considering architectures for robotic systems… Stationary base, moving appendages – Industrial Robots Mobile Robotic Systems

21. Research Focus - Task 2 What is Architecture? A principled way of organizing a system.

22. Research Focus - Task 2 How are systems classified? Robotic Control Systems are classified in two ways: 1. By their relationship to the primitives of Sense, Plan, Act. 2. By the way that sensory data is processed and distributed.

23. Research Focus - Task 2 Paradigm – most basic representation • Hierarchical Paradigm 2. Reactive Paradigm 3. Hybrid Paradigm

24. Research Focus - Task 2 Task 2: Report of AI Robotic Architectures • In the domain of calculus problems, Cartesian and polar coordinates represent two differentparadigmsfor viewing and manipulating a problem. Both produce the correct answer, but one takes less work for certain problems. • Applying the right paradigm makesproblem solving easier. Therefore, knowing the paradigms of AI robotics is one key to being able to successfully implement a robotic system for a particular application.

25. Research Focus - Task 2 1. The Hierarchical Paradigm

26. Sense Plan Act Research Focus - Task 2 1. The Hierarchical Paradigm 1. Monolithic – never changes 2. Horizontal Decomposition

27. Research Focus - Task 2 • 1. The Hierarchical Paradigm • Fundamental Issues • The flow of control between the components is unidirectional and linear. Information flows from sensors to the world model, to planning, to acting (effectors); never in the reverse direction. • The execution of this paradigm is analogous to execution of a computer program. • The information is in the composite structure, not the primitives. • The intelligence of the system lives in the planner or the programmer, not the execution mechanism.

28. Research Focus - Task 2 • 1. The Hierarchical Paradigm • Fundamental Issues - Advantages • The primary advantage of the Hierarchical Paradigm is that it provides an ordering of Sense-Plan-Act. • Ordering facilitates attention. • Excellent choice when the global model is highly structured.

29. Research Focus - Task 2 • 1. The Hierarchical Paradigm • Fundamental Issues - Disadvantages • Planning and World-Modeling are difficult problems. Dependence on the Global World model. • Open-loop plan execution is clearly inadequate in the face of environmental uncertainty and unpredictability. • Sensing and Acting are always disconnected.

30. Research Focus - Task 2 Sensors Extract Features Combine features into Model Plan Tasks Task Execution Motor Control Actuators 1. Sense 2. Plan 3. Act Hierarchical Paradigm: horizontal S, P, A decomposition.

31. build maps explore Actuators Sensors wander avoid collisions sense sense sense sense act act act act Research Focus - Task 2 2. Reactive Paradigm Reactive Paradigm: Vertical decomposition of tasks into a Sense-Act organization

32. Research Focus - Task 2 • 2. Reactive Paradigm • Fundamental Issues • Reactive robotic architectures are derived from Behavior-based systems. • Behavior-based refers to the fact that these systems exhibit various behaviors, some of which are emergent. • Planning requires a relatively complete knowledge about the world as well as some predictions about the outcome of its action.

33. Research Focus - Task 2 • 2. Reactive Paradigm • Issues & Characterization • Behavior representation – how is action represented? • What time scale is used for action? • Basis for behavior specification – is a biological model used? • These systems are characterized by tight coupling between sensors and actuators, minimal computation, and a task-achieving “behavior” problem decomposition.

34. Research Focus - Task 2 • 2. Reactive Paradigm • Other Characteristics: • Under a vertical decomposition, an robotic agent starts with primitive survival behaviors and evolves new layers of behaviors which either reuse the lower, older behaviors, inhibit older behaviors, or create parallel tracks of more advanced behaviors. • Each layer has access to sensors and actuatorsindependently of any other layers. If anything happens to an advanced behavior, the lower level behaviors would still operate. This return to a lower level mimics degradation of autonomous functions in the brain. • Functions in the brain stem (such as breathing) continueindependently of higher order functions (such as counting, face recognition, task planning), allowing a person who has brain damage from a car wreck to still breathe, etc.

35. Plan Plan Act Sense Sense Sense Plan Act Plan Sense Act Research Focus - Task 2 3. The Hybrid-Deliberative Model (Hierarchical Paradigm) High Level Low Level

36. Research Focus - Task 2 • 3. The Hybrid-Deliberative Model(Hierarchical Paradigm) • Fundamental Issues • Based on Hierarchical Paradigm. (Top-down approach) • Planning requires a relatively complete knowledge about the world as well as some predictions about the outcome of its action. • In a dynamic world, objects may move, so it is dangerous to rely on past information that may no longer be valid. • Representational World Models are constructed from both a priori knowledge about the environment and incoming sensor data in support of deliberation.

37. Research Focus - Task 2 • 3. The Hybrid-Deliberative Model(Hierarchical Paradigm) • Characteristics • Hierarchical in structure; similar to the organization of commercial businesses, or military command and control. • Control occurs in a predetermined manner, flowing through the hierarchy, up and down. • Planning scope changes, descending the hierarchy as time requirements are shorter and spatial consideration become more localized.

38. Behavior percept Behavior percept Behavior Perceptual Schema Motor Schema Perceptual Schema Motor Schema Perceptual Schema Motor Schema percept Sensor 1 Sensor 2 Actuators Research Focus - Task 2 3. The Hybrid Deliberative-Reactive Model Behavior-specific sensing organization similar to the Reactive Paradigm: sensing is local, sensors can be shared, and sensors can be fused locally by a behavior. Chief Difference: the reactive elements can be deliberated instead of reacting automatically! (Bottom-Up Approach)

39. Research Focus - Task 2 3. The Hybrid Deliberative-Reactive Model • Regardless of the bottom-up or top-down approach for including non-behavioral intelligence, architectures which use reactive behaviors, but also incorporate planning, are now referred to as being part of the Hybrid-Deliberative Paradigm. • At first, Hybrids were viewed as an artifact of research, without any real merit for robotic implementations. • Some researchers went so far as to recommend that if a robot was being designed to operate in an unstructured environment, the designer should use the Reactive Paradigm. If the task was to be performed in a knowledge-rich environment, easy-to—model, then the hierarchical paradigm was preferable, because the software could be engineered specifically for the mission. Hybrids were believed to be the worst of both world, saddling the fast execution times of reactivity with the difficulties with the difficulties in developing hierarchical models.

40. Research Focus - Task 2 4. Reference Architectures • Ideally, an architecture is generic; It should provide a method for implementing a paradigm by embodying the principles in a concrete way.It should have many reusable pieces for other robot platforms and tasks. • “Houses must have bedrooms, bathrooms, a kitchen, etc… Restaurants must have a dining area, a kitchen, refrigeration closet, and an office, etc.” • A reference architecture is a sufficiently specific formulation such that hypotheses can be constructed, tested, and either validated or disproved. Once validated, a reference model architecture can form the basis for an engineering methodology whereby intelligent systems can be designed and built to meet specified requirements. • “Our House will have 2 bedrooms, 2 bathrooms, a kitchen, a living room, a garage, etc…” Since our house is the best house for 2 adults and 1 child, every house for 2 adults and 1 child should be modeled in this manner.”

41. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Nested Hierarchical Controller (NHC) • developed by:Meystel.

42. Research Focus - Task 2 • 4. Reference Architectures (Reactive) • Subsumption • Potential Fields

43. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical-Hybrid) • Real-time Control System (RCS) • developed by:Albus(NIST).

44. Pilot sensors sensors sensors Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Nested Hierarchical Controller (NHC) Sense Plan Mission Planner Navigator World Model/ Knowledge Base Act Low-level Controller drive steer

45. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Nested Hierarchical Controller (NHC) • Major Contribution: decomposition of PLANNING into three (3) different functions or subsystems aimed at supporting navigation: • Mission Planner • Navigator • Pilot

46. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Nested Hierarchical Controller (NHC) • Functionality: • 1. Gathers information (sensors) + a priori knowledge = World Model • (SENSE) • 2. Plan what actions the robot should take. • (PLAN) • 3. Planning for navigation – a local procedure. • Mission Planner • Navigator • Pilot • Each has access to the world model in order to compute their portion of planning. • (PLAN) • 4. Pilot module generates specific actions for the robot’s movement. • (ACT)

47. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Nested Hierarchical Controller (NHC) • Advantages: • 1. Interleavesplanning and acting. The robot comes up with a plan, begins to execute the plan, then changes it if the world is different than it expected. • 2. Decomposition is hierarchical in intelligence and scope. • The Mission Planner is “smarter” than the Navigator, who is smarter than the Pilot. The Mission Planner is responsible for a higher level of abstraction than the Navigator, etc. • Other Architectures (RCS) make use of NHC. • Uses parallel processing for navigation planning and control (acting) – does both at the same time.

48. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Nested Hierarchical Controller (NHC) • Disadvantages: • Decomposition of the planning function is appropriate only for navigation tasks. • The division of responsibilities seems less helpful, or clear for tasks such as picking up a box, rather than just moving over to it. • The role of a Pilot in controlling end-effectors is not clear.

49. Research Focus - Task 2 • 4. Reference Architectures (Hierarchical) • Real-time Control System (RCS) • Fundamentals: • Albus (1960’s) noticed a problem: no common terms, no common set of design standards. This made industry and equipment manufacturers leery of AI – an expensive robot that would not be compatible with robots purchased in the future. • Albus developed a very detailed architecture called the Real-time Control System Architecture (RCS) to serve as a guide for manufacturers who wanted to add more intelligence to their robots. RCS uses NHC in its design.

50. Sample Activities Sense Plan Act Mission Planner Plan Plan Act Sense Sense Sense Navigator Plan Act Pilot Plan Sense Act Sense Model Act Research Focus - Task 2 RCS Reference Architecture uses NHC as part of its structure.