Control Agents making Robotics a reality

1. Control Agents making Robotics a reality Dr. Reuven Granot

2. Outline • The need for unmanned systems. • Why tele-robotics? • A new layer of software should be developed in order to support application users in fields of robotics and other semi or autonomous systems to develop more effectively their specific applications. • follow upcoming standards like JAUS and CORBA. • follow the Distributed and Object Oriented paradigm, but be more than an object or an expert system by being reactive, autonomous and proactive. • transparently take care of inter agent communication and other basic tasks a control agent needs. • a control agent is performing some control task while communicating with other agents or humans as needed. • Spark Robotics develops the needed infrastructure for Human Supervised Autonomous Control Agents.

3. The Need of Unmanned Systems Is well recognized to perform tasks that are: • DDD • Dull • Dirty • Dangerous • Distant – at different scale • Macro: space, • Micro: telesurgery, micro and nano devices Illustration from NASA and ORNL publication Robot colony on a site preparation task for a PV tent on Mars.

4. Remote Controlled vehicles in combat environment • RC is still preferred by designers • Simple, but not practical for combat or other very demanding environment because the human operator: • is very much dependent upon the controlled process • needs long readjustment time to switch between the controlled and the local environment. • The state of the art of the current technology has not yet solved the problem of controlling complex tasks autonomously in unexpected contingent environments. • dealing with unexpected contingent events remains to be a major problem of robotics. • Consequence: A human operator should be able to interfere: remains at least in the supervisory loop.

5. The needed control metaphor: Human Supervised Autonomous • All these applications require an effective interface between the machine and a human in charge of operating/ commanding the machine. • We are suggesting to perform Human Supervised Autonomous Control, which is known as tele-robotics. • That be done using the software agent technology

6. The Tele-robotics paradigm Telerobotics is a form of Supervised Autonomous Control. A machine can be distantly operated by: • continuous control: the HO is responsible to continuously supply the robot all the needed control commands. • a coherent cooperation between man and machine, which is known to be a hard task. Supervision and intervention by a human would provide the advantages of on-line fault correction and debugging, and would relax the amount of structure needed in the environment, since a human supervisor could anticipate and account for many unexpected situations.

7. The spectrum of control modes. A telerobot can use: • traded control:control is or at operator or at the autonomous sub-system. • shared control: the instructions given by HO and by the robot are combined. • strict supervisory control: the HO instructs the robot, then observes its autonomous actions. Solid line= major loops are closed through computer, minor loops through human.

8. Human Robot Interaction • In supervised autonomously controlled equipment, a human operator generates tasks, and a computer autonomously closes some of the controlled loops. • Control bandwidth • Robot SW: high • Human response: slow • Human Operator is expected to • Control several machines/ equipment/ systems • be capable to deal with other duties (like a combat environment requests) in somehow relaxed mode of operation. Make the machine an agent in human operator’s service.

9. Software Infrastructure • Architecture • Distributed • Objects or some more sophisticated entities? • Communication • Development Environment • Language • Tools • Reusable units

10. What is a Robot Architecture? • There are many different ways in which a robot control program can be put together. • In order to program a robot in a structured and principled fashion, we use an appropriate robot control architecture.

11. Robot Architecture • A control architecture provides a set of principles for organizing a control system. • It provides structure and constraints which aid the designer in producing a well-behaved controller. • To be successful a system designer has to decide how (in what order? with what priority?) does he put together multiple feedback controllers in a principled fashion and how to scale up control to more complex robots, which generally have to deal with many behaviors at once. • How would you put multiple feedback controllers together? • How would you decide which one to use when and forhow long and in what priority relative to the others?

12. Misconceptions. 1. Programming languages are implementation tools and not architectures. 2. The issue of fundamentalpower or expressivenessof a robot control architecture: claims have been made about one control architecture being able to compute fundamentally more than another. • This cannot be true if we understand that all are grounded in Turing-complete programming languages. • However, the above is not to say that all architectures are the same. On the contrary, architectures impose strong constraints on how robot programs are structured, and the resulting control software ends up looking very different.

13. Robot Architecture Major Classes/Categories Intuitively, this means that there are infinitely many ways to structure a robot program, but they all fall into one ofmajor classes/categories of control: • Deliberative Control : Think hard, act later. • SPA, serial, complete each step first – then proceed • Reactive Control : Don’t think, (re)act. • Direct connection between perception to action, no memory, no planning. • Hybrid Control : Think and act independently, in parallel. • Deliberative and Reactive modules run independently at different time scales • Behavior-Based Control : Think the way you act. • Distributed by behavioral task decomposition • Each behavior has its restricted planning and execution capabilities

14. The Choice of the Control Architecture • When it comes to more complex robots, i.e., robots that have to deal with complex environments and complex tasks, the control architecture becomes very important. • The different properties of an environment that will impact the robot's controller (and therefore the choice of control architecture): • noisy, • speed/response time of sensors and effectors • total/partial hidden state/ observable • discrete v. continuous state ; static v. dynamic ... • Similarly, the properties of the robot's task impact the choice of the control architecture. The task requirements can constrain the architecture choice.

15. Parallel Processing Paradigm. • As robot control is engaged to deal with more complex problems, centralized supervisory architecturesencounter barriers to real time performance caused by computational complexity coupled with insufficient computing power and sensor resources. • Despite startling advances in hardware and software technology and similarly surprising cost reductions, these fundamental barriers remain unchanged. • The parallel-processing paradigm may be the only technology to challenge this fact.

16. Asynchronous and Synchronous processes • The other leading architectural trend is typified by a mixture of asynchronous and synchronous control and data flow. • Asynchronous processes are characterized as loosely coupled and event-driven without strict execution deadlines. • Synchronous processes, in contrast, are tightly coupled, utilize a common clock and demand hard real-time execution.

17. Some Criteria for Selecting a Control Architecture • support for parallelism: the ability of the architecture to execute parallel processes/behaviors at the same time. • hardware targetability: • how well the architecture can be mapped onto real-robot sensors and effectors. • how well the computation can be mapped onto real processing elements (microprocessors). • run-time flexibility: does the architecture allow run-time adjustment and reconfiguration? It is important for adaptation/learning. • modularity: how does the architecture address encapsulation of control, how does it treat abstraction? • Does it allow many levels, going from feedback loops to primitives to agents? • Does it allow re-use of software?

18. Some Criteria for Selecting a Control Architecture • niche targetability: how well the architecture allows the robot to deal with its environment • robustness: how well does the architecture perform if individual components fail? How well does it enable and facilitate writing controllers capable of fault tolerance? • ease of use: how easy to use and accessible is the architecture? Are there programming tools and expertise? • performance: how well does the robot perform using the architecture? Does it act in real-time? Does it get the job done? Is it failure-prone? The above issues allow us to compare and evaluate different architectures relative to specific robotic designs, tasks, and environments. But not all tasks, environments, and designs are comparable.

19. Time Scale. Time-scale is an important way of distinguishing control architectures. • Reactive systemsrespond to the real-time requirements of the environment, • while deliberative system look ahead (plan) and thus work on a longer time-scale. • Hybrid systems must combine the two time-scales in an effective way, usually requiring a middle layer; consequently they are often called three-layer architectures. • Finally, behavior-based systems attempt to bring thedifferent time-scales closer together by distributing slower computation over concurrent behavior modules.

20. Representation • Another key distinguishing feature between architectures is representation of the world/environment, also called world modeling. • Some tasks and architectures involve storing information about the environment internally, in the form of an internal representation of the environment. • For example, while exploring a maze, a robot may want to remember a sequence of moves it has made (e.g., "left, left, right, straight, right, left"), so it can back-track and find its way. • Thus, the robot is constructing a representation of its path through the maze. • The robot can also build a map of the maze, by drawing it using exact lengths of corridors and distances between walls, etc. . • This is also a representation of its environment, a model of the world. • If two robots are working together, and one is much slower than the other, if the fast robot remembers/learns that the other is always slower, that is also a type of a model of the world, in this case, a model of the other robot.

21. Different World Models. • There are numerous aspects of the world that a robot can represent/model, and numerous ways in which it can do it, including: * spatial metric or topological: maps, navigable spaces, structures * objects instances of detectable things in the world • * actions outcomes of specific actions on the self and environment • * self/egostored proprioception: sensing internal state, self- limitations, etc. • * intentionalgoals, intended actions, plans • * symbolicabstract encoding of state/information

22. Amount and Type of Representation • The amount and type of representation or modeling used by a robot is critically related to the type of control architecture it is using. • Some models are very elaborate; they take a long time to construct and are therefore kept around possibly throughout the lifetime of the robot's task (for example detailed metric maps). • Others may be relatively quickly constructed and transient, used quickly and discarded or updated (for example the next few steps in a short plan, the immediate goal, etc.) • How long it takes to construct/build a model is an important aspect of the robot's controller.

23. Amount and Type of Representation • How long it takes to use it is equally important. Consider maps again: • * it takes a long time to construct an accurate and detailed metric map, because it requires exploring and measuring the environment. • * furthermore, it takes time to usesuch a map as well (even if it took no time to construct it, but it was given to the robot by the designer); • * onemust find the free/navigable spaces in the map, and then • * search through those to find the best path to the goal. • Similarly, any internal model can require time to construct and be used, and these timing requirements directly affect the time-scale of the controller.

24. Regarding the architecture of robotic systems, we discussed so far two key issues distinguishing architectures, as had to do with • time-scale (reactive) and • looking ahead (deliberative). • A third key issue we need to consider is modularity, i.e., the way in which the architecture decomposes into components.

25. What is a behavior? • An individual behavior is a stimulus/ response pair for a given environmental setting that is modulated by attention and determined by intention.  Attention: prioritizes tasks and focuses sensory resources and is determined by the current environmental context.  Intention:determineswhich set of behaviors should be active based on the robotic agent’s internal goals and objectives.  Apparent or emergent behavior:the global behavior of the robot as a consequence of the interaction of the active individual behaviors. • Behaviors serve as the basic building blocks for robotic actions.

26. What are Behaviors? • are feedback controllers (closed-loop,extended in time ) • achieve specific tasks/goals • are typically executed in parallel/concurrently • can store state and be used toconstructworld models/representation • typically has the following properties: • can directly connect sensors and effectors • are typically higher-levelthan actions • can also take inputs from other behaviors and send outputs to other behaviors • when assembled into distributed representations, behaviors can be used to look ahead but at a time-scale comparable with the rest of the behavior-based system.

27. Behaviors and Modularity Behavior-based systems are not limitedin the ways that reactive systems are. As a result, behavior-based systems have the following key properties: • 1) the ability to react in real-time. • 2) the ability to use representations to generate efficient (not only reactive) behavior. • 3) the ability to use a uniform structure and representation throughout the system (so no intermediate layer).

28. Assembling Behaviors. • Systems are constructed from multiple behaviors. • Emergent behavior implies a holistic (attention to the “whole”) capability where the sum is considerably greater than its parts. • Emergence is “the appearance of novel properties in whole systems”. • Intelligence emerges from the interaction of the components of the system. • Coordination functions are algorithms used to assemble behaviors. • Conflict can result when two or more behaviors are active, each with its own independent response.

29. The Agent • An agent is a computer system capable of autonomous action in some environments. • A general way in which the term agent is used is to denote a hardware or software-based computer system that enjoys the following properties: • autonomy: agents operate without the direct intervention of humans or others, and have some kind of control over their actions and internal state; • social ability: agents interact with other agents (and possibly humans) via some kind of agent-communication language; • reactivity: agents perceive their environment, (which may be the physical world, a user via a graphical user interface, or a collection of other agents), and respond in a timely fashion to changes that occur in it; • pro-activeness: agents do not simply act in response to their environment; they are able to exhibit goal-directed behavior by taking the initiative.

30. Agents and Behaviors • Behavior is defined as the way how we/people observe the system/robot acts/behaves. • The robot system is NOT aware of what we know about it. • What makes the system act as we observe is its software. • Behaviors are implemented by agents.

31. Interface Agent • A software entity, which is capable to represent the human in the computer SW environment. • It acts on behalf of the human • Follows rules and has a well defined expected attitude/ action. • May be instructed on the fly and may receive during mission updated commands from the human operator. We need to build agents in order to carry out the tasks, without the need to tell the agents how to perform these tasks.

32. Agent control loop • agent starts in some initial internal state i0 . • observes its environment state e, and generates a percept see(e). • internal state of the agent is then updated via next function, becoming next_(i0, see(e)). • the action selected by agent is action (next(i0, see(e)))) This action is then performed. • Goto (2).

33. Agents • Agents may act inside the robot software to implement behaviors: • Feedback controllers • Control subassemblies • Perform Local Goals/ tasks • Differ from Objects • autonomous, reactive and pro-active • encapsulate some state, • are more than expert systems • are situated in their environment and take action instead of just advising to do so.

34. Agent control loop • agent starts in some initial internal state i0 . • observes its environment state e, and generates a percept see(e). • internal state of the agent is then updated via next function, becoming next_(i0, see(e)). • the action selected by agent is action (next(i0, see(e)))) This action is then performed. • Goto (2).

35. Advantages • Software engineering • may be developed as a distributed project • reusability • Distributed control • each primitive agent is responsible for only one primitive task, which controls only one signal. • Man machine interface • Controls interfaceof the human operator with the system.

36. The Control Agent • The agent is a control subassembly. • It may be built upon a primitive task or composed of an assembly of subordinate agents. • The agent hierarchy for a specific task is pre-planned or defined by the human operator as part of the preparation for execution of the task. • The final sequence of operation is deducted from the hierarchy or negotiated between agents in the hierarchy.

37. Human Operator • Monitors the activities and the performance of the assembly of agents. • Responsible for the completion of the major task (global goal) • may interfere by sending change orders. • emergent (executed immediately, without considering any possibility to return to achieve the goal/in the shortest possible way) • “as is ordered” or • normal • checked by the interface agent • which negotiates execution with other agents in order to optimize execution performance • Conflict resolution algorithm • defined as default, or • defined by the human operator in its change order or • suggested to the operator by a simplified decision support algorithm.

38. Task-level supervisory control system block diagram. HO raw robot outputs formatted outputs control signals Controlling agent Task level controller Robot hardware desired tasks • An agent can be considered as a control subassembly, also called behavior. • The feedback is given to the agent in both processed and raw form.

39. RCS Embeds a hierarchy of agents within a hierarchy of organizational units: Intelligent Nodes or RCS_Nodes. JAUS From M. W. Torrie A hierarchy of Commanders different resolution in space and time

40. Commanded Task (Goal) Value Judgment Perceived Objects & Events Plan Evaluation Situation Evaluation Plan Results Update Plan Sensory Processing World Modeling Behavior Generation Predicted Input State Knowledge Database Commanded Actions (Subgoals) Observed Input RCS_Node

41. Intelligent Node within RCS Command tasks (Goals) status Operator Interface Sensory Output RCS_Node VJ Peer Input Output SP BG WM KDb reactive Sensory Input deliberative Command actions (Subgoals) status SP-WM-BG close a reactive feedback control loop BG-WM-VJenable deliberative planning and reasoning

42. Agents in Behavior Generation hierarchy • Tasks are decomposed and assigned in a command chain. • Actions are coordinated • Resources are allocated as plan approved. • Tasks achievements are monitored (VJ) • Execution in parallel