Chapter IV – MULTI-ROBOT COOPERATION

1. Computer Engineering DepartmentINTRODUCTION TO ROBOTICSCOE 484Dr. Mayez Al-Mouhamed SPRING 2008 Chapter IV – MULTI-ROBOT COOPERATION

2. Plan • Introduction to Multi-Robot Cooperation, “A Functional Architecture for a Team of • Fully Autonomous Cooperative Robots”, P. Lima et al., Instituto de Sistemas e Robotica, Portugal. • An application oriented study, “Formulation and Implementation of RelationalBehaviors for Multi-Robot CooperativeSystems”,B van der Vecht and P. Lima.

3. A Functional Architecture for a Team of • Fully Autonomous Cooperative Robots • Pedro Lima, Rodrigo Ventura, Pedro Apar cio, Luis Custodio • Instituto de Sistemas e Robotica • Instituto Superior Tecnico • Av. Rovisco Pais, 1 | 1049-001 Lisboa • PORTUGAL • Abstract • A three-level functional architecture for a team of mobile robots is described in detail, including the definition of the role assigned to each level, the main concepts involved, and the corresponding implementation for each individual • robot. • The architecture is oriented towards teams of fully autonomous cooperative robots, able to carry out different types of cooperative tasks. • Complexity is reduced by the decomposition of team strategies into individual behaviors, which in turn are composed of primitive tasks. • Relationships among robots of the team are modeled upon the joint intentions frame-work. An application to Robotic Soccer and some of its preliminary results are presented.

4. 1- Introduction and Motivation • Different functional architectures have been proposed in distributed artificial intelligence and intelligent control literature to handle the complexity of controlling a fully autonomous mobile robot or a team composed of such • robots. • A common concept among those approaches is the existence of atomic primitive tasks or behaviors which are the kernel of the architecture. • Tasks executed by the robot result from the composition of those entities. • The main difference between the existing approaches concerns the interaction among the atomic entities. • While some authors allow full fexibility, so that a team behavior emerges from a negotiation between running behaviors (Drogoul and Collinot, 1998), others prescribe, with different flexibility levels, the task decomposition into primitive tasks [Albus, 1991], to an extent which may even forbid any direct communication between primitive tasks Saridis, 1995].

5. A three-level functional architecture for a team of mobile robots is introduced in this paper. The architecture is oriented towards teams of fully autonomous cooperative robots, able to carry out different types of cooperative tasks. • The level splitting is inspired by the work of Drogoul, but there are important differences regarding the modeling of the relational level, which describes inter-agent negotiation and role assignment. • The joint intentions framework (Tambe, Jennings, Cohen and Levesque), provides a solid foundation for team work modeling, and will be used in this work to support the implementation of the relational level. • Complexity is reduced by the decomposition of team strategies (i.e., what should be done) into individual behaviors, which in turn are composed of primitive tasks. • A set whose elements are the behaviors assigned to each robot of the team is designated as the tactics (i.e., how to do it) for a given strategy. • An application to Robotic Soccer and some of its preliminary results developed during and after the RoboCup'98 contest are presented. • The paper is organized as follows. Section 2 describes the team and individual architectures, with details of teamwork modeling at the relational level and of the foreseen / current implementation for the introduced concepts. Section 3 maps the concepts onto a robotic soccer team. Section 4 closes the paper with some preliminary conclusions and reference to future work. • Note: Team Strategy = set of strategies, Strategy = {Tactics}, Tactic = {Active Agents}, an agent is a behavior that is assigned to a robot at a given time. Example of Agents are the Attacker, Goally, Supporter, Defender, etc.

6. The Functional Architecture • 2.1 Team Architecture • The team architecture, based on the 3-level agent team architecture first proposed by Drogoul is depicted in Figure 1. Our interpretation of Drogoul's three-levels follows: • Organizational level: establishes the Strategy to be followed by the whole team, given the Team State and World State. • The Team State corresponds to the current set of behaviors under execution. • The World (Game) Stateis divided in two categories: • Game situations reached upon the application of RoboCup tournament rules (e.g., kickoff, end-of-game, penalty-for, penalty-against); • Team evaluation of current game status (e.g., losing & close to the end of the game, ball close to our goal). • Strategies can be divided in, at least, two major parts: • pre-programmed scenarios for game situations in game state category 1 above; • dynamic strategies (e.g., defend, attack, counter-attack), corresponding to game state category 2 above. • Note: A Team State specifies a tactic which is a set of Active Agents. • The World-Game State refers to what it has been achieved since the beginning of the game and how this may influence the selection of a tactic.

7. The Functional Architecture • 2.1 Team Architecture • Relational level: at this level, relationships among robots are established. The robots negotiate and eventually come to an agreement about some team and/or individual goal. • Moreover, behaviors are assigned to the individual robots, after a selection from within behavior sets representative of alternative tactics for the strategy selected by the organizational level. • The selected behavior set depends on the current World State and Team State. Behavior assignments may also be temporarily modified as a result of inter-robot negotiations. • Individual level: encompasses all the available robot behaviors. Those include the primitive tasks (e.g., Seek Ball, Kick Ball, Rotate Left) and their relations. • A behavior corresponds to a set of purposive (i.e. having a goal) primitive tasks sequentially and/or concurrently executed. A primitive task is a sense-think-act loop (STA loop), a generalization of a closed loop control system which may include motor, ball tracking or trajectory following control loops, to name a few. STA loops are composed of the following key components: • Goal: the objective to be accomplished by the primitive task (e.g., moving to a given position plus orientation (pose) set-point, tracking the ball in the image); • Sense: sensor data required to accomplish the goal (e.g., distance to an object, object position in an image); • Think : the actual algorithm which, using the sensor data, does what is required to accomplish the goal (e.g., motion controller, ball visual servoing); • Act: the actions associated to the think algorithm (e.g., moving the wheel motors).

8. Figure 1: The functional architecture of the ISocRob team.

10. The sequence of primitive tasks is traversed as the logical conditions associated with the connections among them become true. The logical conditions are defined over a predicate set. There are two predicate classes: • predicates which check the value of a given variable (e.g., the variable goal in last seen (goal)= left ); • predicates which check the past occurrence of a given event (e.g., the see (ball) predicate checks whether the ball became visible). • It is assumed that the opposite event must occur in order to modify the predicate's truth value. • A world model is required to provide information to the relational and organizational levels regarding the world state. Since all computation is supposed to be distributed over the team members, with no external storage available, a distributed world model representation is required, containing all the relevant information for negotiation between agents, and in general the result • of processing raw data, for primitive tasks usage. A distributed blackboard is proposed to implement the world model [Ventura].

11. The schematic block diagram of the implementation, at each individual robot, of the team architecture is depicted in Figure 2 - (a).

12. 2.2 Individual Robots Architecture • Each individual robot is provided with all the three levels of the team functional architecture. However, the organizational level is only active in one of the robots, assigned as the team captain. • The remaining robots have a dormant organization level, to ensure fault-tolerance: • whenever the captain robot has a malfunction, the next robot in the list takes over as the captain. The list has no special order since, from the hardware stand-point, all robots of the team are currently homogeneous. • In an non-homogeneous population, the potential captains (from a computational capacity stand point) should be sorted according to their descending computational power. • An agent-based programming language is under development Ventura, to provide the team strategist (the coach in robotic soccer) with the means to program the population in order to achieve the strategic objectives, embedded in the behaviors and in the primitive task STA loops. • Each of the above concepts will be implemented as follows: • The strategy is determined at the organizational level by a state-machine whose transitions are traversed upon the matching of specific world states, and whose states define the current strategy. Therefore, strategies change when the world state (as perceived by the team) changes; • Tactics selection, including behavior selection, negotiation, and temporary behaviors modification, is implemented by relational rules at the relational level; • A behavior consists of a state-machine, where each state corresponds to an STA loop and each transition has associated logical conditions defined over the predicate set described in subsection 2.1;

13. 2.2 Individual Robots Architecture (Cont) • Team organization is necessarily a centralized operation. As such, decisions on strategies must be taken by a single agent, designated as the captain. • Thus, the organizational state machine runs in the captain. • To increase team robustness, whenever the current captain does not signal that it is alive for more than a timeout period, a new captain must take control of the team. • A blackboard implements global shared memory and event-based communication. • The blackboard is the sole medium of communication between the agents, supporting the message exchange required for negotiation. One of the key factors will be the distribution of data per robot, which should minimize the need to communicate in order to obtain information (e.g., data obtained from processing an image should be stored in the robot where the image was acquired). • The global team strategy is also stated in the blackboard, as a variable which triggers some of the relational rules. • The schematic block diagram of the implementation, at each individual robot, of the team architecture is depicted in Figure 2 - (a). The behavior selection and spatial coordination are detailed in Figure 2 - (b): the behavior coordinator, selects the correct behavior for the robot, based on the world state and on the strategy and tactics provided by the organizational and relational levels of the team architecture. When a behavior is selected, the corresponding spatial supervisor is also activated. The spatial supervisor ensures that the robot always stays within the influence zone associated to its • active behavior (see Section 3).

14. The behavior selection and spatial coordination are detailed in Figure 2 - (b): the behavior coordinator, selects the correct behavior for the robot, based on the world state and on the strategy and tactics provided by the organizational and relational levels of the team architecture.

15. 2.3 Relational Rules • Relationships among the team robots are established at the relational level of the team functional architecture. • Given a strategy established by the organizational level, different tactics can be used to implement it. • Tactics consist of behavior sets, whose elements are the behaviors assigned to each individual robot of the team. A tactics is chosen based on the current world state, but also on each agent's current internal state. The strategy must specify not only the goal to be attained (e.g. attack, defense) but also criteria to check how close to the goal the team is. • Behaviors can be temporarily modified as a result of inter-robot negotiation, as part of the tactics to attain the goal. • An example is the situation where two teammates, both assigned a forward-like behavior, actively try to get the ball. • In such a case, one of them should signal the other its intention. A negotiation process would follow, where the teammates would determine their distances to the ball, to decide which one should pursue it. • After taking such a decision, the other player should temporarily modify its normal forward-like behavior. The absence of such a relational mechanism leads to situations where team behavior is poor. • Consider the case of two forward-like players with similar behaviors, that often conflict with each other while trying to reach the ball. The key to solve this problem is to endow the team members not only with individual goals, but also make them knowledgeable of the team goals. This is clearly related to concepts such as joint persistent goal, joint intentions and joint commitment [Tambe, Jennings, Cohen and Levesque]. Moreover, it requires communication between team members.

16. 2.3 Relational Rules • For instance, a joint persistent goal is defined by Jennings, as follows: • A team of agents has a joint persistent goal, relative to a main goal q, to achieve a strategy p iff: they all mutually believe that p is currently false; they all mutually believe that they all want p to be eventually true, and until they all come to mutually believe either that p is true, that p will never be true, or that q is false, they will continue to mutually believe that they each have p as a weak achievement goal relative to q. • The example above can be interpreted under this definition. The strategy p (e.g., attack) is a weak achievement goal relative to the main goal q of scoring a goal. Suppose the two players both assumed the Forward behavior as part of the selected tactics. They will pursue the strong goal q • (i.e., they will attempt to score a goal) by executing their Forward behaviors so as to attain p. • A criterion to check whether p is attained is to • Determine whether the players are able to keep playing within their assigned influence zone. • Both players will continue to work towards meeting this and the other criteria which define the attack strategy until they all come to mutually believe either that all the criteria were met, that the criteria will never be attained (e.g., after a time-out), or that 'scoring a goal' is no longer the main team endeavor (e.g., because the game is over). • Working towards meeting the criteria includes temporarily modifying their behaviors to cope with the team goal (e.g., refraining from pursuing the ball). This distinguishes a group of non-cooperative agents whose individual goals just happen to be the same, from a group of cooperative agents which share a common aim. The latter exhibits cooperation and coordination, while in the former the • individual agents compete when the resources are scarce [Jennings].

17. 2.3 Relational Rules • The relational rules implement a recipe which is commonly agreed by all the agents of a team Jennings. • This recipe is embedded in the rules and may either be prescribed initially (i.e., before joint action is started) or evolve over time. • We are currently looking at the possibility of changing the recipe over time using reinforcement learning techniques, based on a performance function which weights the reliability (i.e., the ability to meet specifications) and the cost (computational or other) of a given recipe [Lima and Saridis]. • Reinforcement learning should be able to determine the recipe (to achieve a joint intention) that best balances cost and reliability.

18. 3 Application to a Soccer Robot Team Some of the concepts described in the previous section will now be mapped onto a team of fully autonomous soccer robots. Figure 3 presents a functional division of the field in several regions. These are zones where robots try to locate themselves inside the field, according to their assigned behaviors, e.g., defenders should stay inside the D zone and Forward players should stay inside the F zone. This division helps the assignment of influence areas to players. Figure 3: The field division in actuation areas. Besides Defense (D), MidDefense (MD), MidForward (MF) and Forward (F), further divisions are introduced to increase the field resolution. Along the field longitudinal axes, the field is divided in Left (L), Center (C) and Right (R) parts. This division is particularly useful when the team has more than one player acting in the same functional area (e.g., L and R defenders).

19. 3.1 Player Behaviors • As explained before, behaviors are composed of primitive tasks sequentially or concurrently executed. A field influence zone is associated to each behavior. Several behaviors must be implemented in a robot soccer team. • The most significant ones, whose influence zones are depicted in Figure 4, are: • GoalKeeper: Defends the goal. To do that, it continuously looks for the ball and, if necessary, leaves the goal area and kicks it away. The state machine in Figure 5-a implements such a behavior. The influence zone is defined by the goal area lines and is shown in Figure 4-a. • Defender: The defender mission is to move the ball from the vicinity of its team goal to the opponents field. If possible, it should try to move the ball to the vicinity of a MidFielder. It should return to its original position (D) when the ball is once again in the opponent's field (see Figure 4-b). GoalKeeper

20. 3.1 Player Behaviors • MidFielder: Such as in real soccer, the MidFielder is able to play in a variety of positions. Its influence zone lies within the MD and MF areas (see Figure 4-c). This player natural ability is to receive the ball from its own team field and decide what to do, based on the other players availability. If a Forward is in the near vicinity of the opponents goal (F area), the MidFielder should try to pass it the ball. • Forward: The Forward behavior induces the player to be in the F zone (see Figure 4-d). If the ball goes into our field, the Forward's mission is to keep track of the ball, although it should not move out of its zone by its own initiative. When the ball moves into the F zone, it must try to take control over it and kick it into the opponents goal. Such a behavior is implemented by the state machine of Figure 5-b. • An alternative implementation would consist of letting the Forward players move up and down the • field, using the lateral L and R corridors.

21. 3.2 Relational Behavior Modification • Individual behaviors can be temporarily modified to allow cooperative relations between teammates as explained in Subsection 2.3. Fig 5-b depicts the state machine which implements the Forward behavior endowed with states and state transitions representing relational rules. • The negotiation implemented corresponds to the example in Subsection 2.3 concerning two Forward players who actively try to get the ball. • The additional state ShouldIGo is entered by a Forward player upon its detection of the ball predicate seeball. • In the ShouldIGo state a message is broadcasted through the blackboard to all teamates stating that this Forward player saw the ball and also the estimate of its distance to the ball. • The occurrence of messages from the other teamates stating that they saw the ball and including the estimate of their distance to the ball is also checked in the ShouldIGo state. The distance to the ball of all the Forward players who saw the ball recently is continuously sorted.

23. 3.2 Relational Behavior Modification • The distance to the ball of all the Forward players who saw the ball recently is continuously sorted. • The sorted distances are checked by the closest distance predicate associated to all the dashed arcs of the state machine in Fig 5-b which are responsible by the temporary behavior modifications with respect to the nonrelational Forward behavior of Fig 5-a. • The most important modification consists of not immediately kicking KickBall state or following (FlwBall state) the ball upon its detection in state SrcBall but rather moving to state ShouldIGo where distances to the ball are compared. • Should the Forward be the closest to the ball among all its Forward teammates or the only one who sees the ball the state machine execution proceeds as in the nonrelational Forward behavior Otherwise the behavior is modified by making the player move to a location close to the other teams goal states SrcGoal and MoveGoal

25. 3.3 Game State • The game state refers to either situations reached as a result of the application of RoboCup rules or to an evaluation of the current game status. State changes are induced by the time ow and teams actions during the game. Examples of game states are as follows: • Game situations: • Game-start - This happens in the beginning of the game, after a goal or when the game restarts after a break; • penalty-for, penalty-against; • end-of-game - This is signaled by an external event (e.g., two whistle blows). • Evaluation of game status: • ball-our-ofield - One of our players has ball possession. The ball is in our field; • ball-nour-oteld - None of our players has ball possession. The ball is in the other team eld; • losing & close to the end of the game; • ball close to our goal • .

26. 3.4 Scenarios for Game Situations Pre-defined scenarios are usually associated with the game states corresponding to game situations (see above). An example is the game-start situation where the players must move to their predetermined start positions (see Fig 6). Self-location of the players must be accomplished at this state as they must be correctly positioned prior to the start of the game. After positioning the players will wait for the external kickoff signal (eg a whistle blow) that signals the start of the game. Fig 6 Players position at game startup

27. 3.5 Dynamic Strategies and Tactics • During the game the ball moves inside and outside of the team midfield. Depending on factors such as the ball position and motion the current game state the current score the number of available players and their behaviors the opponents positions the elapsed time and the current strategy the team strategy may change. This is inspired by real soccer. Possible strategies are: • Defense - The ball must be prevented from entering our field. Should that happen it must be moved into the opponents field. Several defense tactics exist. Two examples of tactics for the Defense strategy are • Strong Defense SD: This strategy points towards creating a continuous physical barrier between the ball and our goal It is aimed at avoiding opponent players from moving towards our goal When repositioning the Defender players should try to avoid occluding the GoalKeeper visibility of the field, ie the DC zone should be free of players see Fig 7-a. • Medium Defense MD: Points towards a strong defense and a good recovery mechanism essential to counterattack The concept is illustrated in Fig 7- b. The difference between the SD and the MD is that in MD not all players are moved into our field. This makes the transition to CounterAttack easier as one of the players stays in the opponent field. Fig 7 - Strong and Medium Defense tactics for the Defense strategy

28. 3.5 Dynamic Strategies and Tactics • Counterattack - A counterattack happens if the team is positioned to move the ball quickly into the opponent field and score a goal. It requires a Defender to handle a possible interception of the ball by an opponent a MidFielder to pass the ball into the Forward area and a Forward player to kick • the ball into the opponent goal. • Attack - Under this strategy the whole team moves forward Besides the GK there is only one player in our field. This movement requires two MidFielders one in the MD zone and another in MF zone and one Forward. The idea is to have the ball passed from the M zone into the F zone where a Forward player is to pick it up and kick at the goal Fig 7 - Strong and Medium Defense tactics for the Defense strategy

29. Preliminary Conclusions and Future Work • Currently our midsize real robots are capable of simple but essential behaviors composed of primitive tasks such as following a ball kicking a ball scoring goals and defending the goal using vision based sensors see Fig. • Current available behaviors include shooting at an empty goal starting from increasingly more difficult situations or defending the goal by permanently tracking the ball and kicking it out of the goal area as soon as it gets too close. • One Forward vs one Defender and a GoalKeeper have also been successfully tested in live action. • Our current and future work is centered on four main topics development of the self-localization system based on a vision camera and a mirror update and tuning of the primitive tasks software • design and implementation of an agent-based programming language suitable for multi-agent systems study and development of a teamwork model and its integration with the team functional architecture. • Among those self-localization is perhaps the most essential. The functional architecture described in this paper relies on a at least rough awareness by each robot of its location in the field and consequently of the team current disposition in the field. • The work has been carried out in a bottom-up fashion since we believe that many conceptual issues can be raised from and are strongly constrained by the actual implementation problems Nevertheless the basic framework described in the paper concerning hardware software and functional architectures was designed in a top-down fashion in the beginning of the project and has been essentially kept unchanged so far

30. Formulation and Implementation of RelationalBehaviors for • Multi-Robot CooperativeSystems • Bob van der Vecht and Pedro Lima • Abstract. This paper introduces: • A general formulation of relational behaviorsfor cooperative real robots and an example of its implementationusing the pass between soccer robots of the Middle-Sized League ofRoboCup. • The formulation is based on the Joint Commitment Theoryand the pass implementation is supported by past work on soccer robotsnavigation. • Results of experiments with real robots under controlled situations • (i.e., not during a game) are presented to illustrate the described • concepts.

31. Formulation and Implementation of Relational Behaviours for Multi-Robot Cooperative Systems Bob van der Vecht1;2 and Pedro Lima2 1 Department of Articial Intelligence, University of Groningen Grote Kruisstraat 2/1, 9712 TS, Groningen, Netherlands 2 Institute for Systems and Robotics, Instituto Superior Tecnico, Av. Rovisco Pais, 1 - 1049-001 Lisboa, Portugal bvecht@isr.ist.utl.pt, pal@isr.ist.utl.pt Abstract. This paper introduces a general formulation of relational behaviours for cooperative real robots and an example of its implementation using the pass between soccer robots of the Middle-Sized League of RoboCup. The formulation is based on the Joint Commitment Theory and the pass implementation is supported by past work on soccer robots navigation. Results of experiments with real robots under controlled situations (i.e., not during a game) are presented to illustrate the described concepts. 1 Introduction Showing cooperation among robots from a team is probably one the top goals of RoboCup related research. Furthermore, it is desirable to formulate cooperative behaviours within a formal framework extendable to applications other than robotic soccer. Surprisingly, not many references to work on these two topics can be found in the RoboCup-related literature or in other publications referring cooperation among real robots. An example of a tool that enables the development of applications based on the Joint Commitment Theory [1], including communications, has been thoroughly reported by Tambe, e.g., in [9]. Yokota et al. use explicit communication to achieve cooperation and synchronization in real robots. However, no explicit logical commitments are described [10]. Emergent cooperative behaviour among virtual agents is also described in [7], where a pass behaviour by implicit communication (observing the other robots behaviour) is implemented in a team of RoboCup Simulation League, and [6], where also a pass between RoboCup Simulation League agents is described such that conditions to pass are learned by a neural network. Again, there is no commitment between players, therefore the relational behaviour may be kept by one of the team mates even if the other has to withdraw its relational behaviour. In this paper a general formulation of relational behaviours for cooperative real robots is introduced. Most of the paper describes an example of implementation of this formulation using the pass between soccer robots of the Middle-Sized League (MSL) of RoboCup. The formulation is based on the Joint Commitment Theory [1], and the pass implementation is supported by past work of the ISocRob team on soccer robots navigation [3]. In the pass relational behaviour, two participants set up a long term commitment, in which several individual behaviours are executed. One of the robots is referred to as the kicker ; he starts having the ball and will try to kick the ball in the direction of the other robot, the receiver, who has to intercept the ball. In order to accomplish a pass successfully, two components of the commitment should be working well: the synchronization of both players' actions and the execution of their individual skills. The synchronization has to be achieved by communication. In Fig. 1, an example of the pass commitment has been illustrated by two state machines. In the following sections, the theory behind the pass commitment as shown in Fig. 1 will be described. First the individual decision making and the behaviour synchronization will be explained and then the individual primitive behaviours. Results of experiments with real robots under controlled situations (i.e., not during a game) are also presented to illustrate the key primitive behaviours described. FAILURE FAILURE FAILURE SUCCESS/ FAILURE SUCCESS SUCCESS SUCCESS/ Kicker Receiver Synchronization finish standBy aimAndPass standBy intercept finish Fig. 1. Simple example of the pass commitment 2 Decision Making and Synchronization In Fig. 1, several individual behaviours can be found within the commitment. At any time the participants have to select the correct primitive behaviour individually. The architecture that is used for behaviour coordination and decision making considers three types of behaviours: organizational, relational and individual [4], [5]. Organizational behaviours concern decisions involving the whole team, e.g., player role selection such as defender or attacker. The relational behaviours concern more than one player. Commitments among team mates are established here. Finally, at the individual level, primitive behaviours are selected, and motor commands are used to inuence the environment in the desired way. Behaviour selection is done in a module called the logic machine, explained in [8]. In this module, only the organizational and the individual levels were implemented. The work described here addresses the relational layer. The layers are processed sequentially. This means for the robot that he rst chooses a role, next he selects a commitment, and the individual behaviour is selected after knowing the robot's role and commitment. 2.1 Relational Behaviour Selection Joint Commitment Theory is used in this work to select relational behaviours [1]. Predened logical conditions can establish a commitment between two agents. Once a robot is committed to a relational behaviour, he will pursue this task until one or more conditions become false, or until the goal has been accomplished. In the described project, the initiative for a relational behaviour is taken by one of the agents, who sets a request for a relational behaviour. A potential partner checks if the conditions to accept are valid. If so, the commitment is established. During the execution of the commitment the changing environment can lead to failure or success at any time. In that case the commitment will be ended. In general, within a commitment three phases can be distinguished: Setup, Loop and End. During the setup and ending of a commitment, a robot is not executing a relational behaviour. The logic machine will not select any relational behaviour, so the commitment will be ignored during the primitive behaviour selection. Only in the Loop phase participants will select primitive behaviours concerning the commitment in order to achieve their joint goal. 2.2 Primitive Behaviour Selection The selection of a primitive behaviour within the Loop phase of a commitment will be explained using the example of the pass commitment of Fig. 1. Three individual behaviours can be found there; standBy for both participants, aimAnd- Pass for the kicker and intercept for the receiver. The pass commitment has been split up in several states from the beginning until the end, referred to as commitment states. { request and accept in the Setup phase. { prepare and intercept in the Loop phase. { done and failed in the End phase. In general, the states in the Setup and End phase will be the same for all commitments, only the Loop changes. Splitting the Loop phase in states allows synchronized execution of the pass. Here, each commitment state is linked to (a set of) primitive behaviours for both robots, see Table 1. When the commitment runs as planned, the pass states will be run through sequentially, from request until done. An error at any time can lead to the state failed. Note that pass states in the Setup and End phase do not lead to a primitive behaviour from the relational pass. Table 1 describes the pass commitment of Fig. 1. New commitments or other versions of commitments can be created under the same framework. One can change the individual behaviours that have been linked to the commitment states, or extend the Loop with more pass states and behaviours. Table 1. Primitive behaviour selection by the logic machine in all pass states during the pass commitment. Setup Loop End Default Commitment States: request accept prepare intercept done failed none Kicker | | aimAndPass standBy | | | Receiver | | standBy Intercept | | | 2.3 Synchronization To synchronize the behaviours, the participants will use explicit communication. Four variables, containing the identities of the participants and their commitment states, are kept in the agent's memory. Each of these four variables will be sent to the other participant in the relational behaviour when it is changed. Following setup conditions, one agent sets a pass request, and a partner can enter the accept state. When the commitment has been established, the following rules will be looped: { Synchronize commitment state with the state of the parter, if partner has moved on to a next state. { Switch to new state by yourself, if predened switch conditions allow that. { Select a basic behaviour using Table 1. This happens until the done or failed state is reached. Then the commitment will be nished and all variables will be cleared. Synchronization is achieved at each moment an agent switches to a new commitment state, since the partner will always follow. By synchronizing commitment states each iteration, the commitment states of both agents can only be one step away from each other, and this dierence will be corrected in the next loop. It is possible to let agents execute a sequence of primitive behaviours in one state. They will run asynchronously. 3 The Individual Behaviours The primitive behaviours are running in a control module. Here the world situation is evaluated continuously, and motor commands will be send to the robot in order to inuence the environment. The world model of the robot can be seen as a map with all identied objects in a xy-coordinate system. In the standBy behaviour, the robot will stay at the same position and will try to keep its front towards the ball position. This behaviour has been implemented earlier in the ISocRob team [5]. The other two behaviours, aimAndPass and intercept, had to be developed from scratch. 3.1 The aimAndPass Behaviour When the pass commitment is started, the kicker has the ball. He wants to rotate with the ball to a certain direction and then shoot. This has been implemented in the aimAndPass behaviour. In earlier research a controller has been developed to dribble with a ball to a goal posture [3]. A navigation algorithm described in [3] is used which takes the robot to a goal posture while avoiding obstacles, using a modied potential elds method that takes into account the non-holonomic nature of the robot. The motor commands given by this navigation algorithm are passed to a dribble lter which adapts them to the extra constraints of keeping the ball close. Parameters of the navigation algorithm can be changed to return a very strong angular acceleration towards the desired direction. Since the dribble lter stays with the dribble constraints, a controller is achieved which rotates with ball to the desired direction at the maximum turn angle. 3.2 The intercept Behaviour In order to intercept a moving ball, the intercept behaviour controller has been designed. The literature on visual servoing has solutions for similar problems [2], but only for a situation when the robot tracking the object is not moving within an environment cluttered with obstacles. Therefore, the solution described here is based on the previous mentioned navigation algorithm, [3], using a modied potential elds method. The intercept controller continuously estimates the interception point given the positions of ball and robot, and their predicted path. After a certain time t, position of ball and robot should be the same. The corresponding xy-coordinates indicate the interception point and they will be passed to the navigating algorithm. Interception point estimation is shown in Fig. 2. Ball (X B , Y B ) VB Interception a point (X, Y) VB VBY X V’R q Robot orientation Robot(X R , Y R ) X Y Fig. 2. Interception point calculation. Figure 2 shows a ball path estimation with position (xb; yb) and velocity (vbx; vby), and a robot with position (xr; yr). For the estimation, a constant velocity is assumed. The robot path is estimated with the assumptions that the robot will move with a constant velocity in a straight line towards the interception point. This implies that the orientation of the robot will not change during the movement to the interception point and thus the initial orientation is ignored. An assumed average forward speed v0 r is used. Now, the robot path can be given as function of time, using the robot's current position (xr; yr), its absolute speed v0 r (given in the x-direction) and the orientation towards the interception point , as shown in Figure 2. Since the interception point is still to be calculated, is still unknown. Variables and t can be eliminated from respectively the x and y-component from both path estimations. The calculated t represents the time it takes for the ball and the robot to get to the interception point. By replacing t in the ball path estimation, xy-coordinates for the interception point are achieved. Note that the assumption that the robot moves in a straight line to the interception point, may lead to an error in the estimation of the interception point, since the robots are non-holonomic. However, the estimation is iteratively applied and becomes more accurate at each new iteration step, until when the robot faces the correct heading, where the straight line motion assumption is fully correct. 4 Implementation and Results 4.1 Implementation of the Commitment The theory for joint commitments has been implemented in Nomadic Super Scout II robots of the Robocup Middle-Sized League team ISocRob. The decision making and the synchronization have been implemented successfully. The framework for joint commitments requires the denition of the following aspects of a relational behaviour: { List of commitment states { Setup conditions of the commitment { Switch conditions to switch between commitment states { Related individual behaviours within each commitment state The framework takes care of all communication and the synchronized execution of the relational behaviour. For the implemented version of the pass commitment, mentioned in section 1, the following pass situation has been dened: a defender has the ball on its own half of the eld and he will pass the ball to an attacker, who is on the other half of the eld. The division in states and the related primitive behaviours are shown in Table 1. When the kicker has successfuly executed the aimAndPass behaviour, he switches to the intercept state. The receiver moves to the same state, and starts the intercept behaviour. Note that in this case the kicker switches the pass state to intercept, and by doing so he tells the receiver to start the intercept behaviour. More reactive approaches of the pass commitment can dene conditions that allow the receiver to switch to the intercept state, following his own observations. The results of the execution of the pass behaviour are strongly related to the results of the individual behaviours. AimAndPass and intercept will be analysed in the following sections. 4.2 AimAndPass The aimAndPass behaviour has been implemented following the methods described in previous sections. The performance of the behaviour is as expected. Figure 3(a) shows the result of a test with a real robot on a MSL soccer eld. The robot start position is at the middle of the eld, and his target is the middle of the goal which is located diagonally behind him. The robot turns with the ball and shoots in the desired direction. However, since the algorithm works with coordinates, errors can occur when the robot is badly localized. A small dierence in the kicker orientation leads to a signicant spatial error when the distance to the target grows. Those errors caused by localization can be avoided if the camera is used directly to determine the target direction. (a) (b) Fig. 3. Individual behaviour results: a) aimAndPass; b) intercept ball 4.3 Intercept For the intercept behaviour, the assumed average robot speed, v0 r, has been set to 0.5 m/s. An example of the interception of a moving ball by a real robot in a MSL soccer eld is shown in Fig. 3(b). Clearly it can be seen that the robot takes the ball velocity into account, and moves directly to the interception point. In this example, the ball rolls with a speed of 1.2 m/s in a straight line. The ball path that is shown is the path as it is observed by the robot. Balls with a high velocity are likely to bump away after the interception. 5 Conclusions In this paper the general formulation of relational behaviours among real robots, based on the Joint Commitment Theory, has been introduced through an illustrative example concerning a pass behaviour between RoboCup MSL robots. The formulation uses individual decision making and behaviour synchronization among intervening robots and has been tested successfully during laboratory games without an opponent. Results of the implementation of the key individual behaviours (aimAndPass, intercept) in real robots were presented. Future work will concern the development of new relational behaviours under this framework, as well as the rening of the individual behaviours, particularly by using team mates visual recognition to eliminate the self-localization error and the required communication during the aimAndPass behaviour, and by providing the robots with force-controlled kicking ability, so as to enable ball interception by the receiver robot, by reducing the ball speed for pass kicks, as compared to goal kicks. References 1. P. R. Cohen, H. J. Levesque, "Teamwork". Nous, Vol 35, (1991) 2. P. Corke, Visual Control of Robots: High-performance Visual Servoing, Research Studies Press LTD, (1996) 3. B. Damas, L. Custodio, P. Lima, "A Modied Potential Fields Method for Robot Navigation Applied to Dribbling in Robotic Soccer", Proceedings of RoboCup 2002 Symposium, (2003) 4. A. Drogoul, A. Collinot, "Applying an Agent-Oriented Methodology to the Design of Articial Organizations: A Case Study in Robotic Soccer", Autonomous Agents and Multi-Agent Systems, Vol 1, (1998) 5. P. Lima, L. M. Custodio, et al., "ISocRob 2003: Team Description Paper", Proceedings of the RoboCup 2003 Symposium, (2003) 6. H. Matsubara, I. Noda and K. Hiraki, "Learning of Cooperative Actions in Multi- Agent Systems: a Case Study of Pass in Soccer.", Adaptation, Coevolution and Learning in Multi-agent Systems: Papers from the AAAI-96 Spring Symposium, (1996). 7. E. Pagello A. D'Angelo, F. Montsello, F. Garelli, C. Ferrari, "Cooperative Behaviors in Multi-Robot Systems Through Implicit Communication", Robotics and Autonomous Systems, Vol 29, (1999) 8. V. Pires, M. Arroz, L. Custodio, "Logic Based Hybrid Decision System for a Multirobot Team", Proceedings of 8th Conference on Intelligent Autonomous Systems, (2004) 9. M. Tambe, "Towards Flexible Teamwork", Journal of Articial Intelligence Research, Vol 7, (1997) 10. K.Yokota, K. Ozaki, N. Watanabe, A. Matsumoto, D. Koyama, T. Ishikawa, K. Kawabata, H. Kaetsu, and H. Asama, "Cooperative Team Play Based on Communication.", Proceedings of RoboCup 1998 Symposium, (1999) • 1 – Introduction • Showing cooperation among robots from a team is probably one the top goals of RoboCup related research. • Furthermore, it is desirable to formulate cooperative behaviors within a formal framework extendable to applications other than robotic soccer. Surprisingly, not many references to work on these two topics can be found in the RoboCup-related literature or in other publications referring cooperation among real robots. • An example of a tool that enables the development of applications based on the Joint Commitment Theory [1], including communications, has been thoroughly reported by Tambe, e.g., in [9]. Yokota et al. use explicit communication to achieve cooperation and synchronization in real robots. However, no explicit logical commitments are described [10]. • Emergent cooperative behavior among virtual agents is also described in [7], where a pass behavior by implicit communication (observing the other robots behavior) is implemented in a teamof RoboCup Simulation League, and [6], where also a pass between RoboCupSimulation League agents is described such that conditions to pass are learnedby a neural network. Again, there is no commitment between players, thereforethe relational behavior may be kept by one of the team mates even if the other has to withdraw its relational behavior.

32. Formulation and Implementation of Relational Behaviours for Multi-Robot Cooperative Systems Bob van der Vecht1;2 and Pedro Lima2 1 Department of Articial Intelligence, University of Groningen Grote Kruisstraat 2/1, 9712 TS, Groningen, Netherlands 2 Institute for Systems and Robotics, Instituto Superior Tecnico, Av. Rovisco Pais, 1 - 1049-001 Lisboa, Portugal bvecht@isr.ist.utl.pt, pal@isr.ist.utl.pt Abstract. This paper introduces a general formulation of relational behaviours for cooperative real robots and an example of its implementation using the pass between soccer robots of the Middle-Sized League of RoboCup. The formulation is based on the Joint Commitment Theory and the pass implementation is supported by past work on soccer robots navigation. Results of experiments with real robots under controlled situations (i.e., not during a game) are presented to illustrate the described concepts. 1 Introduction Showing cooperation among robots from a team is probably one the top goals of RoboCup related research. Furthermore, it is desirable to formulate cooperative behaviours within a formal framework extendable to applications other than robotic soccer. Surprisingly, not many references to work on these two topics can be found in the RoboCup-related literature or in other publications referring cooperation among real robots. An example of a tool that enables the development of applications based on the Joint Commitment Theory [1], including communications, has been thoroughly reported by Tambe, e.g., in [9]. Yokota et al. use explicit communication to achieve cooperation and synchronization in real robots. However, no explicit logical commitments are described [10]. Emergent cooperative behaviour among virtual agents is also described in [7], where a pass behaviour by implicit communication (observing the other robots behaviour) is implemented in a team of RoboCup Simulation League, and [6], where also a pass between RoboCup Simulation League agents is described such that conditions to pass are learned by a neural network. Again, there is no commitment between players, therefore the relational behaviour may be kept by one of the team mates even if the other has to withdraw its relational behaviour. In this paper a general formulation of relational behaviours for cooperative real robots is introduced. Most of the paper describes an example of implementation of this formulation using the pass between soccer robots of the Middle-Sized League (MSL) of RoboCup. The formulation is based on the Joint Commitment Theory [1], and the pass implementation is supported by past work of the ISocRob team on soccer robots navigation [3]. In the pass relational behaviour, two participants set up a long term commitment, in which several individual behaviours are executed. One of the robots is referred to as the kicker ; he starts having the ball and will try to kick the ball in the direction of the other robot, the receiver, who has to intercept the ball. In order to accomplish a pass successfully, two components of the commitment should be working well: the synchronization of both players' actions and the execution of their individual skills. The synchronization has to be achieved by communication. In Fig. 1, an example of the pass commitment has been illustrated by two state machines. In the following sections, the theory behind the pass commitment as shown in Fig. 1 will be described. First the individual decision making and the behaviour synchronization will be explained and then the individual primitive behaviours. Results of experiments with real robots under controlled situations (i.e., not during a game) are also presented to illustrate the key primitive behaviours described. FAILURE FAILURE FAILURE SUCCESS/ FAILURE SUCCESS SUCCESS SUCCESS/ Kicker Receiver Synchronization finish standBy aimAndPass standBy intercept finish Fig. 1. Simple example of the pass commitment 2 Decision Making and Synchronization In Fig. 1, several individual behaviours can be found within the commitment. At any time the participants have to select the correct primitive behaviour individually. The architecture that is used for behaviour coordination and decision making considers three types of behaviours: organizational, relational and individual [4], [5]. Organizational behaviours concern decisions involving the whole team, e.g., player role selection such as defender or attacker. The relational behaviours concern more than one player. Commitments among team mates are established here. Finally, at the individual level, primitive behaviours are selected, and motor commands are used to inuence the environment in the desired way. Behaviour selection is done in a module called the logic machine, explained in [8]. In this module, only the organizational and the individual levels were implemented. The work described here addresses the relational layer. The layers are processed sequentially. This means for the robot that he rst chooses a role, next he selects a commitment, and the individual behaviour is selected after knowing the robot's role and commitment. 2.1 Relational Behaviour Selection Joint Commitment Theory is used in this work to select relational behaviours [1]. Predened logical conditions can establish a commitment between two agents. Once a robot is committed to a relational behaviour, he will pursue this task until one or more conditions become false, or until the goal has been accomplished. In the described project, the initiative for a relational behaviour is taken by one of the agents, who sets a request for a relational behaviour. A potential partner checks if the conditions to accept are valid. If so, the commitment is established. During the execution of the commitment the changing environment can lead to failure or success at any time. In that case the commitment will be ended. In general, within a commitment three phases can be distinguished: Setup, Loop and End. During the setup and ending of a commitment, a robot is not executing a relational behaviour. The logic machine will not select any relational behaviour, so the commitment will be ignored during the primitive behaviour selection. Only in the Loop phase participants will select primitive behaviours concerning the commitment in order to achieve their joint goal. 2.2 Primitive Behaviour Selection The selection of a primitive behaviour within the Loop phase of a commitment will be explained using the example of the pass commitment of Fig. 1. Three individual behaviours can be found there; standBy for both participants, aimAnd- Pass for the kicker and intercept for the receiver. The pass commitment has been split up in several states from the beginning until the end, referred to as commitment states. { request and accept in the Setup phase. { prepare and intercept in the Loop phase. { done and failed in the End phase. In general, the states in the Setup and End phase will be the same for all commitments, only the Loop changes. Splitting the Loop phase in states allows synchronized execution of the pass. Here, each commitment state is linked to (a set of) primitive behaviours for both robots, see Table 1. When the commitment runs as planned, the pass states will be run through sequentially, from request until done. An error at any time can lead to the state failed. Note that pass states in the Setup and End phase do not lead to a primitive behaviour from the relational pass. Table 1 describes the pass commitment of Fig. 1. New commitments or other versions of commitments can be created under the same framework. One can change the individual behaviours that have been linked to the commitment states, or extend the Loop with more pass states and behaviours. Table 1. Primitive behaviour selection by the logic machine in all pass states during the pass commitment. Setup Loop End Default Commitment States: request accept prepare intercept done failed none Kicker | | aimAndPass standBy | | | Receiver | | standBy Intercept | | | 2.3 Synchronization To synchronize the behaviours, the participants will use explicit communication. Four variables, containing the identities of the participants and their commitment states, are kept in the agent's memory. Each of these four variables will be sent to the other participant in the relational behaviour when it is changed. Following setup conditions, one agent sets a pass request, and a partner can enter the accept state. When the commitment has been established, the following rules will be looped: { Synchronize commitment state with the state of the parter, if partner has moved on to a next state. { Switch to new state by yourself, if predened switch conditions allow that. { Select a basic behaviour using Table 1. This happens until the done or failed state is reached. Then the commitment will be nished and all variables will be cleared. Synchronization is achieved at each moment an agent switches to a new commitment state, since the partner will always follow. By synchronizing commitment states each iteration, the commitment states of both agents can only be one step away from each other, and this dierence will be corrected in the next loop. It is possible to let agents execute a sequence of primitive behaviours in one state. They will run asynchronously. 3 The Individual Behaviours The primitive behaviours are running in a control module. Here the world situation is evaluated continuously, and motor commands will be send to the robot in order to inuence the environment. The world model of the robot can be seen as a map with all identied objects in a xy-coordinate system. In the standBy behaviour, the robot will stay at the same position and will try to keep its front towards the ball position. This behaviour has been implemented earlier in the ISocRob team [5]. The other two behaviours, aimAndPass and intercept, had to be developed from scratch. 3.1 The aimAndPass Behaviour When the pass commitment is started, the kicker has the ball. He wants to rotate with the ball to a certain direction and then shoot. This has been implemented in the aimAndPass behaviour. In earlier research a controller has been developed to dribble with a ball to a goal posture [3]. A navigation algorithm described in [3] is used which takes the robot to a goal posture while avoiding obstacles, using a modied potential elds method that takes into account the non-holonomic nature of the robot. The motor commands given by this navigation algorithm are passed to a dribble lter which adapts them to the extra constraints of keeping the ball close. Parameters of the navigation algorithm can be changed to return a very strong angular acceleration towards the desired direction. Since the dribble lter stays with the dribble constraints, a controller is achieved which rotates with ball to the desired direction at the maximum turn angle. 3.2 The intercept Behaviour In order to intercept a moving ball, the intercept behaviour controller has been designed. The literature on visual servoing has solutions for similar problems [2], but only for a situation when the robot tracking the object is not moving within an environment cluttered with obstacles. Therefore, the solution described here is based on the previous mentioned navigation algorithm, [3], using a modied potential elds method. The intercept controller continuously estimates the interception point given the positions of ball and robot, and their predicted path. After a certain time t, position of ball and robot should be the same. The corresponding xy-coordinates indicate the interception point and they will be passed to the navigating algorithm. Interception point estimation is shown in Fig. 2. Ball (X B , Y B ) VB Interception a point (X, Y) VB VBY X V’R q Robot orientation Robot(X R , Y R ) X Y Fig. 2. Interception point calculation. Figure 2 shows a ball path estimation with position (xb; yb) and velocity (vbx; vby), and a robot with position (xr; yr). For the estimation, a constant velocity is assumed. The robot path is estimated with the assumptions that the robot will move with a constant velocity in a straight line towards the interception point. This implies that the orientation of the robot will not change during the movement to the interception point and thus the initial orientation is ignored. An assumed average forward speed v0 r is used. Now, the robot path can be given as function of time, using the robot's current position (xr; yr), its absolute speed v0 r (given in the x-direction) and the orientation towards the interception point , as shown in Figure 2. Since the interception point is still to be calculated, is still unknown. Variables and t can be eliminated from respectively the x and y-component from both path estimations. The calculated t represents the time it takes for the ball and the robot to get to the interception point. By replacing t in the ball path estimation, xy-coordinates for the interception point are achieved. Note that the assumption that the robot moves in a straight line to the interception point, may lead to an error in the estimation of the interception point, since the robots are non-holonomic. However, the estimation is iteratively applied and becomes more accurate at each new iteration step, until when the robot faces the correct heading, where the straight line motion assumption is fully correct. 4 Implementation and Results 4.1 Implementation of the Commitment The theory for joint commitments has been implemented in Nomadic Super Scout II robots of the Robocup Middle-Sized League team ISocRob. The decision making and the synchronization have been implemented successfully. The framework for joint commitments requires the denition of the following aspects of a relational behaviour: { List of commitment states { Setup conditions of the commitment { Switch conditions to switch between commitment states { Related individual behaviours within each commitment state The framework takes care of all communication and the synchronized execution of the relational behaviour. For the implemented version of the pass commitment, mentioned in section 1, the following pass situation has been dened: a defender has the ball on its own half of the eld and he will pass the ball to an attacker, who is on the other half of the eld. The division in states and the related primitive behaviours are shown in Table 1. When the kicker has successfuly executed the aimAndPass behaviour, he switches to the intercept state. The receiver moves to the same state, and starts the intercept behaviour. Note that in this case the kicker switches the pass state to intercept, and by doing so he tells the receiver to start the intercept behaviour. More reactive approaches of the pass commitment can dene conditions that allow the receiver to switch to the intercept state, following his own observations. The results of the execution of the pass behaviour are strongly related to the results of the individual behaviours. AimAndPass and intercept will be analysed in the following sections. 4.2 AimAndPass The aimAndPass behaviour has been implemented following the methods described in previous sections. The performance of the behaviour is as expected. Figure 3(a) shows the result of a test with a real robot on a MSL soccer eld. The robot start position is at the middle of the eld, and his target is the middle of the goal which is located diagonally behind him. The robot turns with the ball and shoots in the desired direction. However, since the algorithm works with coordinates, errors can occur when the robot is badly localized. A small dierence in the kicker orientation leads to a signicant spatial error when the distance to the target grows. Those errors caused by localization can be avoided if the camera is used directly to determine the target direction. (a) (b) Fig. 3. Individual behaviour results: a) aimAndPass; b) intercept ball 4.3 Intercept For the intercept behaviour, the assumed average robot speed, v0 r, has been set to 0.5 m/s. An example of the interception of a moving ball by a real robot in a MSL soccer eld is shown in Fig. 3(b). Clearly it can be seen that the robot takes the ball velocity into account, and moves directly to the interception point. In this example, the ball rolls with a speed of 1.2 m/s in a straight line. The ball path that is shown is the path as it is observed by the robot. Balls with a high velocity are likely to bump away after the interception. 5 Conclusions In this paper the general formulation of relational behaviours among real robots, based on the Joint Commitment Theory, has been introduced through an illustrative example concerning a pass behaviour between RoboCup MSL robots. The formulation uses individual decision making and behaviour synchronization among intervening robots and has been tested successfully during laboratory games without an opponent. Results of the implementation of the key individual behaviours (aimAndPass, intercept) in real robots were presented. Future work will concern the development of new relational behaviours under this framework, as well as the rening of the individual behaviours, particularly by using team mates visual recognition to eliminate the self-localization error and the required communication during the aimAndPass behaviour, and by providing the robots with force-controlled kicking ability, so as to enable ball interception by the receiver robot, by reducing the ball speed for pass kicks, as compared to goal kicks. References 1. P. R. Cohen, H. J. Levesque, "Teamwork". Nous, Vol 35, (1991) 2. P. Corke, Visual Control of Robots: High-performance Visual Servoing, Research Studies Press LTD, (1996) 3. B. Damas, L. Custodio, P. Lima, "A Modied Potential Fields Method for Robot Navigation Applied to Dribbling in Robotic Soccer", Proceedings of RoboCup 2002 Symposium, (2003) 4. A. Drogoul, A. Collinot, "Applying an Agent-Oriented Methodology to the Design of Articial Organizations: A Case Study in Robotic Soccer", Autonomous Agents and Multi-Agent Systems, Vol 1, (1998) 5. P. Lima, L. M. Custodio, et al., "ISocRob 2003: Team Description Paper", Proceedings of the RoboCup 2003 Symposium, (2003) 6. H. Matsubara, I. Noda and K. Hiraki, "Learning of Cooperative Actions in Multi- Agent Systems: a Case Study of Pass in Soccer.", Adaptation, Coevolution and Learning in Multi-agent Systems: Papers from the AAAI-96 Spring Symposium, (1996). 7. E. Pagello A. D'Angelo, F. Montsello, F. Garelli, C. Ferrari, "Cooperative Behaviors in Multi-Robot Systems Through Implicit Communication", Robotics and Autonomous Systems, Vol 29, (1999) 8. V. Pires, M. Arroz, L. Custodio, "Logic Based Hybrid Decision System for a Multirobot Team", Proceedings of 8th Conference on Intelligent Autonomous Systems, (2004) 9. M. Tambe, "Towards Flexible Teamwork", Journal of Articial Intelligence Research, Vol 7, (1997) 10. K.Yokota, K. Ozaki, N. Watanabe, A. Matsumoto, D. Koyama, T. Ishikawa, K. Kawabata, H. Kaetsu, and H. Asama, "Cooperative Team Play Based on Communication.", Proceedings of RoboCup 1998 Symposium, (1999) • In this paper: • A general formulation of relational behaviors for cooperativereal robots is introduced. Most of the paper describes an example of implementationof this formulation using the pass between soccer robots of the Middle-SizedLeague (MSL) of RoboCup. • The formulation is based on the Joint CommitmentTheory [1], and the pass implementation is supported by past work of theISocRob team on soccer robots navigation [3]. • In the pass relational behavior, • two participants set up a long term commitment,in which several individual behaviors are executed. • One of the robots isreferred to as the kicker; he starts having the ball and will try to kick the ball inthe direction of the other robot, the receiver, who has to intercept the ball. • In orderto accomplish a pass successfully, • Two components of the commitment shouldbe working well: the synchronization of both players' actions and the executionof their individual skills. • The synchronization has to be achieved by communication.In Fig. 1, an example of the pass commitment has been illustrated bytwo state machines. • In the following: the theory behind the pass commitment will be described. First the individual decision making and the behaviorsynchronization will be explained and then the individual primitive behaviors.Results of experiments with real robots under controlled situations (i.e., notduring a game) are also presented to illustrate the key primitive behaviors described.

33. Fig. 1. Simple example of the pass commitment

34. 2 - Decision Making and Synchronization • In Fig. 1, several individual behaviors can be found within the commitment. • At any time the participants have to select the correct primitive behavior individually. • The architecture that is used for behavior coordination and decisionmaking considers three types of behaviors: • Organizational behaviors concern decisions involving the wholeteam (strategy), e.g., player role selection such as defender or attacker. • The relational behaviorsconcern more than one player. Commitments among team mates are establishedhere. • The individual level, primitive behaviors are selected,and motor commands are used to influence the environment in the desired way. • Behavior selection is done in a module called the logic machine [8] where only the organizational and the individual levels wereimplemented. The work described here addresses the relational layer. • The layersare processed sequentially: • The system chooses a role, • Next it selects a commitment, and • The individual behavior is selected afterknowing the robot's role and commitment.

35. 2.1 Relational Behavior Selection • Joint Commitment Theory is used in this work to select relational behaviors [1]. Predefined logical conditions can establish a commitment between two agents: • Once a robot is committed to a relational behavior, • it will pursue this task until one or more conditions become false, • or until the goal has been accomplished. • In the described project: • the initiative for a relational behavior is taken by one of the agents, who sets a request for a relational behavior. • A potential partner checks if the conditions to accept are valid. If so, the commitment is established. • During the execution of the commitment the changing environment can lead to failure or success at any time, in which case the commitment will be ended. • In general, a commitment consists of three phases which can be distinguished by: Setup, Loop and End: • During the setup and ending of a commitment, a robot is not executing a relational behavior. • The logic machine will not select any relational behavior, so the commitment will be ignored during the primitive behavior selection. • Only in the Loop phase participants will select primitive behaviors concerning the commitment in order to achieve their joint goal.

36. 2.2 Primitive Behavior Selection • The selection of a primitive behavior within the Loop phase of a commitment will be explained using the example of the pass commitment of Fig. 1. • Three individual behaviors can be found there; • standBy for both participants, • aimAnd-Pass for the kicker and • intercept for the receiver. • The pass commitment has been split up in several states from the beginning until the end, referred to as commitment states: • request and accept in the Setup phase. • prepare and intercept in the Loop phase. • done and failed in the End phase.

37. 2.2 Primitive Behavior Selection • In general: • the states in the Setup and End phase will be the same for all commitments, only the Loop changes. • Splitting the Loop phase in states allows synchronized execution of the pass. Here, each commitment state is linked to (a set of) primitive behaviors for both robots, see Table 1. When the commitment runs as planned, the pass states will be run through sequentially, from request until done. An error at any time can lead to the state failed. • Note that pass states in the Setup and End phase do not lead to a primitive behavior from the relational pass. Table 1 describes the pass commitment of Fig. 1. New commitments or other versions of commitments can be created under the same framework. One can change the individual behaviors that have been linked to the commitment states, or extend the Loop with more pass states and behaviors.

38. 2.3 Synchronization • To synchronize the behaviors, the participants will use explicit communication: • Four variables, containing the identities of the participants and their commitmentstates, are kept in the agent's memory. • Each of these four variables will besent to the other participant in the relational behavior when it is changed. • Following setup conditions, one agent sets a pass request, and a partner canenter the accept state. When the commitment has been established, the followingrules will be looped: • Synchronize commitment state with the state of the partner, if partner hasmoved on to a next state. • Switch to new state by yourself, if predefined switch conditions allow that. • Select a basic behavior using Table 1. • This happens until the done or failed state is reached. Then the commitmentwill be finished and all variables will be cleared. • Synchronization is achieved at each moment an agent switches to a newcommitment state, since the partner will always follow. By synchronizing commitmentstates each iteration, the commitment states of both agents can onlybe one step away from each other, and this difference will be corrected in the • next loop. It is possible to let agents execute a sequence of primitive behaviors in one state. They will run asynchronously.

39. 3 The Individual Behaviours The primitive behaviors are running in a control module. Here the world situationis evaluated continuously, and motor commands will be send to the robotin order to influence the environment. The world model of the robot can be seenas a map with all identified objects in a xy-coordinate system. In the standBybehavior, the robot will stay at the same position and will try to keep its fronttowards the ball position. This behavior has been implemented earlier in theISocRob team [5]. The other two behaviors, aimAndPass and intercept, hadto be developed from scratch. 3.1 The aimAndPass Behavior When the pass commitment is started, the kicker has the ball. He wants to rotatewith the ball to a certain direction and then shoot. This has been implementedin the aimAndPass behavior. In earlier research a controller has been developed to dribble with a ball toa goal posture [3]. A navigation algorithm described in [3] is used which takesthe robot to a goal posture while avoiding obstacles, using a modified potential fields method that takes into account the non-holonomic nature of the robot. Themotor commands given by this navigation algorithm are passed to a dribble filterwhich adapts them to the extra constraints of keeping the ball close. Parametersof the navigation algorithm can be changed to return a very strong angularacceleration towards the desired direction. Since the dribble filter stays with thedribble constraints, a controller is achieved which rotates with ball to the desired direction at the maximum turn angle.

40. In order to intercept a moving ball, the intercept behavior controller has been designed. The literature on visual servoing has solutions for similar problems [2], but only for a situation when the robot tracking the object is not moving within an environment cluttered with obstacles. • Therefore, the solution described here is based on the previous mentioned navigation algorithm, [3], using a modified potential fields method. • The intercept controller continuously estimates the interception point given the positions of ball and robot, and their predicted path. • After a certain time t, position of ball and robot should be the same. • The corresponding xy-coordinates indicate the interception point and they will be passed to the navigating algorithm. Interception point estimation is shown in Fig. 2. • Figure 2 shows a ball path estimation with position (xb; yb) and velocity (vbx; vby), and a robot with position (xr; yr). For the estimation, a constant velocity is assumed.

41. The robot path is estimated with the assumptions that the robot will movewith a constant velocity in a straight line towards the interception point. Thisimplies that the orientation of the robot will not change during the movementto the interception point and thus the initial orientation is ignored. An assumedaverage forward speed vr is used. • Now, the robot path can be given as functionof time, using the robot's current position (xr; yr), its absolute speed vr (given inthe x-direction) and the orientation towards the interception point , as shown inFigure 2. • Since the interception point is still to be calculated, alpha is still unknown.Variables alpha and t can be eliminated from respectively the x and y-componentfrom both path estimations. The calculated t represents the time it takes for theball and the robot to get to the interception point. By replacing t in the ballpath estimation, xy-coordinates for the interception point are achieved. • Note that the assumption that the robot moves in a straight line to theinterception point, may lead to an error in the estimation of the interceptionpoint, since the robots are non-holonomic. • However, the estimation is iterativelyapplied and becomes more accurate at each new iteration step, until when therobot faces the correct heading, where the straight line motion assumption isfully correct.

42. 4 Implementation and Results • 4.1 Implementation of the Commitment • The theory for joint commitments has been implemented in Nomadic SuperScout II robots of the Robocup Middle-Sized League team ISocRob. The decisionmaking and the synchronization have been implemented successfully. Theframework for joint commitments requires the definition of the following aspectsof a relational behavior: • List of commitment states • Setup conditions of the commitment • Switch conditions to switch between commitment states • Related individual behaviours within each commitment state • The framework takes care of all communication and the synchronized executionof the relational behavior. For the implemented version of the pass commitment, mentioned in section1, the following pass situation has been defined: • A defender has the ball on itsown half of the field and he will pass the ball to an attacker, who is on the otherhalf of the field. • The division in states and the related primitive behaviors areshown in Table 1. When the kicker has successfully executed the aimAndPassbehavior, he switches to the intercept state. • The receiver moves to the samestate, and starts the intercept behavior.Note that in this case the kicker switches the pass state to intercept, andby doing so he tells the receiver to start the intercept behavior. • More reactiveapproaches of the pass commitment can define conditions that allow the receiverto switch to the intercept state, following his own observations. • The results of the execution of the pass behavior are strongly related to theresults of the individual behaviors.

44. 4.2 AimAndPass • The aimAndPass behavior has been implemented following the methods describedin previous sections. The performance of the behaviour is as expected. • Figure 3(a) shows the result of a test with a real robot on a MSL soccer field. • The robot start position is at the middle of the field, and his target is the middleof the goal which is located diagonally behind him. • The robot turns withthe ball and shoots in the desired direction. However, since the algorithm workswith coordinates, errors can occur when the robot is badly localized. • A smalldifference in the kicker orientation leads to a significant spatial error when thedistance to the target grows. Those errors caused by localization can be avoidedif the camera is used directly to determine the target direction. • Fig. 3. Individual behaviour results: a) aimAndPass; b) intercept ball

45. 4.2 AimAndPass • 4.3 Intercept • For the intercept behavior, the assumed average robot speed, v0’ has been set to 0.5 m/s. An example of the interception of a moving ball by a real robot ina MSL soccer field is shown in Fig. 3(b). • Clearly it can be seen that the robottakes the ball velocity into account, and moves directly to the interception point.In this example, the ball rolls with a speed of 1.2 m/s in a straight line. • The ballpath that is shown is the path as it is observed by the robot. Balls with a highvelocity are likely to bump away after the interception.

46. 5 Conclusions • In this paper the general formulation of relational behaviors among real robots,based on the Joint Commitment Theory, has been introduced through an illustrativeexample concerning a pass behavior between RoboCup MSL robots. • The formulation uses individual decision making and behavior synchronizationamong intervening robots and has been tested successfully during laboratorygames without an opponent. Results of the implementation of the key individualbehaviors (aimAndPass, intercept) in real robots were presented. • Future work will concern the development of new relational behaviors underthis framework, as well as the refining of the individual behaviors, particularlyby using team mates visual recognition to eliminate the self-localization errorand the required communication during the aimAndPass behavior, and byproviding the robots with force-controlled kicking ability, so as to enable ball • interception by the receiver robot, by reducing the ball speed for pass kicks, as compared to goal kicks.