Behavior Coordination Mechanisms – State-of-the-Art

1. Behavior Coordination Mechanisms – State-of-the-Art Paper by: Paolo Pirjanian (USC) Presented by: Chris Martin

2. Subject of Paper • Overview of different Action Selection Mechanisms (ASMs) that “solve” the Action Selection Problem (ASP) • A major issue in behavior-based control (BBC) systems is the formulation of effective mechanisms for coordination of the behaviors’ activities into strategies for rational and coherent behavior

3. Research Areas Tackling ASP • Ethology • Artificial Life (AL) • Virtual Reality (VR) • Software Agents • Robotics (Physical Agents)

4. What is the ASP? • Deals with the agent selecting “the most appropriate” or “the most relevant” next action at a particular moment in a particular situation • Choosing a “good enough” or “satisficing” behavior

5. “Good enough” behaviors • According to Mae’s, to produce a “good enough” behavior, need the following requirements • Goal orientedness • Situatedness • Persistence • Planning • Robustness • reactivity

6. “Good enough” behaviors (2) • Tyrell adds the following to Mae’s list: • Deal with all types of subproblems • Compromise actions • Opportunism (in contrast to persistence)

7. Actions and Agents • Based on the Webster dictionary, two relevant definitions of action for two parts of the system: • 1. “the condition of acting or moving, as opposed to rest” = motor movements • 2. “habitual deeds; hence conduct; behavior” = activation of a behavior • Agents have 2 roles in an agency: actions and action selection mechanisms (ASMs) • With respect to its subordinates, agent is ASM • With respect to its superior, agent is action

8. Types of ASMs • Two types: Arbitration and Command Fusion (CF) ASMs • One characteristic defines the division: the number of behaviors handled • Arbitrary ASMs allow one or one set of behaviors to take control at any one time • Command Fusion ASMs allow multiple behaviors to contribute to final control of the robot

9. Arbitration ASMs – 3 Types • 1. Priority-based • Action selection consists of higher level behaviors overriding the output of lower level behaviors • 2. State-based -- 4 different examples • A. Discrete Event Systems • Behavior selection done using state transitions • Detection of certain events shift the system to a new state and new behavior

10. Arbitration ASMs (2)State-based cont. • B. Temporal Sequencing • At each state a behavior is activated and perceptual triggers cause state transition • C. Bayesian Decision Analysis • Choose action that maximizes expected utility of agent (cost/benefit) • D. Reinforcement Learning • Two kinds: • Hierarchical Q-learning: problem broken into smaller problems each learned separately through Q-learning • W-learning: each module/behavior recommends an action with some weight and action with highest weight selected and executed

11. Arbitration ASMs (3) • 3. Winner-Take-All -- one example • Activation Networks • A set of behaviors reduce the difference between the system’s present state and goal state • By exchange of activation energy, the behaviors compete and cooperate to select an action • The system emergently chooses and performs the next step of the sequence

12. Command Fusion ASMs • Combine recommendations from multiple behaviors to form a control action that represents a consensus • Proceeds in 3 steps • Action recommendations • Behavior aggregation • Action selection

13. CF ASMs – 4 Types • 1. Voting -- 3 examples • A. DAMN • Behavior votes for or against set of actions • Each behavior assigned weight by mode manager • ‘Voter’ selects ‘best’ action • Experiments show DAMN superior to other ASMs

14. CF ASMs – 4 Types (2)Voting cont. • B. SAMBA • Primitive behaviors produce reactions in form of primitive action maps • Behavior outputs generated from 4 primitive action maps • Command arbiter combines maps by multiplying each by a gain and adding the results • C. Action Voting • Each behavior votes for an action and votes against undesirable actions • The votes are summed and action with highest value is selected

15. CF ASMs – 4 Types (3) • 2. Fuzzy Command Fusion -- 2 examples • A. Fuzzy/Multivalued Logic Approach • Control schema • Behavior schema • Planners • B. Fuzzy DAMN • Outputs of behaviors are cast as discrete membership functions over the set of possible actions • Weighted sum replaced with fuzzy inferencing methods • Max-vote replaced with defuzzifcation techniques

16. CF ASMs – 4 Types (4) • 3. Multiple Objective Behavior Coordination (MOBC) • Each behavior calculates an objective function over a set of permissible actions • Action that maximizes the objective function is the best “satisficing” objective • Action selection is comprised of generating and then selecting a set of “satisficing” solutions among a set of efficient solutions known as Pareto-optimal solutions

17. CF ASMs – 4 Types (5) • 4. Superposition Based Command Fusion -- 2 examples • A. Potential Fields • Approach to motion planning • Robot moves under the influence of an artificial potential field produced by an attractive force at the goal configuration and repulsive forces at obstacles • B. Motor Schemas • Generates a vector which encodes the direction and intensity of motor action (calculates a potential field for current configuration of the robot) • These vectors are added to generate a combined motor action • This is then multiplied by a gain value • Used on AuRA