Multi-agent Planning

1. Multi-agent Planning Amin Atrash

2. Papers • Dynamic Planning for Multiple Mobile Robots • Barry L. Brummit, Anthony Stentz • OBDD-based Universal Planning: Specifying and Solving Planning Problems for Synchronized Agents in Non-Deterministic Domains • Rune M. Jensen, Manuela M. Veloso

3. Dynamic Mission Planning for Multiple Mobile Robots • Goal: Coordinate the actions of multiple robots to achieve a goal. • Dynamically reassign goals to robots as information about the environment is updated. • Handle multiple robots, multiple goals, and dynamic environments.

4. Architecture • Local Navigator – Takes local information and chooses steering direction for robot. (obstacle avoidance). • Dynamic Planner – Updates path of robot to goal based on updated maps (D*). • Mission Planner – Updates goal assignments to robots.

5. Scenario • Multiple Travelling Salesman Problem. • M goals with N robots. • M dynamic planners running, each maintaining a path from each robot to the planner's assigned goal (D* planners). • Robots moving in randomly generated environment. • As environment is updated, D* planners update path to all goals, and mission planners reassign goals to robots. • Mission Planner uses exhaustive search of possible combinations.

6. Results/Conclusions • With 3 robots and 6 goals, there was 25% improvement using dynamic mission planner compared to baseline planner which never changed initial goal assignments. • Shown that complex missions can be performed with using reasonable computation.

7. OBDD-based Universal Planning: Special and Solving Planning Problems for Synchronized Agents in Non-deterministic Domains • Uses Ordered Binary Decision Diagrams (OBDDs) to encode a domain as a non-deterministic finite automaton then apply fast model checking. • Develop NADL.

8. Idea • Given a domain. • Generate NFA. Transitions defined by OBDD. • Use model checking to find solution. • Should generate universal plans – set of state-action rules which cover all possible situations in non-deterministic environment. • All planning is done prior to execution. • NADL – language for encoding a domain. • Non-deterministic Agent Domain Language.

9. ODBB x1 • Ordered Binary Decision Diagrams. • Represent boolean functions. • Efficient representation because number of nodes is often much smaller than number of truth assignments. • Operation complexity bound by number of nodes. x2 1 0

10. NADL • State variables, system agents, environment agents, initial conditions, goal conditions. • Each action has fixed equal duration. • All agents each perform one action. • All agents together for action tuple: Joint action. • Actions defined as set of state variables, precondition formula, and effect formula. • Non-determinism occurs when actions do not restrict all variables to a specific value and with non-deterministic selection of actions.

11. NADL Example Variables nat(4)pos bool robot_works system agt:Robot Lift-Block con: pos pre: pos<3 eff: robot_works -> pos' = pos+1, pos' = pos Lower-Block con: pos pre: pos>0 eff: robot_works -> pos' = pos+1, pos' = pos environmnet agt:Baby Hit-Robot con: robot_works pre: true eff: robot_works robot_works’ initially pos = 0 and robot_works goal pos = 3

12. NADL, NFAs, and OBDDs • Given an NADL description, a Non-deterministic Finite Automata (NFA) can be generated. • OBDD used to represent transition function. • Define set of variables to represent current states, joint actions, and next state. • Generate OBDD.

13. OBDD-based Planning • Preimage(V) – all states, s', such that there exists action, a, in s' which will lead to a state, s in V. • Strong planning. • For a state belonging to the preimage of a set of states, V, there exists at least one input, i, where all transitions from s associated to i lead into V. • Start with set of goal states. • Iterate a backwards BFS. • Stop when all initial states are included in set of visited states. • Strong cyclic planning – similar to strong planning but also considers plans with loops.

14. ODBB-based Planning Initial Goal pre1 pre2 pre3

15. Optimistic Planning • Strong planning is pessimistic. • Will avoid short path with chance of entering failed state for longer safer path. • Usually not feasible in real world. • Especially with non-deterministic domains. • Optimistic planning – In scenarios where a strong plan cannot be found, an optimistic plan can be used. • Considers actions which can lead to failed states.

16. Results – Deterministic Domains • Gripper Domain - Able to solve larger problems than other planners • Movie Domain – Outperformed other traditional planners and returned optimal plan • Logistics Domain – Unable to solve problem. • Possibly due to bad representation or variable ordering • Obstacle Domain

17. Results – Power Plant • Power Plant Domain – 4 heat exchangers, 4 turbines, 1 reactor. • Good, bad, and failed state. • Heat exchangers can fail and need to be blocked. • Turbines can fail and need to be stopped. • Need at least on heat exchanger and turbine working.

18. Results – Soccer Domain • Two teams of players in grid world. • Players can move or pass ball. • Goal: Have player in front of opponent goal without any opponents in area.

19. Conclusions • Developed expressive description language • Applied OBDD planning • Proposed “optimistic planning.” • Showed use in multiagent non-deterministic domains