Abstract Reasoning for Multiagent Coordination and Planning

1. Abstract Reasoning for Multiagent Coordination and Planning Bradley J. Clement

2. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

3. Manufacturing Example Production, Inventory, andFacility Managers

4. Problem Characteristics • Managers must coordinate or risk failure. • Managers develop plans independently. • Managers need sound and complete coordination algorithm. • Managers may need to make coordination decisions quickly. • Managers must reason about concurrent action to use resources efficiently. • Managers may need plans that handle unexpected events.

5. Problem Find preferable elaborations or modifications to a group of agents’ plans that achieve their goals while striking a balance among the following objectives: • Coordination (or planning) should be sound & complete. • Agents should not coordinate (reason about subgoals) where there are no conflicts. • Agents should act as soon as possible. • Agents should accomplish goals efficiently. • Agents should act concurrently. • Agents should maximize utility. • Agents should be able to handle unexpected events.

6. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

7. Approach • Reason about plans at abstract levels to reduce the information needed to make efficient coordination and planning decisions • concurrent hierarchical plan representation • summarize constraints of abstract tasks from those of tasks in their decompositions • use this summary information to reason about interactions of abstract plans • Construct sound and complete coordination & planning algorithms • Explore techniques and heuristics for decomposition search based on summary information • Analyze complexity of abstract reasoning • Evaluate in different domains

8. crisper solutions lowercoordinationcost coordination levels moreflexibility Approach • Complete at high level using summary information to gain flexibility in execution • Better solutions may exist at lower levels • Summary information aids in pruning subplans to resolve threats

9. How Approach Addresses Problem • Coordination (or planning) decisions should be sound & complete. • Formalize summary information and algorithms • Agents should not coordinate (reason about subgoals) where there are no conflicts. • Use decomposition techniques and heuristics to focus search • Agents should act as soon as possible. • Find solutions efficiently at multiple levels of abstraction • Agents should accomplish goals efficiently. • Agents should act concurrently. • Reason about concurrent interactions at abstract levels • Agents should maximize utility. • Use decomposition techniques and heuristics to guide search to better solutions • Agents should be able to handle unexpected events. • Preserve decomposition choices by finding abstract solutions

10. Approach - Limitations • Do not offer algorithms/protocols that determine optimal balancing of problem objectives • do give mechanisms that enable tradeoffs • Do not investigate alternative coordination/negotiation protocols • instead, identify who needs to coordinate, what needs to be coordinated, and alternative settlements • Planning language • Only grounded, propositional states formalized • mention how uninstantiated variables are implemented • Metric resource usage is instantaneous

11. Contributions • Algorithms for deriving and reasoning about summary information • Sound and complete concurrent hierarchical coordination & planning algorithms • Integration of summary information in a local search planner • Search techniques and heuristics that efficiently guide decomposition and prune the search space • Complexity analyses and experiments that show where abstract reasoning exponentially reduces cost of computation and communication

12. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

13. Related Approaches

14. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

15. B B B B Concurrent Hierarchical Plans (CHiPs) • pre, in, & postconditions - sets of literals for a set of propositions • type - and, or, primitive • subplans - execute all for and, one for or; empty for primitive • order - conjunction of point or interval relations B - before

16. Summary Conditions • existence: must, may • timing: always, sometimes, first, last • external preconditions • external postconditions pre:available(A), available(M1), available(M2) pre:available(A), available(M1) pre:available(A), available(M2) pre:available(A), available(M2) pre:available(A) pre:available(A)

17. Deriving Summary Conditions • Can be run offline for a domain • Recursive algorithm bottoming out at primitives • Derived from those of immediate subplans • O(n2c2) for n non-primitive plans in hierarchy and c conditions in each set of pre, in, and postconditions • Properties of summary conditions are proven based on procedure • Proven procedures for determining must/may - achieve/undo/clobber

18. Metric Resource Usage interval of task • Depletable resource • usage carries over after end of task • gas = gas - 5 • Non-depletable • usage is only local • zero after end of task • machines = machines - 2 • Replenishing a resource • negative usage • gas = gas + 10 • can be depletable or non-depletable

19. Summarizing Resource Usage summarized resource usage  < local_min_range, local_max_range, persist_range > • Captures uncertainty of decomposition choices and temporal uncertainty of partially ordered actions • Can be used to determine if a resource usage may, must, or must not cause a conflict 40 30 20 10 0 -7 -20 < [-20, -7],[30, 40],[10, 20] >

20. Resource Summarization Algorithm • Can be run offline for a domain model • Run separately for each resource • Recursive from leaves up hierarchy • Summarizes parent from summarizations of immediate children • Considers all legal orderings of children • Considers all subintervals where upper and lower bounds of children’s resource usage may be reached • Exponential with number of immediate children, so summarization is really constant for one resource and O(r) for r resources

21. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

22. Determining Temporal Relations • CanAnyWay({relations}, {psum, qsum}) - relationscan hold for any wayp and q can be executed • MightSomeWay({relations}, {psum, qsum}) - relationsmight hold for some wayp and q can be executed B - before produce H maintenance O - overlaps CanAnyWay({before}, {produce_H, maintenance}) ØCanAnyWay({overlaps}, {produce_H, maintenance}) MightSomeWay({overlaps}, {produce_H, maintenance})

23. Concurrent Hierarchical Plan Coordination • Agents individually derive summary information for their plan hierarchies • Coordinator requests summary information for expansions of agents’ hierarchies from the top down • After each expansion, try to resolve threats by adding ordering constraints • Algorithm shown to be sound and complete

24. Search for Coordinated Plan blocked • search state • set of expanded plans • set of blocked subplans • set of temporal constraints • search operators • expand • block • constrain temporal constraints blocked

25. Easier to Coordinate at Higher Levels • Number of plan steps per level grows exponentially down the hierarchy O(bi) • In worst case, summary information for each plan grows exponentially up the hierarchy O(bd-ic) • Number of orderings of plans grows exponentially down hierarchy O(bi!) • Resolving threats is NP-complete (reduced from Hamiltonian Path) • In worst case, search space reduced by O(kbd-bi). • In best case, O(kbd-bib2(d-i)). b - branching factor i - level d - depth c - conditions per plan

26. Search Techniques • Prune inconsistent global plans • Branch & bound - abstract solutions help prune space where cost is higher • “Expand most threats first” (EMTF) • expand subplan involved in most threats • focuses search on driving down to source of conflict • “Fewest threats first” (FTF) • search plan states with fewest threats first • or subplans involved in most threats are blocked first

27. Evacuation Domain Experiments • Compare different strategies of ordering search states and ordering expansions • FAF-FAF • DFS-ExCon • FTF-EMTF • FTF-ExCon • 4 - 12 locations • 2 - 4 transports • no, partial, & complete overlap in locations visited

28. Evacuation Domain Experiments Summary information decomposition techniques outperform previous state-of-the-art by orders of magnitude

29. Evacuation Domain Experiments Decomposition techniques using summary information dominate previous heuristics in finding optimal solutions • FTF especially effective compared to random, DFS, and FAF • EMTF not especially more effective than ExCon but finds solutions more regularly • Overall performance differs by orders of magnitude

30. Communication in Manufacturing Domain • Centralized coordinator • Measure delay with varying bandwidth and latency: (n-2)l + s/bn = number of messagess = total size of messagesl = latencyb = bandwidth

31. Communication in Manufacturing Domain • Agents can minimize communication by sending summary information at intermediate levels with a particular granularity. • Sending all plan information at once can be exponentially more expensive: O(bd-i). • Sending summary information one task at a time can cause exponentially greater latency: O(bi). • However, if summary information does not collapse up the hierarchy, and coordination must occur at primitive levels, sending all at once is best. • Domain modeler can perform similar experiment to determine appropriate granularity to send summary information.

32. Multi-Level Coordination Agent (MCA) • Centrally coordinates plans of requesting agents in episodes • Requests summary information as needed or summarizes given hierarchies • Displays discovered solutions that are “better” or Pareto-optimal • Sends synchronization and decomposition choice constraints to agents upon selection of a solution

33. Forces separated by Firestorm N Cape Amstado Kaso Lagoon GAO W E Jacal S Caca LAKE CACA Daka Binni Gao forces Mawli Amisa False Gao forces White Caca Laki Safari Park Afram Pra Ofin Kapowa Agadez Forces FIRESTORM Cape Vincent Ankobra Tana False Agadez forces Black Caca AGADEZ

34. Multi-Level Coordination of Military Coalitions

35. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

36. Concurrent Hierarchical Refinement Planner • Simple modification to coordination algorithm • discover whether potential internal conflicts exist during summarization • must expand any task with potential internal conflicts • Derive summary information for hierarchy expanded to primitive level (iteratively expand for infinite recursion of methods) • Expand hierarchy from the top down, selecting or blocking or decomposition choices • After each expansion, try to resolve threats • add ordering constraints • check CAW and MSW • Sound and complete • Same complexity benefits as coordination algorithm

37. lowerplanningcost planning levels crisper solutions moreflexibility Summary Information in Local Search Planners • Local plan-space search involves modifying (e.g. deleting, moving, adding, etc.) tasks in an existing plan. • Hierarchy is used to pass parameters, specify temporal constraints, and explore alternative decompositions for subtasks. • Planners like ASPEN fix the start times and durations of activities and track states and resources within a time horizon. • Algorithms for reasoning about summary states and resources are used to track uncertain states/resources for abstract tasks. • Using summary information results in more efficient planning and scheduling.

38. Complexity Analyses: Local Search level branchingfactor b 0 1 . . . d 1 2 n • Moving an activity hierarchy is a factor of O(b2(d-i)) more complex at level d than i if summary information fully collapses up the hierarchy. • If no information collapses, moving a hierarchy has the same complexity at all levels O(vnb2d). • The number of potential temporal constraint conflicts is a factor of O(bd-i) greater at level d than i. • Thus, reasoning at abstract levels can resolve conflicts exponentially faster. c constraintsper hierarchy vvariables

39. Decomposition Strategies • Level expansion • repair conflicts at current level of abstraction until conflicts cannot be further resolved • then decompose all activities to next level and begin repairing again • Expand most threats first (EMTF) • instead of moving activity to resolve conflict, decompose with some probability (decomposition rate) • expands activities involved in greater numbers of conflicts (threats) • FTF (fewest-threats-first) heuristic tests each decomposition choice and picks those with fewer conflicts with greater probability.

40. Multi-Rover Domain • 2 to 5 rovers • Triangulated field of 9 to 105 waypoints • 6 to 30 science locations assigned according to a multiple travelling salesman algorithm • Rovers’ plans contain 3 shortest path choices to reach next science location • Paths between waypoints have capacities for a certain number of rovers • Rovers cannot be at same location at the same time • Rovers cannot cannot cross a path in opposite directions at the same time • Rovers communicate with the lander over a shared channel for telemetry--different paths require more bandwidth than others

41. Experiments using ASPEN for a Multi-Rover Domain Performance improves greatly when activities share a common resource. Rarely shared resources (only path variables) Mix of rarely shared (paths) and often shared(channel) resources Often shared (channel) resource only

42. Overview • Problem description • Summary of approach • Related work • Representations and supporting algorithms • CHiPs • Metric resources • Summary information • Coordination algorithm • Complexity analyses • Decomposition search techniques • Applications and experiments • Planning • Concurrent hierarchical refinement and local search planners • Scheduling complexity • Mars rovers experiments • Conclusion

43. Contributions • Algorithms for deriving and reasoning about summary information for propositional state and metric resources • must/may assert, achieve, clobber, undo • CAW & MSW to determine whether abstract plans are conflict free or unresolvable • toolbox of sound and complete algorithms for constructing efficient coordination and planning algorithms

44. Contributions • Coordination and planning algorithms • sound, complete concurrent hierarchical coordination • sound, complete concurrent hierarchical planner • iterative repair planner employing abstract reasoning with summary information • evaluated in manufacturing, evacuation, military operations, and Mars rovers domains

45. Contributions • Complexity analyses and experiments • Finding solutions at abstract levels is exponentially less complex O(kbd-bi) in number of tasks for both refinement and local search. • Finding abstract solutions is exponentially less complex when summarization collapses constraints O(b2(d-i)) for both refinement and local search. • Experiments support the analyses in evacuation and Mars rovers domains. • Communication delay can be reduced exponentially by • gradually sending summary information O(bd-i) and • sending at an appropriate granularityO(bi). • Extension of work by Korf ’87 and Knoblock ‘91 showing how hierarchical coordination/planning can obtain exponential speedups when subgoals interact

46. Contributions • Decomposition search techniques • EMTF, FTF (for refinement and local search) • Pruning of inconsistent and costlier search space • Evaluation against prior heuristics showing stronger ability to find optimal solutions at lower abstraction levels

47. Future Directions • Applying summary information to other classes of coordination/planning • state-based search • complex resources • more expressive temporal models • Summarizing other information • constraint hierarchies (in addition to task hierarchies) • reasoning about uncertainty and risk • Coordination protocols based on summary information • organization and scaling of agent groups • BDI-based multiagent mental models • Coordinating continuously • Interfacing deliberative and reactive coordination • Exploiting synergy while coordinating • Case-based coordination