Distributed Problem Solving & Planning

1. CISC 889: Agents, Web Services and Grid Computing Fall 2003 Distributed Problem Solving & Planning By: Sachin Kamboj

2. Outline • Introduction • Motivation • Distributed Problem Solving Strategies • Task Sharing • Result Sharing • Distributed Planning • Distributed Planning and Execution • Comparison with Web Services and Grid Computing

3. Introduction • Distributed Problem Solving • Agents work together to solve problems that require collective effort. • Requires: • Coherence • Need to want to work together • Competence • Need to know how to work together • Distributed Planning • Problem Solving in which the agents work together to construct a plan.

4. Motivation • Resources may be distributed • Using distributed resources concurrently can allow a speedup of problem solving due to parallelism.

5. Motivation • Problem solving capabilities may be distributed • Separate processes may solve different parts of the problem. Online Auction on EBay Product Search Bid Manager Credit Card Billing Product Delivery

6. Agent 1 Agent 2 Agent 3 Agent 4 Motivation • Data may be distributed • Example: In distributed sensor networks

7. Motivation • Results may be distributed • The results may have to be acted upon by multiple agents.

8. Distributed Problem Solving Strategies • Task Sharing • Result Sharing • Distributed Planning • Distributed Planning and Execution

9. Task Sharing

10. Task Sharing • Task Decomposition • Generate a set of tasks to be passed to others • Decompose large tasks into subtasks that can be tackled by different agents • Task Allocation • Assign subtasks to appropriate agents • Task Accomplishment • Agents accomplish task, either individually or through further decomposition • Result Synthesis • Combine the results to generate the final results

11. Task Sharing Example: Towers of Hanoi • Task Decomposition: • Means-End Analysis: • Move the largest disk that is misplaced to its destination peg. • Get to the state where the largest disk can be moved. • Move the other disks on top of the largest disk to a peg other than the destination of the largest so that the largest can be moved. • Task Allocation: • Allocate sub-problems to identical agents capable of solving it. • Task Accomplishment: • If the recursive sub-problems have same start and goal states terminate. • Result Synthesis

12. Task Sharing Example: Towers of Hanoi

13. Car Design Agent Engine Transmission Brakes Aerodynamics … … Task Sharing in Heterogeneous Systems • Motivation: • Difficult to build agents that are competent in every possible task. • Will result in agents that are “Jacks of all trades and masters of none” • Omni-capable agents would only use a small proportion of their capabilities at any one time.

14. Task Sharing in Heterogeneous Systems • Solution: • Build specialist agents that can work together to solve problems beyond their individual capabilities. • Dynamic combination of heterogeneous agents… Car Design Agent Engine Design Agent Body Design Agent Transmission Design Agent Brakes Design Agent Fuel Injection Design Agent Cylinder Design Agent

15. Task Sharing in Heterogeneous Systems • Problem: • How can an agent that has to solve a problem find the “specialists”? • Naïve Solution: • Have a table that identifies the capabilities of the agents. • Use table to select appropriate agents and give them the sub-problem. • Problems: • How is selection done if several candidate agents exist?

16. Task Sharing in Heterogeneous SystemsSelection of Agents • Announce a sub-problem to specific agents. • Request return bids describing acceptance/availability • Assign sub-problem to the highest bidder • Problem: • What if no agents bid for a sub-problem? • Solution: Use one of the following: • Broadcast Contracting • Retry • Announcement Revision • Alternative Decomposition

17. Result Sharing

18. Results Sharing • Motivation: • Results of different problem solvers on the same task may differ… • Sharing results can improve group performance by increasing: • Confidence • Completeness • Precision • Timeliness • Agents need to know how to assimilate results from different agents • Assimilation may be non-trivial/costly/require large amounts of communication and incur overheads in management.

19. Functionally Accurate Cooperation • Agents lack information about what is happening elsewhere • This information might impact their calculations. • As a result agents form tentative results that might be unnecessary • Functionally Accurate: • Agents get the answer eventually • But with many possible false starts • Cooperative: • Iterative exchange between agents is required

20. Functionally Accurate Cooperation • Advantages: • Exchanging tentative partial solutions can impact: • Completeness • Partial solutions are synthesized into larger solutions that solve more of the overall solution • Precision • Agents may use results from other agents to refine their own solutions • Confidence • Confidence measures can be combined to increase overall confidence in the solution.

21. Functionally Accurate Cooperation • Problems: • Iterative exchange of partial results should lead to some agent having enough information to keep moving the overall problem solving forward • A large amount of communication overhead is involved in the sharing of partial results. • May result in wasted computations • May lead to Distraction: • Agents gravitate towards doing the same problem solving actions.

22. Shared RepositoriesMotivation • Sharing partial results involves a lot of communication overhead Agent 7 Agent 3 Agent 1 Agent 6 Agent 4 Agent 2 Agent 8 Agent 9 Agent 5

23. Shared Repositories • Instead of agents multicasting partial results to all other agents, the results are stored in a central repository • Also known as a blackboard Agent 1 Agent 3 Agent 4 Agent 7 Shared Repository Agent 9 Agent 6 Agent 2 Agent 5 Agent 8

24. Shared Repositories • Support search through alternative designs • Agents with different design criteria can revise and critique the alternatives • Similar to a distributed constraint satisfaction problem

25. Negotiated Search • Negotiation: • An agent generates a proposal and other agents review it: • If an agent does not like the proposal, it rejects it • Agents might also generate counter-proposals • Proposals are reviewed iteratively • Negotiated Search: • A use of shared repositories • Agents concurrently investigate multiple solution paths • Conflicts between solution are resolved using negotiation • Agents have operators for initiating solutions, extending solutions, critiquing solutions and relaxing solution requirements

26. Distributed Constrained Heuristic Search • Used to handle distributed resource allocation • An agent is associated with each resource • DCHS works as follows: • Agent begins with a problem topology • Agent determines resource requirement and computes a demand profile • Agent sends profile to the resource • Resource computes aggregate demand and informs the agents making the demands • Agent uses aggregate demand to identify preferred resource • Agent requests the resource to reserve the specified interval • Resource grants reservation if possible • Agent processes the response from the resource

27. Organizational Structuring • An alternative strategy for reducing communication overhead • Exploits the task decomposition structure/ organizational structure of agents • Define roles, resources and preferences for the agents • Associates with each agent the type of tasks that it can do • Agents need to be informed of partial results that can influence how they carry out their responsibilities • Agents need not be told of results that do not affect their actions • Make use of stored pattern-response rules

28. Communication Strategies • Concerned with timing issues • Agents need to know not only what results to send but also when to send them • Sending a result too soon can: • Clutter up the recipient memory • Distract the recipient away from useful work • Failure to send results can: • Lead to delays • Result in a failure of the system to converge on reasonable solutions

29. Communication Strategies • When possibility of cluttering memory is short: • Send all partial results • If exchange of partial results leads a subset of agents into redundant exploration of a part of the solution space: • Refrain from sending partial results • Alternatively, only send results when requested • Especially when space of results formed is large and only a few are needed by others.

30. Distributed Planning

31. Traditional Planning • Set of states • Conjunction of terms that are true about the world • Set of operators • Preconditions • Things that have to be true about the world to apply the operator • Effects • Changes to the current state (Add List, Delete List) • Goals • Planning is a backward search from the goal state to the start state • At each step, the agent tries to find an operator whose effect achieves a goal or a precondition of another operator that has already been applied

32. Start At (Home), Sells (SM, Bananas), Sells (SM, Milk), Sells (HW, Drill) At (HWS) At (Home) At (HWS), Sells (HWS, Drill) At (SM), Sells (SM, Milk) At (x) At (SM), Sells (SM, Bananas) At (x) Go (SM) Buy (Drill) Buy (Milk) Go (HWS) Go (HWS) Go (SM) Buy (Bananas) Have (Drill), Have (Milk), Have (Bananas), At (Home) Finish Traditional Planning(Example)

33. Traditional Planning(Simplifying Assumptions) • Planning agent is omniscient • The action that the agents can perform have definite outcomes • The goals presented to the agents are categorical • They are either achieved or not • There is no partial satisfaction of goals • The agent is the only source of change in the environment • The goals presented remain unchanged throughout the process of planning and execution • Actions do not have temporal extent or duration of occurrence

34. Traditional Planning(Consequences of Underlying Assumptions) • There is no need to interleave planning and execution • Because agent has all the knowledge it needs at planning time • No need for re-planning • A plan can always be executed successfully • Planning problems do not compete with each other for the planning agent’s attention. • Goals are all known at the outset and remain fixed throughout planning and execution • Assumptions fail to hold for realistic situations in which planning is applied…

35. Distributed Planning(Example Scenario) • You have a project due on Monday • You decide to work on it over the weekend • Exact time is not fixed: will depend on other plans • Tentative Plans: • Saturday Morning: Run errands • Saturday Night: Watch ‘The Matrix’ • Sunday: Finish off project • Saturday Afternoon • Receive an invitation from a friend to her birthday party on Sunday evening • Revise Plans: • Saturday Morning: Run errands • Saturday Evening/Night: Work on Project • Sunday Morning: Finish off project • Sunday Night: Go to friends birthday party

36. Distributed Planning • Aims to overcome some of the limitations of traditional planning • Handle the example scenario just presented • Specialization of distributed problem solving, where the problem to be solved is to design a plan • Distributed means: • Planning is distributed • Plans are distributed • Both planning and plans are distributed

37. Distributed PlanningCentralized Planning for Distributed Plans • Formulation of plans that are to be executed in a distributed fashion in a centralized manner • Example: • A partial order planner can generate plans that do not have a strict ordering between actions • Such actions can be executed in parallel by multiple agents • Algorithm: • Generate a partial order plan • Decompose plan into sub-plans • Insert synchronization actions into sub-plans • Allocate sub-plan to agents using task passing mechanisms • Initiate plan execution and monitor progress • Algorithm is a specialization of task sharing

38. Distributed PlanningDistributed Planning for Centralized Plans • Formulating a complex plan may require collaboration among a variety of cooperative planning specialists • Similar to task-sharing and result-sharing problem solving • Overall problem formulation task is decomposed and distributed among various planning specialists • Each planning specialist generates its portion of the plan • Interaction between specialists might be through the exchange of partially specified plans • Example: • Machining of tools in the manufacturing domain

39. Distributed PlanningDistributed Planning for Distributed Plans • Both the planning process and its results are distributed • Multi-agent plans are not represented in entirety anywhere in the system • No one agent knows the complete plan • Distributed pieces of a plan should be compatible • Techniques that can be used: • Plan merging • Iterative plan formation • Negotiation in distributed planning

40. Distributed Planning for Distributed PlansPlan Merging • Each agent formulates its own plan individually • Agents work together to ensure that their separate plans can be executed without conflict • Local plans are exchanged • Messaging and/or timing commitments are imposed to resolve negative plan interactions • Centralized plan coordination approach • An agent collects together the individual plans • Agent analyzes the plans to discover the sequence of actions that leads to a conflict • Reachability Analysis • Enumerate all possible states of the world that can be reached from the initial state by executing actions • Plans are modified to remove conflicts

41. Distributed Planning for Distributed PlansIterative Plan Formation • Not all problems allow agents to develop their own local plans • Require agents to search through larger spaces of plans rather than each proposing a single specific plan • Each agent might construct a set of all feasible plans • The planning process consists of fitting together subsets of agents plans • Example methods: • Plan Combination Search • Distributed Hierarchical Planning

42. Distributed Planning and Execution • Plan merging: • Each agent individually builds a plan. • Individual plans are coordinated • Results of coordination are executed. • If one or more of the agents should fail in carrying out their plans, the whole coordinated plan set might fail • Post-planning coordination • Done after one or more plans fail • Methods: • Contingency planning • Each agent formulates alternative plans to respond to possible contingencies • Monitoring and replanning • If there is a deviation from a plan stop all execution and repeat the plan-coordinate-execute cycle

43. Distributed Planning and ExecutionPre-planning coordination • Coordination is done in such a way that no matter what happens during execution they agents will remain coordinated with each other • Introduces coordination restriction • Such restrictions usually take the form of social laws • A prohibition against particular choices of actions in particular contexts • For example, entering an intersection on a red light • Restrictions are derived by working back from undesirable states: • Combination of actions that lead to those states are found • Restrictions are imposed so that those actions cannot occur

44. Distributed Planning and ExecutionInterleaving Planning, Coordination & Execution • In certain situations, it might be impractical to distinguish between planning and execution • The environment may be highly dynamic • Assumptions made by the agent while planning may no longer be true • A single agent cannot decide on the future course of action without exchanging info with other agents • Example situations which required interleaving: • Distributed sensor networks • Dynamic resource allocation • May make use of partial global planning

45. Comparison with Web Services and Grid Computing • Method of distributed problem solving depends on the application • Includes a large variety of techniques • There is no one “standard” or silver bullet. • Same methods may be used in web services and grid computing • Distributed constrained heuristic search may be used in grid computing for the allocation of resources • Web services choreography is essentially a form of planning with preconditions, post-conditions, triggering events and flows

46. References • Edmund H. Durfee: “Distributed Problem Solving and Planning”, in Chapter 3, Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, Gerhard Weiss (Editor)

47. Thank You!! Questions?