1 / 17

Meta-Level Control in Multi-Agent Systems

Meta-Level Control in Multi-Agent Systems. Anita Raja and Victor Lesser Department of Computer Science University of Massachusetts Amherst, MA 01002. Bounded Rationality.

nhi
Download Presentation

Meta-Level Control in Multi-Agent Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Meta-Level Control inMulti-Agent Systems Anita Raja and Victor Lesser Department of Computer Science University of Massachusetts Amherst, MA 01002

  2. Bounded Rationality “A theory of rationality that does not give an account of problem-solving in the face of complexity is sadly incomplete. It is worse than incomplete; it can be seriously misleading by providing solutions that are without operational significance” Herb Simon, 1958 Basic Insight: Computations are actions with costs

  3. Motivation • Control actions like scheduling and coordination can be expensive • Current multi-agent systems do not explicitly reason about these costs • Need to account for costs at all levels of reasoning to provide accurate solutions • Build meta-level control framework with minimum cost that reasons about cost of different control actions

  4. Assumptions • Agent can pursue multiple tasks simultaneously • Agent can partially fulfill or omit tasks • Agent can coordinate with other agents to complete tasks • Tasks have varying arrival times, deadlines and associated utilities • Tasks have alternate ways of being achieved • Objective function: MAX utility over a fixed time horizon

  5. Agent Architecture

  6. Meta-level Decision Taxonomy • Whether to accept, delay or reject an incoming new task? • How much effort to put into reasoning about a new task? • Whether to negotiate with another agent about task transfer? • Whether to renegotiate in case of failure of previous negotiation? • Whether to re-evaluate current plan when a task completes?

  7. Decision Tree for New task arrival event

  8. Some State Features

  9. Some Heuristic Decisions • If current schedule has low priority (expected quality is low) and incoming task is of high priority (high expected quality with tight deadline), then drop current schedule and schedule new task immediately. • If current schedule has very high priority and new task has low expected utility and a tight deadline, drop the new task • If current task to be scheduled has high execution uncertainty associated with it and a deadline which is not tight, then introduce high slack in the schedule and use medium scheduling effort

  10. Related Work • Monitoring Progress of Anytime Algorithms (Hansen & Zilberstein) • Uses dynamic programming for computation of a non-myopic stopping rule • Predictability versus Responsiveness (Durfee & Lesser) • Control amount of coordination using a user specified buffer • Meta-level Control of Coordination Protocols (Kuwabara) • Detects and handles exceptions by switching between protocols • Does not account for overhead of reasoning process

  11. Evaluation • Compare system using hand-generated MLC heuristics to • Naïve multi-agent system with no explicit MLC • Deterministic choice MLC • Random choice MLC • MLC with knowledge of environment characteristics including arrival model • Environments are characterized by the following parameters • Type of tasks : Simple (S), Complex (C), Combination (A) • Frequency of Arrivals: High (H), Medium (M), Low (L) • Deadline Tightness: High (H), Medium (M), Low (L)

  12. An Example

  13. Evaluation, Continued

  14. Evaluation, Continued

  15. Contributions • Meta-level control in a complex environment • Designed agent architecture that reasons about overhead at all levels of the decision process • Parametric control algorithm which reasons about effort and slack • Identified state features for control using reinforcement learning

  16. Future Work • Implement Reinforcement-Learning based control algorithm • Function approximation (Sarsa() linear tile-coding) • MDP states will be abstractions of actual system state • Study effectiveness of RL algorithm on complex domain • Compare performance of heuristic approach to RL approach

  17. Research Questions • What are the major obstacles to efficient meta-level control? • How can costs be accurately included at all levels of reasoning? • How to deal with the huge, complex state space? • Is reinforcement learning a feasible approach to learn good meta-level control policies?

More Related