CS483/683 Multi-Agent Systems

1. Computer Science & Engineering, University of Nevada, Reno CS483/683 Multi-Agent Systems • Lectures 5-6: • From Satisfaction to Optimization • ADOPT: Asynchronous Distributed Optimization 2-4 February 2010 Instructor: Kostas Bekris

2. Distributed Constrained Optimization • DCOP: • A set of n variables V={X1, ..., Xn} • Each variable has a discrete domain: D1, ..., Dn • Each variable is assigned to an agent • Only the agent who is assigned a variable has control over its value • And knowledge of its domain • Assign values to variable so as to optimize a global objective function • The optimization of the function satisfies a set of constraints • Requirements: • Distributed computation using only local communication • Fast, asynchronous computation - agents should work in parallel • Quality guarantees are needed • Provably optimal solutions whenever possible • Trade-off between computation and solution quality • Bounded-error approximation: Guarantee solution within a distance from optimal, less time than the optimal

3. ADOPT: Asynchronous Distributed Optimization • The objective function is a summation over a set of cost functions • F(A)=∑Xi,Xj∈V fij(di,dj) • Xi←di, Xj←dj in A • We want to find A* that minimizes F(A) • e.g. F( { (X1,0), (X2,0), (X3,0), (X4,0) } ) = 4 • F( { (X1,1), (X2,1), (X3,1), (X4,1) } ) = 0 X1 X2 X3 X4 Constraint Graph

4. Assumptions • 1. Summation operation over cost function • Associative • Commutative • Monotonic • Cost of a solution can only increase as more costs are aggregated (i.e., we cannot have negative cost) • 2. Constraints are at most binary • There are ways to extend to constraints that involve a larger number of variables • 3. Each agent is assigned a single variable • There is a way to extend to the case that an agent must handle multiple variables

5. Key Ideas in ADOPT • 1. Opportunistic best-first search • Agents are prioritized in a tree structure • an agent has a single parent and multiple children • Each agent keeps on choosing the best value based on the current available information • i.e., chooses the variable which implies the smallest lower bound • lower bounds do not need global information to be estimated • Each agent maintains a lower and an upper bound for the cost of its subtrees • and informs its parent about its own bounds • Strategy allows agents to abandon partial solutions which have not been proven to be suboptimal • they may have to reconsider the same assignments into the future X1 X2 X3 X4 Communication Graph

6. Key Ideas in ADOPT • 2. Backtrack Threshold • When an agent knows from previous search experiences that lb is a lower bound for its subtree • inform the subtree agents not to bother searching for a solution whose cost is lower than lb • In the general case, remembering these lower bounds for past assignments requires exponential space • Approach remembers only one value and then cost is subdivided to children arbitrarily and adapted on the fly as new computations are executed • 3. Built-in Termination Detection • Keeping track of bounds (lower and upper bound for the cost function) on each agent • allows to keep track of the progress towards the optimum solution • and automatically terminates when necessary

7. Messages and Data Structures X1 • VALUE message (like ok?) • Send selected value to children along the • constraint graph • COST message (like NoGood) • Send to parents along the communication graph • THRESHOLD message • Send to children along the communication graph • Each agent maintains the “context” • (like the “agent_view”) • A recorf of higher priority neighbors’ current variable assignment • Two contexts are compatible if they do not disagree on any variable assignment X2 X3 X4 Constraint Graph X1 X2 X3 X4 Communication Graph

8. COST message • Xk transmits COST message to Xi • Message contains • context of Xk • lb of Xk • up of Xk • When Xi receives the message it stores • lb(d,Xk) • ub(d,Xk) • where d is the assignment of Xi in Xk’s context • If context of Xi is incompatible with the context of Xk: • lb(d,Xk) = 0 • ub(d,Xk) = ∞ Xi Xk

9. Costs and Bounds X1 • Local cost: δ(di) = ∑(Xj,dj) fij(di,dj) • di : assignment of agent Xi • Xj : higher priority neighbors than Xi • Lower bound for value d: • ∀ d ∈ Di: • LB(d) = δ(d) + ∑ Xk ∈ children lb(d,Xk) • similarly for the upper bound for value d • Lower bound: • LB = min d ∈ Di LB(d) • similarly for the upper bound • For leaves: LB(d) = UB(d) = δ(d) • If not a leaf but has not get received a COST message: LB = δ(d) and UB = ∞ X2 X3 X4 Constraint Graph X1 X2 X3 X4 Communication Graph

10. When does Xi change value? • Whenever LB(di) exceeds the backtrack threshold value, Xi changes its variable value to one with smaller lower bound • The threshold is updated with the following three ways: • Its value can increase whenever Xi determines that LB is greater than the current threshold • guarantees that there is always a variable with a lower bound than the threshold • Its value can decrease whenever Xi determines that UB is lower than the current threshold • Invariant: LB ≤ threshold ≤ UB • Its value is also updated whenever a THRESHOLD message is received from a parent • a parent subdivides its own threshold value among its children • t(d,Xk): the threshold on cost allocated by parent Xi to child Xk • then the value of t(d,Xk) respects the following invariants: • threshold = δ(di) + ∑ Xk ∈ children t(di,Xk) • ∀ d ∈ Di, ∀ Xk ∈ children: lb(d,Xk) ≤ t(d,Xk) ≤ ub(d,Xk)

11. Example • All agents begin concurrently choosing 0. • Each agents send a VALUE message to lower priority agents along the constraint graph • We will follow one specific execution path - there are many possible X1 X1 X2 X2 X3 X4 X3 X4 Constraint Graph Communication Graph

12. Example X1=0 • X2 receives X1’s VALUE message • and records this value to its context • X2’s context: {X1=0} • Then it computes bounds: • LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 1 • LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 2 • LB(0) < LB(1) ⇒ LB = LB(0) = 1 • Similarly: UB = ∞ • threshold was set to LB(0), equal to 1 so the invariant holds • Transmits a COST message to X1: • COST( {X1=0}, 1, ∞ ) X2=0 X4=0 X3=0

13. Example X1=0 • X3 receives X1’s and X2’s VALUE messages • and records these values to its context • X3’s context: {X1=0, X2=0} • Then it computes bounds: • LB(0) = δ(0) = 1 + 1 = 2 • LB(1) = δ(1) = 2 + 2 = 4 • LB(0) < LB(1) ⇒ LB = LB(0) = 2 • Similarly: UB = 2 • threshold was LB(0) = 2 so the invariant holds • Transmits a COST message to X2: • COST( {X1=0, X2=0}, 2, 2 ) • Similarly with X4... but no reference to X1 X2=0 X4=0 X3=0

14. Example X1=1 • X1 receives X2’s COST message • COST( {X1=0}, 1, ∞ ) • test if compatible with its current context • store: lb(0,X2) = 1 and ub(0,X2) = ∞ • Then it computes bounds: • LB(0) = δ(0) + lb(0,X2) = 0 + 1 = 1 • LB(1) = δ(1) + lb(1,X2) = 0 + 0 = 0 • LB(1) < LB(0) ⇒ LB = LB(1) = 0 • Similarly: UB = ∞ • threshold was 0, but LB(0) = 1: • violation of the invariant, change assignment • Send VALUE messages to children X2=0 X4=0 X3=0

15. Example X1=1 • Assume COST messages from X3 and X4 are delayed... instead VALUE message from X1 arrives first at X2 • Current context at X2: {X1=1} • When X2 receives the COST messages from X3 its context will be incompatible with X2’s • the bounds in the message will not be stored • The message from X4 is not incompatible: • store lb(0,x4) = 1 and up(0,x4) = 1 • Then it computes bounds: • LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 2+ 0 +1 = 3 • LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 0 +0 +0 = 0 • LB(1) < LB(0) ⇒ LB = LB(1) = 1 • Similarly: UB = ∞ X2=1 X4=0 X3=0

16. Example X1=1 • X2 will inform X3 and X4 about the changes. • Similar changes will take place on X3 and X4: • 1 will be selected as the value • COST messages will be transmitted: • from X4 to X2: ( {X2=1}, 0, 0 ) • from X3 to X1 and X2: ( {X1=1,X2=1}, 0, 0 ) • from X2 to X1: ( {X1=1}, 0, 0) • this is after receiving the COST messages from X3 and X4 • Upon receipt of the COST message of X2 at X1: • LB = UB = threshold = 0 • X1 sends TERMINATE messages to other agents. X2=1 X4=1 X3=1