Distributed cooperation and coordination using the Max-Sum algorithm

1. Distributed cooperation and coordination using the Max-Sum algorithm Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-08), 12-16 May 2008, Estoril, Portugal. Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on Agent Technology for Sensor Networks, Estoril, Portugal.

2. What is Max-Sum? • Max-sum is a derivative of the sum-product algorithm, formed by transforming the max-product algorithm into log-space.

3. What is Sum-Product? • General form of the forward-backward algorithm • Iterative algorithm to compute the marginals of functions • a.k.a Belief Propagation (when applied to probabilistic graphical models) • Information Theory • Machine Learning • Vision

4. An abstract problem Consider a function, F, that is dependent on N variables, x = {x1 . . . xN}, and is defined as a product of M factors such that: where each of the factors fm(xm) is a function of a subset xm of the variables that make up x.

5. An abstract problem Find the marginal function zn(xn) that describes the dependency of F(x) on variable xn: The marginal zn(a) is defined to be the sum of F(x) over all configurations of variables where xn = a.

6. Factor graph representation • Bipartite graph consisting of two types of nodes • Variable nodes • Function nodes

7. Factor graph representation • Example: F = f1(x1,x2)f2(x2,x3)

8. What is Sum-Product? • Given a factor graph • Apply a local message passing procedure • Compute the marginals of functions

9. Two local message types • From variable to function: • From function to variable:

10. Canonical update procedure • Graph is cycle-free (a tree) • Leaf nodes send identity messages to parents • When a node receives a message on some edge, it computes and transmits the outgoing message for all other edges • After variable node has received a message from every neighbor, it can compute its exact marginal value:

11. Message Passing

12. Message Passing

13. Message Passing

14. “Loopy” Belief Propagation • Sum-product algorithm is only guaranteed to converge on graphs containing up to one loop • Could oscillate, or fail to converge on cyclic graphs • Empirically, it often converges pretty well on many types of graphs

15. Choose αnmsuch that “Loopy” Belief Propagation • All variable messages initialized to 1 • All messages updated at every iteration • Normalize variable to function message:

16. What is Max-Product? • Instead of computing the marginal functions, we can calculate arg maxxF(x) • Replace function-to-variable message:

17. What is Max-Product? • Now, when we take the product of the messages: • For arg maxxF(x), each component is given by: • A.k.a: Viterbi algorithm

18. What is Max-Sum? • Suppose we have an F(x) that is the sum of function nodes, e.g. • Use max-product in logarithm space

19. Two new local message types • From variable to function: • From function to variable:

20. What is Max-Sum? • Now, when we take the sum of the messages:

21. Advantages • Existing theoretical results • Simple, scalable local algorithm • Exponential only in # of neighbors in graph • Already decentralized and asynchronous • Partition nodes as desired • Only exchange local messages • Can estimate variables before convergence

22. Caveats • No strong theoretical results for loopy BP • Convergence not guaranteed • Must exchange local messages between related factor and variable nodes • Not necessarily easy to evaluate functions and compute messages

23. How to use Max-Sum to solve your distributed problem: Construct utility function

24. How to use Max-Sum to solve your distributed problem: Turn it into factor graph Construct utility function

25. Application #1:Decentralized Coordination Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-08), 12-16 May 2008, Estoril, Portugal.

26. Problem • Solve graph coloring on sensor network • Utility can be formulated for each agent: Penalty for same color: A priori color preference:

27. Factor-graph Formulation • Utility function is only dependent on neighbors • Let each sensor maintain its state and utility nodes: • Exchange local messages as per Max-Sum

28. Comparison Algorithms • BR - Best Response • Each agent chooses the “best” state for its variable (i.e. the one that minimizes the conflicts) according to the current states of its neighbors • When a state change occurs, the agent sends the updated state to all of its neighbors • Represents a lower bound on the performance of any algorithm since it uses the minimum computation and communication possible • DSA - Distributed Stochastic Algorithm • Same as best response, except that an agent only actually performs the state change according to a predefined probability (called the activation probability) • Whenever an agent’s preferred state actually changes, it sends a message to its neighbors informing them of this fact • Set the activation probability to 0.6 (additional experiments indicate little sensitivity to the exact value of this parameter in our setting) • DPOP - Dynamic Programming Optimization Protocol • A complete algorithm that maintains optimality by preprocessing the constraint graph to produce a pseudo-tree, and then performs local message passing on this tree

29. Test Conditions • Known colorable • 3-color random graphs • 10, 20, 30, 40, 50 nodes • Link density of 3 • 50 instances • ADOPT graph repository • Random graphs • 10, 14, 18, 25 nodes • Link density of 3 • Not generally colorable with 3-colors • Lattice graphs • 4-color colorable • 36, 49, 64, 81, 100 nodes • Nodes connected to eight neighbors

30. Conflicts over time

31. Conflicts over time

32. Conflicts over time

33. What happened here? • Combination of small and large irregular loops • Cycling causes poor convergence • Modify utility function:

34. Summary of conflict results

35. Message Size

36. Message Loss

37. Hardware Implementation • http://www.youtube.com/v/T6H1AwQ2gXw • http://www.youtube.com/v/D6vWvs3Lsj0

38. Application #2:Cooperative Control Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on Agent Technology for Sensor Networks, Estoril, Portugal.

39. Problem • Sensor network doing distributed tracking • Sensors share information using decentralized data fusion (DDF)

40. Problem • Select control parameters for each sensor • Maximize sum of global info for all targets

41. Problem • Define U as a summation of target factors Uj over only sensors that can observe target, e.g. where Ub is defined as:

42. ≠ Problem • Unfortunately, we can’t break into smaller factors:

43. Factor-graph Formulation • Each sensor maintains variable node for its own control parameter • Define a function node for each target • Maintained by the sensor that first saw target • Use DDF to determine edges of graph • Add/remove edges as target moves in and out of sensor range • Exchange local messages as per Max-Sum

44. Sensor 1 θ1 UA Sensor 2 Sensor 3 θ2 UB θ2 Sensor 4 Sensor 5 θ4 θ5 UC Factor-graph Formulation

45. Test conditions • Sensors initialized with weak priors and target responsibilities • Performance evaluated using global information from one sensor

46. Global Information

47. Adjusting negotiation time

48. How to use Max-Sum to solve your distributed problem: Turn into factor graph Construct utility function