An Integrative Principled Approach to Network Science for Autonomic Networks John S. Baras

1. An Integrative Principled Approach to Network Science for Autonomic Networks John S. Baras Institute for Systems Research University of Maryland 301-405-6606 baras@isr.umd.edu Network Science Workshop August 31-September 1, 2006 Athens, Greece

2. Autonomous Swarms

3. Networks Constrained Coalitional Games Iterative Dynamics on Graphs Trust-Reputation-Profiling Direct and Indirect Trust Computation Component Based Networking Network Design and Trade-offs Outline

4. A collection of nodes, agents, … that collaborate to accomplish actions, gains, … that cannot be accomplished with out such collaboration Most significant concept for autonomous, or autonomic networks What is a Network?

5. The nodes gain from collaborating To collaborate they need to communicate, and this represents cost Trade-off: gain from collaboration vs cost Multiple metrics involved typically Many problems in communication networks, sensor networks, economic networks, social networks, biological networks, can be traced to this key trade off The Fundamental Trade-off

6. What form communications take? How are they represented? How are costs generated? How connectivity is controlled? Does agent behavior influence connectivity? Communication patterns for learning. Connectivity can be physical, or logical (relational) Links-graphs, neighborhoods, MRF, etc Modeling Communication Patterns

7. Almost all functionalities Emergent properties based on local interactions and information Cooperative comms – process overheard info – spatial diversity Cooperation – games - dimensioning Example: Cooperation in MANET

8. Networks Constrained Coalitional Games Iterative Dynamics on Graphs Trust and Collaboration Direct and Indirect Trust Computation Component Based Networking Network Design and Trade-offs Outline

9. Cooperative Game in characteristic function formG= {N, v}, N= {1, 2, …, N}, v :2NR , on all subsets S (coalitions) of N S a coalition, v(S ) is “interpreted ” as the maximum utilityS can get without the cooperation of players in N\S S a coalition, v|S is the restriction of v to the player setS v|S (T ) = v(S ) for each T  S {S , v|S } a subgame of the game {N, v} Gsuperadditive: S, T  N, ST= , v(S T )  v(S ) + v(T ) Gmonotone: S  T implies v(S )  v(T ) Cooperative Games

10. Feasible payoff vectors Efficient payoff vectors Individually rational payoff vectors Imputation set:Set of all individually rational and efficient payoffs Solutions associates with each game G a subset of I*(N, v) Can be characterized either by math relations or axioms Helps capture different notions of “desirable” properties of solutions xdominatesythrough coalitionS (x Sy)if xi > yi, iS, x(S)  v(S) xdominatesy (x y)if x Sy for some coalition S Cooperative Games and Payoffs

11. Gconvex: for each iN, S T, implies di(S )  di(T ) increasingmarginal returns contribution of I Grational: v(N )  iv({i}) Cooperative Games

12. Core (stable, reasonable payoffs): gives each coalition at least as much as could get by itself Convex and average convex games have nonempty cores For a set of games the core is the unique solution that is individually rational, superadditive, nonempty and satisfies the reduced game property Cooperative Games: Solution Concepts • Two interpretations of the core C(N, v) • All imputations such that no group of players has an inventive to split off from the grand coalition N and form a smaller coalition S • No group of players gets more than what they collectively add to the value obtainable by the grand coalition N • C(N, v) is nonempty iff {N, v} is balanced

13. Cooperative Games: Solution Concepts • Stable sets: V I , there is no x, y V s.t. xy, and if yV, there is xV s.t. xy • Nucleolus: excesse(S, x) = v(S ) – x(S ) measure of dissatisfaction of coalition S for payoffx Set (x) = (e(S, x))S N ; solution obtained by min {((x)) | x  I(N, v)}. Minimize maximal complaint. • The Nucleolus is always in the core

14. Nucleolus is the individually rational payoff that lexicographically minimizes the excess vector Leads to iterative procedure for getting there Use a small set of linear programs that iteratively minimize the highest excess, then the second highest excess, etc. A solution concept is the Nucleolus if and only if it is anonymous (ind. of payer labeling), covariant (ind. of scale expressing preferences), satisfies the reduced game property Cooperative Games: Solution Concepts • Shapley Value: solution  with components the expected marginal contribution made by i when entering coalition N • T is a carrier, if v(S ) = v(S T), v(S ) = iSi (v). Shapley Value is the unique solution that has this property, is anonymous and additive • For convex games Shapley Value is in the core • Kernel, Bargaining Set: consider coalition structures, their stability, objections and counterobjections

15. Networks and Constraints All coalitions cannot be formed To coordinate (collaborate) agents need to communicate Network (N, L) Edges – links between payers i and jdirectly connected i and jpath connected Cooperation components Links between players in S , L(S ) Network (S , L(S )) induces a partition of S Cycle Free and Cycle Complete networks Wheels

16. Constrained Coalitions • Network-restricted cooperation game or constrained coalition {N, vL} • {N, v, L} communication situation • Characteristic function • Myerson value : Shapley value of {N, vL} • Component decomposability, component efficiency, fairness

17. Network Formation • Form links pairwise • Iterative game • Better understanding of topologies – dynamics – topology control • Network formation with costs for establishing links • {N, v, L, c} {N, vL,c} • Stability vs efficiency of the resulting network • Small world graphs

18. Networks Constrained Coalitional Games Iterative Dynamics on Graphs Trust and Collaboration Direct and Indirect Trust Computation Component Based Networking Network Design and Trade-offs Outline

19. Example: Trust Management System Prior trust relations Trust Decision Trust Credential Credential Distribution Evaluation Policy Local observations Local key exchanges Applications

20. The network is modeled as a directed graph G(V,E) G is the trust graph A directed link from node i to node j corresponds to the trust relation i has on j The weight cij represents the opinion of i on j, Trust evaluation is to estimate the trustworthiness of nodes ti represents node i being either GOOD or BAD, denoted as ti=1 or -1 si is the estimated trust value of node i si is a subjective concept, while ti is an existing but unknown fact Objective: to drive si as close to ti as possible based on available Jij Trust Evaluation in Autonomic Networks

21. In homogenous networks, the trustworthiness of an agent is based on other peers’ opinion The most straightforward scheme is to ask neighbors to “vote” for it Values of the votes are equal to cij Iterative voting rule: Evaluation starts from a small set of trusted nodes Our interest is to study evolution of the estimated trust value si and its property at the equilibrium Local Voting Rule

22. Trust revocation: Changes in topology, membership, secure paths Referees of a node may change, trust evidence for a node may change Votes timeout or negative votes Trust spreads Trust-connected network Trust Dynamics • Trust spreading Initial “islands” of trusts

23. We use the weighted average as the voting rule, where weights are ‘vote values’ (all quantities nonnegative) is the degree of node i n represents discrete time Assume is a constant, i.e. it doesn’t change with time, which is true when considering the steady state The voting rule can be written in system equation Deterministic Voting Rule

24. We introduce the notion of headers Headers are pre-trusted agents and only vote for nodes that they fully trust. If node i is trusted with bi headers, it gets bi more votes with value 1. Let B = diag[b1 , b2 ,…, bN ]. The system equation changes to Convergence Theorem: Given a virtuous network, in order to have a trust connected graph, the number of headers of each node must satisfy This theorem proves, as well as provides, a network design method to establish a fully trusted network by introducing headers Voting with Headers

25. Stochastic threshold rule with uncertainty parameter b: Where Update sequence – random asynchronous updates Difficult to achieve synchronicity in autonomic networks The probability that node i is chosen as the target at each iteration is fixed as qi Stochastic Threshold Rule Zi(k) is the normalization factor

26. The steady state can be derived using the Markov chain If and , the voting rule converges to the steady state with a unique stationary distribution The unique stationary distribution is where and Z is the normalization function Criterion: probability of correct estimation Convergence

27. Trust in Virtuous Networks All nodes are good and have full confidence in their neighbors. We study Pcorrect at steady state. Left figure: The threshold should be less or equal to 0, otherwise the trust estimate of each node converges to -1. Right figure: When threshold is equal to 0 -- phase transition. Small change on the parameter results in opposite performance of the voting rule.

28. All nodes are good, but because of uncertainty and incompleteness, Jij’s are random variables Assume Assume that the probability of a good node having an incorrect opinion on its neighbors is pe Virtuous Networks with Uncertainty • Simulation results • When peis larger, the system more probably stays in the random phase. • When pe is large enough (pe > 0.15), the system always stays in the random phase. • Theoretical analysis: replica method in spin glasses

29. Random Graph (Erdös and Rényi, 1960) Nodes link to each other randomly Small-world model (Watts & Strogatz,1998) Short average distance (six degree of separation) Large clustering coefficient Scale-free model (Barabási & Albert, 1999) The distribution of degrees follows the power law Existence of hubs Rich get richer Recent research discovered lots of complex networks being scale-free Network Topology

30. The time for updating rule to reach steady state, i.e., how fast the trust values converge. Perron-Frobenius Theorem in algebraic graph theory: For a stochastic matrix A isthe largest eigenvalue of A, which is 1 and isthe second largest eigenvalue of A. The convergence rate of An is of order Normalized adjacency matrices are stochastic matrices, therefore those with smaller converge faster. What kind of networks or which network topology has smaller second largest eigenvalue Spreading Speed and Topology

31. Adding just 1% more edges, spreading finishes in 10 times less rounds. Network Topology and Deterministic Voting Rule • We consider the Φsmall-world model proposed by Watts and Strogatz • High clustering coefficient and small average graphical distance between any pair. • We use Φ-model, which is modeled by adding small number of new edges into a regular lattice.

32. B Small-world model: Prw represents “short cuts fraction” on a regular lattice Regular lattice: Prw=0; Random graph: Prw=1 Prw in [0.1,0.01] is the area for the small world model Network Topology and Stochastic Voting Rule • The performance of the voting rule increases as Prw increases. • A more random graph has shorter average distance • Accuracy of trust information degenerates over the path length, so a short spreading path has more accurate information and leads to good result

33. Networks Constrained Coalitional Games Iterative Dynamics on Graphs Trust and Collaboration Direct and Indirect Trust Computation Component Based Networking Network Design and Trade-offs Outline

34. Statistical Physics models for magnetization Ising and Spin Glass Models • Orientation of each particle’s spin depends on its neighbors • Ising Model: behavior of simple magnets • Spin Glass Model: complex materials • Math interpretation: • s = {s1, s2,…, sn} is a configurationof n particle spins, where sj = 1 or -1 , spin j is up or down • Energy for configuration s • Ising Model: Jij = J for all i, j • Spin Glass Model: Jij depend on i,j and can be random

35. Ising/SG Models and Games • Ising/SG models can be interpreted as dynamic (repeated) games: • The value of si represents whether node i is willing to cooperate or not • each particle selects spin to maximize its own payoff • Ising model (Jij = J>0) : align its spin with the majority of neighbors spin • High T, conservative agents, not willing to change, small payoffs • Low T, aggressive agents, larger payoffs • Collection of local decisions reduces the total energy of the interacting particles • Inspires an approach where trust is an incentive for cooperation • Jij can be interpreted as the worth of player j to player i • decide to cooperate or not based on benefit from cooperation and trust values of neighbors

36. Statistical Mechanics of Spin Glasses • Statistical Mechanics primary object of interest • Recent excitement: computation of ground state, partition functionZ, NP - complete, Replica Method • Application and extensions to several well known problems: turbocodes, image restoration, neural networks, learning, associative memory, SAT, knapsack, SA, number parttioning, graph partitioning, CDMA, MIMO, …

37. 2 6 J21 J12 3 Subset S={1,2,3,4} v(S)=J12+J21+J14+J41+J43+J34 -J36 -J15 1 5 J34 J14 J41 4 J43 Spin Glass Cooperative Game • Spin glass model as a cooperative game (spin glass game) • SN= {1, 2, …, n} is a coalition, in which all nodes cooperate • Interaction topology (Jij’s) moderates effects pos. and neg. feedback • v(S) : value of the characteristic function of the game , v: 2NR, which is the maximum payoff S can get without cooperation from other nodes N/S. • The cooperative game is denoted as Γ =(N, v) • Object: to find what form or policy for Jij can induce all (or most) nodes to cooperate: maximize the coalition

38. In autonomic networks Cooperation is restricted to only local interactions Decision is made by each node individually Nodes are self-interested Explain and analyze emergent properties Game theoretic methods Provide a framework for modeling individual interactions Understand complex global structures and dynamics of a system composed of a large number of agents with simple local interactions Guide for analytical approach Examples: Ising – spin glass models, prisoner’s dilemma Goal: how to encourage nodes to collaborate in games? Incentive: trust systems to promote cooperation and circumvent misbehaving nodes. Cooperation and Games

39. Trust is an incentive for collaboration Nodes who refrain from cooperation get lower trust values They will be eventually penalized because other nodes tend to only cooperate with highly trusted ones. Assume, for node i, that the loss for not cooperating with node j is a nondecreasing function of Jji as f (Jji), and the new characteristic function is Theorem : if , the core is nonempty and is a feasible payoff allocation in the core. By introducing a trust mechanism, all nodes are induced to collaborate without any negotiation Trust as Mechanism to Induce Collaboration – Profiling--Reputation

40. Dynamic Coalition Formation • System model • Two linked dynamics • Trust propagation • Game evolution • Stability of dynamic coalition • Nash equilibrium: no node will gain if it changes its current strategy, while others keep unchanged.

41. Theorem: , there exists τ0, such that for a reestablishing period τ > τ0 The iterated game converges to Nash equilibrium; In the Nash equilibrium, all nodes cooperate with all their neighbors. Comparison of games with (without) trust mechanism, strategy update: Results of Game Evolution Percentage of cooperating pairs vs negative links Average payoffs vs negative links

42. Networks Constrained Coalitional Games Iterative Dynamics on Graphs Trust and Collaboration Direct and Indirect Trust Computation Component Based Networking Network Design and Trade-offs Outline

43. 1 C C 2 D C … … … n D C n+1 ?? ?? Example: Direct Network Trust • Direct trust is based on past interactions between User A and User B. • It is A’s belief about B’s future behavior. • Helps A decide for himself and based on local information what to do next. A B

44. Example: Indirect Network Trust User 8 asks for access to User 1’s files.User 1 and User 8 have no previous interaction What should User 1 do? 2 1 7 Use transitivity of trust (i.e. use references) 4 6 3 5 8

45. User i is of type ti{Good, Bad} chooses action ai{C,D}, i=1…N receives payoff Ri=R(ai,a(i),ti) wants to maximize his own payoff (local behavior) Direct Trust

46. Questions we are investigating How can collaboration of Good nodes be achieved? Maximization of the Good node payoff How quickly can it be achieved? Repeated interactions How many bad nodes can destroy it? Within our framework, the following parameters affect the answers to the above questions. Payoffs Strategies Topology Direct Trust

47. Prior probability (reputation, profiling) for user types Bayes-Nash equilibrium Strategy for User i : Direct Trust evolving reputation

48. Two sequences evolving with time: Vector of actions (strategies), time 1:n Set of vectors of neighbor probabilities (reputations), time 1:n Direct Trust

49. Where is trust in all this? Remember:“Direct trust is based on past interactions between User A and User B.It is A’s belief about B’s future behavior.Helps A decide what to do next.” Trust is how users use the history of past actions to decide what to do next. Quantified with updated probabilities (reputations) pi. Example: Direct Trust -- Learning

50.  is used to combine edge weights along a path:  is used to combine path weights: ab a b Semirings-Definitions a ab b