Criticality-based Analysis and Design of Unstructured P2P Networks as “ Complex Systems ”

1. Farnoush Banaei-Kashani and Cyrus Shahabi Criticality-based Analysis and Design of Unstructured P2P Networks as“Complex Systems” Mohammad Al-Rifai

2. Outline • Introduction • Motivation • Flooding search • Probabilistic Flooding • Percolation Theory • TTL selection policy • Summary • Questions Mohammad Al Rifai

3. Introduction • Motivation - improving scalability of flooding search applied in unstructured P2P networks (Gnutella) • Proposed approach - recognizing P2P networks as Complex Systems, and exploiting the accurate statistical models used to characterize them for formal analysis and efficient design of P2P networks. Mohammad Al Rifai

4. Introduction • Flooding search • Each query is flooded through the entire network Mohammad Al Rifai

5. Introduction • Flooding search • Each query is flooded through the entire network • Algorithm: • a node initiates a query, • sets TTL value, • sends the query to all of • its neighbors. Mohammad Al Rifai

6. Introduction • Flooding search • Each query is flooded through the entire network • Algorithm: • a node initiates a query, • sets TTL value, • sends the query to all of • its neighbors. • each receiver of the query • decrements TTL by one, • forwards the query to its • neighbors in turn, and so on Mohammad Al Rifai

7. Introduction • Flooding search • Each query is flooded through the entire network • Algorithm: • a node initiates a query, • sets TTL value, • sends the query to all of • its neighbors. • each receiver of the query • decrements TTL by one, • forwards the query to its • neighbors in turn, and so on • the flooding continues till the object is found. Mohammad Al Rifai

8. Introduction • Flooding search • Each query is flooded through the entire network • Algorithm: • a node initiates a query, • sets TTL value, • sends the query to all of • its neighbors. • each receiver of the query • decrements TTL by one, • forwards the query to its • neighbors in turn, and so on • the flooding continues till the object is found. Mohammad Al Rifai

9. Introduction • Flooding search • Problems: • extra overhead through • duplicated queries • initial TTL is set regardless of • the size of the network Mohammad Al Rifai

10. Introduction • Flooding search • Problems: • extra overhead through • duplicated queries • initial TTL is set regardless of • the size of the network • does not scale Mohammad Al Rifai

11. Introduction • Flooding search • Problems: • extra overhead through • duplicated queries • initial TTL is set regardless of • the size of the network • does not scale • Proposed solutions: • 1- Probabilistic flooding search • 2- TTL self selection policy Mohammad Al Rifai

12. I- Probabilistic Flooding • Each node forwards the query to its neighbors with probability p, and drops the query with probability • (1 –p). • The normal flooding search is an extreme case of probabilistic flooding with p=1. Mohammad Al Rifai

13. I- Probabilistic Flooding • Each node forwards the query to its neighbors with probability p, and drops the query with probability • (1 –p). • The normal flooding search is an extreme case of probabilistic flooding with p=1. • By decreasing the value of p, the probabilistic flooding cuts some paths • (not only redundant ones). Mohammad Al Rifai

14. I- Probabilistic Flooding • Each node forwards the query to its neighbors with probability p, and drops the query with probability • (1 –p). • The normal flooding search is an extreme case of probabilistic flooding with p=1. • By decreasing the value of p, the probabilistic flooding cuts some paths • (not only redundant ones). Mohammad Al Rifai

15. I- Probabilistic Flooding • Each node forwards the query to its neighbors with probability p, and drops the query with probability • (1 –p). • The normal flooding search is an extreme case of probabilistic flooding with p=1. • decreasing the value of p furthermore towards 0, cuts more and more paths, and turns out law reachability, thus an inefficient search. Mohammad Al Rifai

16. I- Probabilistic Flooding • Each node forwards the query to its neighbors with probability p, and drops the query with probability • (1 –p). • The normal flooding search is an extreme case of probabilistic flooding with p=1. • decreasing the value of p furthermore towards 0, cuts more and more paths, and turns out law reachability, thus an inefficient search. Mohammad Al Rifai

17. I- Probabilistic Flooding • Goal: • all redundant paths must be cut effectively to eliminate duplicated queries and avoid the overhead cost, while full reachability must be preserved. • How? • pmust be tuned to an optimal (critical) operating point pc. • to achieve that, the system must be formally modeled. Mohammad Al Rifai

18. I- Probabilistic Flooding • Formalizing and modeling the P2P networks • unstructured P2P networksare large-scale, dynamic, and self-configure systems, which are the main characteristics of Complex Systems. • Hence, P2P networks can be recognized as Complex Systems, and theoretical and statistical models applied on Complex Systems can be exploited with P2P networks. • Percolation Theory isone of the most important theories applied on Complex Systems that can help to find the critical value pc. Mohammad Al Rifai

19. I- Probabilistic Flooding – Percolation Theory Given a 2D lattice of some sites (dots)and bonds (lines) connecting neighboring sites as shown Mohammad Al Rifai

20. I- Probabilistic Flooding – Percolation Theory Given a 2D lattice of some sites (dots)and bonds (lines) connecting neighboring sites as shown (in terms of P2P networks, sites are nodes and bonds are links between them) Mohammad Al Rifai

21. I- Probabilistic Flooding – Percolation Theory Given a 2D lattice of some sites (dots)and bonds (lines) connecting neighboring sites as shown (in terms of P2P networks, sites are nodes and bonds are links between them) Assuming that each bond can be open with probability p, or closed with probability (1 –p). depending on p, some clusters (sites connected by open bonds) starts to appear. Mohammad Al Rifai

22. I- Probabilistic Flooding – Percolation Theory Given a 2D lattice of some sites (dots)and bonds (lines) connecting neighboring sites as shown. (in terms of P2P networks, sites are nodes and bonds are links between them) Assuming that each bond can be open with probability p, or closed with probability (1 –p). The larger the value of p, the larger the size of clusters is. Mohammad Al Rifai

23. Giant cluster I- Probabilistic Flooding – Percolation Theory Given a 2D lattice of some sites (dots)and bonds (lines) connecting neighboring sites as shown. (in terms of P2P networks, sites are nodes and bonds are links between them) Assuming that each bond can be open with probability p, or closed with probability (1 –p). Due to Percolation Theory: above a threshold probability pc, a giant cluster spanning the whole lattice starts to appear. Mohammad Al Rifai

24. I- Probabilistic Flooding – Percolation Theory • Unstructured P2P networks are random graphs of size N ∞, • with connectivity distribution P(k). • nodes and links between them may be thought of as sites and • bonds respectively in terms of Percolation Theory. Mohammad Al Rifai

25. I- Probabilistic Flooding – Percolation Theory • Unstructured P2P networks are random graphs of size N ∞, • with connectivity distribution P(k). • nodes and links between them may be thought of as sites and • bonds respectively in terms of Percolation Theory. Percolation Theory verifies that once probabilistic flooding is applied, above a threshold pc the giant cluster spans the whole network with minimum connectivity. Mohammad Al Rifai

26. I- Probabilistic Flooding – Percolation Theory • Unstructured P2P networks are random graphs of size N ∞, • with connectivity distribution P(k). • nodes and links between them may be thought of as sites and • bonds respectively in terms of Percolation Theory. Percolation Theory verifies that once probabilistic flooding is applied, above a threshold pc the giant cluster spans the whole network with minimum connectivity. How couldpcbe computed ? Mohammad Al Rifai

27. j i I- Probabilistic Flooding Analysis: The following assumption has been made: “percolation threshold takes place when each node i connected to a node j in the spanning cluster, is also connected to at least one other node” Mohammad Al Rifai

28. I- Probabilistic Flooding Analysis: this criterion can be written as follows: Mohammad Al Rifai

29. I- Probabilistic Flooding ki: the degree of node i Analysis: this criterion can be written as follows: Expected value of ki Mohammad Al Rifai

30. I- Probabilistic Flooding Analysis: this criterion can be written as follows: Mohammad Al Rifai

31. I- Probabilistic Flooding Conditional probability of a node i having ki degree, given that it is connected to j Analysis: this criterion can be written as follows: Mohammad Al Rifai

32. But due to Bayes rule, I- Probabilistic Flooding Analysis: this criterion can be written as follows: Mohammad Al Rifai

33. But due to Bayes rule, I- Probabilistic Flooding Analysis: this criterion can be written as follows: where, Mohammad Al Rifai

34. But due to Bayes rule, I- Probabilistic Flooding Analysis: this criterion can be written as follows: where, N : total number of nodes Mohammad Al Rifai

35. But due to Bayes rule, I- Probabilistic Flooding Thus, at criticality: Analysis: this criterion can be written as follows: where, Mohammad Al Rifai

36. Given the connectivity distribution of the network using probability flooding results in the effective connectivity distribution as follows: I- Probabilistic Flooding Analysis: Mohammad Al Rifai

37. Given the connectivity distribution of the network using probability flooding results in the effective connectivity distribution as follows: (2) I- Probabilistic Flooding Analysis: Mohammad Al Rifai

38. Given the connectivity distribution of the network using probability flooding results in the effective connectivity distribution as follows: (2) I- Probabilistic Flooding Analysis: Mohammad Al Rifai

39. Given the connectivity distribution of the network using probability flooding results in the effective connectivity distribution as follows: (2) I- Probabilistic Flooding Analysis: Mohammad Al Rifai

40. …(3) I- Probabilistic Flooding Analysis: Mohammad Al Rifai

41. …(4) I- Probabilistic Flooding Analysis: Mohammad Al Rifai

42. I- Probabilistic Flooding Analysis: from (3) and (4) the ratio of the second to first moment is: …(5) is the ratio of the second to first moment of the actual graph. Mohammad Al Rifai

43. from (3) and (4) the ratio of the second to first moment is: Exponential cutoff factor required for representing real-world networks I- Probabilistic Flooding Power-law exponent Analysis: …(5) C is a normalization factor is the ratio of the second to first moment of the actual graph. Gnutella network follows power-law connectivity distribution (6) Mohammad Al Rifai

44. I- Probabilistic Flooding Analysis: the ratio α is computed from equation (6), (7) Hence, pc is a factor of cutoff-index v andτ Mohammad Al Rifai

45. I- Probabilistic Flooding Liτ(x): τ-th Ploylogarithm of x Analysis: the ratio α is computed from equation (6), (7) Hence, pc is a factor of cutoff-index v andτ Mohammad Al Rifai

46. I- Probabilistic Flooding Analysis: the ratio α is computed from equation (6), (7) Hence, pc is a factor of cutoff-index v andτ For Gnutella, the power-law exponent is estimated as low as 1.4 and as high as 2.3 in different times, and v is in the range of 100 to 1000. Mohammad Al Rifai

47. I- Probabilistic Flooding 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 pc Power-law Exponent τ = 2.3 Power-law Exponent τ= 1.4 Critical probability can be less than 0.01 Hence, flooding cost is reduced by more than 99% without losing reachability v 100 200 300 400 500 600 700 800 900 1000 Cut-off index v Mohammad Al Rifai

48. I- Probabilistic Flooding 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0.01 i.e. scalable search pc Power-law Exponent τ = 2.3 Power-law Exponent τ= 1.4 Critical probability can be less than 0.01 Hence, flooding cost is reduced by more than 99% without losing reachability v 100 200 300 400 500 600 700 800 900 1000 Cut-off index v Mohammad Al Rifai

49. i.e. not scalable II- TTL selection policy Problem: in normal flooding search TTL is restricted to the initial value set by the search originator regardless of the actual size of the network. Mohammad Al Rifai

50. Problem: in normal flooding search TTL is restricted to the initial value set by the search originator regardless of the actual size of the network. i.e. not scalable II- TTL selection policy Solution: selection policy is based on the typical length λ of the shortest path between two randomly chosen nodes on any random graph, which is provided by Newman as follows: Mohammad Al Rifai