1 / 44

Distributed Slicing in Dynamic Systems

Distributed Slicing in Dynamic Systems A. Fern á ndez, V. Gramoli , E. Jiménez, A-M. Kermarrec, M. Raynal. Context. 1. Distributed System Interconnected set of nodes Heterogeneous environment Resources are heterogeneously spread Bandwidth Processing Power Storage Space Uptime …

monifa
Download Presentation

Distributed Slicing in Dynamic Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Slicing in Dynamic Systems A. Fernández, V. Gramoli, E. Jiménez, A-M. Kermarrec, M. Raynal

  2. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Context 1 • Distributed System • Interconnected set of nodes • Heterogeneous environment • Resources are heterogeneously spread • Bandwidth • Processing Power • Storage Space • Uptime • … • Large-scale dynamic environment • Nodes leave and join at any time • Large amount of nodes • No global information 0,3 1,5 98 100

  3. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Problem • What means rich/poor in this context?

  4. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Problem • What means rich/poor in this context? Am I rich or poor? 20

  5. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Problem • What means rich/poor in this context? 6 rich? 2 20 5 1

  6. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Problem • What means rich/poor in this context? 150 poor? 300 20 220 815

  7. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Applications • Remedy the Gnutella problem in Peer-to-peer • Gnutella performance was limited by poor nodes • Kazaa/Skype sollicitate rich nodes rather than poor nodes • Allocating resources/nodes to services • Streaming service needs nodes with highest bandwidth • Non-critical service can run on unstable nodes • File-sharing service requires nodes with many files • …

  8. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Objectives Classifying nodes into categories, slices Based on individual characteristics: attributes A slice corresponds to a portion of the system Typically, answering the question:

  9. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Objectives • Classifying nodes into categories, slices • Based on individual characteristics: attributes • A slice corresponds to a portion of the system • Typically, answering the question: HOW RICH AM I COMPARED TO OTHERS?

  10. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Classifying the system nodes

  11. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Classifying the system nodes 65 71 75 8 27 68 72 20 62 48 89 70 59 …using their attribute values (assume a single attribute for simplicity reason)

  12. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Classifying the system nodes 65 71 75 8 27 68 72 20 62 48 89 70 59 Attribute values ai 0 100

  13. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Classifying the system nodes 65 71 75 8 27 68 72 20 62 48 89 70 59 Attribute values ai 0 100 Normalized Indices pi 1 0

  14. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Classifying the system nodes 65 71 75 8 27 68 72 20 62 48 89 70 59 Attribute values ai 0 100 Normalized Indices pi 1 0 Slices si #1 #2 #3 #4 1 0

  15. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Model Dynamic system System of n nodes Nodes join and leave the system at any time Nodes may crash too Each node i knows its attribute value ai, its position estimate pi’, the slices (ex: 10 equally sized slices, each containing 10% of the nodes), a communication view Vi: constant number of neighbors j, their position estimate pj’, their attribute aj, (and their age)

  16. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Underlying Gossip-based Overlay • Each node i periodically: • Exchanges information with its neighbors (nodes of its view) • Views, values… • Computes its new view and its new state • View, value… • Dynamic overlay: • Failed nodes are naturally removed from views • Joining nodes are naturally added into views • Scalable overlay: • Limited amount of information stored • Limited amount of information exchanged

  17. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Jelasity, Kermarrec 2006 (JK) • Each node i sets pi’ as a uniform random value • Each node i looks for a misplaced neighbor j A neighbor j is misplaced with i iff (ai – aj)(pi’ – pj’) < 0 • Then, nodes i and j exchange pi’ and pj’ • The sequence of position estimates matches the sequence of attribute values For any node couple i and j, ai<aj <=> pi’<pj’ Convergence to this ordering is exponentially fast (in the number of execution)

  18. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering • Similar to JK • Uses Local Disorder Measure LDM • Let lpj’ (resp. lpj) be the normalized index of j random value(resp. attribute value) among all j of Vi  i • LDM(i) = ∑j in Vi  i (lpj’ – lpj)² • Protocol Do periodically { • Update view Vi using an underlying protocol. • Choose the neighbor j that minimizes the LDM(i). • Exchange random values pi’ and pj’ • Update the random value pi’ w/ pj’ if necessary. • Update slice assignment si’ := s : pi’ in s. }

  19. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering • Similar to JK • Uses Local Disorder Measure LDM • Letlpj’(resp.lpj) be the normalized index ofj random value(resp. attribute value) among all j of Vi  i • LDM(i) = ∑j in Vi  i (lpj’ – lpj)² • Protocol Do periodically { • Update view Vi using an underlying protocol. • Choose the neighbor j that minimizes the LDM(i). • Exchange random values pi’ and pj’. • Update the random value pi’ w/ pj’ if necessary. • Update slice assignment si’ := s : pi’ in s. } Difference with JK

  20. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 48 89 70 59 9/11 4/11

  21. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 48 9/11 89 70 59 7/11

  22. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 48 89 70 59

  23. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 2/11 48 7/11 89 70 59

  24. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 48 89 70 59

  25. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 48 4/11 89 70 59 2/11

  26. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 1: Ordering 65 71 75 8 27 68 72 20 62 48 89 70 59

  27. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Result: Slight convergence speed up n = 104 #slices = 10 |V| = 20

  28. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Problem: Wrong Slice Assignment • If random values are not perfectly uniformly distributed • …some nodes might never find their slice • e.g. the 3 nodes of S2 in the example above Normalized indices 1 0 Slices #1 #2 #3 #4

  29. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking • No random values! • Protocol Do periodically { • Update/Shuffle view Vi using an underlying protocol. l += #neighbors with lower attribute value. g += #neighbors • Sends ai to a randomly chosen neighbor • Sends ai to the neighbor that is the closest to a slice boundary • Update slice assignment si’ = s such that l/g in s. } Upon reception { Receive aj from j if (aj < ai) l += 1; g+=1 ; Update slice assignment si’ = s such that l/g in s. }

  30. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking • No random values! • Protocol Do periodically { • Update/Shuffle view Viusing an underlying protocol. • l += #neighbors with lower attribute value. • g += #neighbors • Sends ai to a randomly chosen neighbor • Sends ai to the neighbor that is the closest to a slice boundary • Update slice assignment si’ = s such that l/g in s. } Upon reception { Receive aj from j if (aj < ai) l += 1; g+=1 ; Update slice assignment si’ = s such that l/g in s. } Same number of messages

  31. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 72 20 62 48 89 70 59 4/11

  32. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 72 20 62 68 48 89 70 59 89

  33. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 72 20 62 48 89 70 59 0/2

  34. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 72 20 62 20 72 48 89 70 59

  35. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 72 20 62 48 89 70 59 1/4

  36. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 75 72 20 62 48 48 89 70 59

  37. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Algorithm 2: Ranking 65 71 75 8 27 68 72 20 62 48 89 70 59 1/3

  38. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Result 1: Unlimited convergence n = 104 #slices = 100 |V| = 20

  39. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Result 1: Unlimited convergence n = 104 #slices = 100 |V| = 20 Ranking precision keeps improving

  40. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Result 2: Tolerating Dynamism Churn is correlated with attribute values! e.g. the attribute is the remaining batery lifetime or available storage space. Churn

  41. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Performance Analysis • d, is the distance from pi’ to the closest slice boundary. • For confidence coefficient of 99,99%, the required number of attribute value drawn is mi≥ z pi’ (1 – pi’) / d2, with z <16, a constant.

  42. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Conclusion • Churn-tolerant algorithm Gossip-based mechanisms. Slice belongingness re-approximation. • Scalable algorithm Limited number of neighbors. Size of the system is unknown. • Applications of the distributed slicing Resource allocation Supernodes / Ultrapeers election • Future work Can we obtain convergence speed of the first algorithm with the accuracy of the second algorithm?

  43. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal References • Ordered Slicing of Very Large-Scale Overlay Networks M. Jelasity and A.-M. Kermarrec In Proc. of the 6th IEEE Conference on P2P Computing, 2006. • Randomized Algorithms R. Motwani, P. Raghavan Cambridge University Press, 1995 • Time Bounds for Selection M. Blum, R. Floyd, V. Pratt, R. Rivest, and R. Tarjan Journal Computer and System Sciences 7:448-461, 1972

  44. Fernandez, Gramoli, Jimenez, Kermarrec, Raynal Result 3: Feasibility

More Related