1 / 48

Deterministic Distributed Resource Discovery

Deterministic Distributed Resource Discovery. Shay Kutten Technion David Peleg Weizmann Inst. Uzi Vishkin Univ. of Maryland & Technion. Distributed Resource Discovery [HLL99]. router. B. C. A. pointer. The corresponding directed graph (on nodes A , B , C ):

octavious-puckett
Download Presentation

Deterministic Distributed Resource Discovery

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deterministic Distributed Resource Discovery Shay Kutten Technion David Peleg Weizmann Inst. Uzi Vishkin Univ. of Maryland & Technion

  2. Distributed Resource Discovery [HLL99] router B C A pointer The corresponding directed graph (on nodes A,B,C): (assumption: weakly connected) A B C Problem: compute the connected component

  3. Distributed Resource Discovery- [Harcol-Balter, Leighton, Lewin 1999] (“Names Dropper” protocol) Licensed to AKAMAI, a leading Web caching provider The resource discovery task: the set of servers changes Find the current set (towards deciding which Akamai server to assign to each user)

  4. Akamai servers need to learn how to reach each other via the Internet(towards deciding which Akamai server is “close” to the user). user CNN Internet AkamaiServer

  5. Think Distributed a pointer used by the algorithm Mr. and Ms. B 132.12.16.64 IP street Interland A? A B a pointer used by the algorithm (one per node, initially pointing at the node itself) A Address “known” to Node A (“knowledge graph”)

  6. When the Algorithm is started C D A B Weakly connected directed graph a pointer used by the algorithm (one per node, initially pointing at the node itself) A Address “known” to Node A (“knowledge graph”)

  7. Algorithm Actions C D A C B • Node A learns C’s addr. From B, • thus, directed edge (A  C) is added to knowledge graph.

  8. Algorithm Actions C D A C B • Node A learns C’s addr. From B, • thus, directed edge (A  C) is added to knowledge graph. (2) Node D who knows of C, change PTR to point at C, thus, pointer graph changes. D

  9. Algorithm Actions C change D A C B • Node A learns C’s addr. From B, • thus, directed edge (A  C) is added to knowledge graph. (2) Node D who knows of C, change PTR to point at C, thus, pointer graph changes. D

  10. Algorithm’s result pointers form star, knowledge graph is strongly connected, Plus, possibly, additional links (possibly clique) .

  11. Model [HLL99] - synchronous (not necessary for our alg) - simultaneously start (for our alg: just “close”) - complete comm. Graph Complexity: #Msgs (connections [HLL99]), #bits (#pointers*log n), time (#rounds)

  12. Motivation (2) Gnutella Napster server (mp3 music files) I have “Madonna” I want “Love Song” I have “Love Song”

  13. Motivation (2) Gnutella Napster server Ask A A I want Love Song I have Love Song

  14. Motivation (2) Gnutella Napster server Ask A A I want Love Song

  15. Motivation (2) Gnutella Napster server Ask A A Love Song

  16. Motivation (2) Gnutella vs. Napster Napster server If the court closes The service collapsed

  17. Motivation (2) Gnutella vs. Napster If the court closes The service collapsed Napster server

  18. Motivation (2) Gnutella vs. Napster If the court closes the service collapses Napster server

  19. Motivation (2) Gnutella vs. Napster Napster server If the court closes the service collapses Gnutella: attempts to solve by having every client=server Same problem: user knows some others find more increase connectivity to withstand disconnections

  20. Motivation(2) Gnutella Bob Users get disconnected often Alice Carol David Initially Alice knows only Bob & Carol (personally, or from ICQ, or from Google…) and can connect via them To others they know.

  21. Motivation(2) gnutella Users get disconnected often Bob Alice Carol David Initially Alice knows only Bob & Carol (personally, or from ICQ, or from Google…) Alice learned from Carol how to reach David, so that when Bob & Carol are not online Alice is still connected.

  22. Motivation: “Peer to Peer” Applications(P2P) • Akamai http://www.akamai.com 1999 • Gnutella http://gnutella.wego.com (originally Nullsoft (Winamp)) 2000 • Gpulp:general Purpose Location Protocol • http://www.gpulp.com • (4) Genny http://www.gnutelladev.com • (5) Freenet http://freenet.sourceforge.net 2000 • (6) JXTA http://www.jxta.org(Sun Microsystems) • April 2001 • (7) Retsina: Discovery of Infrastructure in Multi-Agent Systems (CMU)

  23. deterministic randomized termination detection no detection O(log n) rounds O(log n) O(n log n) messages O(n log n) O(E log n) bits O(n log n) 2 2 2 2 3 0 This paper [HLL99]

  24. D 5 12 E F G H I Algorithm strategies (1) Shrink tall trees to make them stars [SV82] (2) Merge stars to get one tree (changed from [SV82]) (3) Carefully connect weakly connected (explained later; one of the differences from [Shiloach, Vishkin 82])

  25. D A E B F C G H I Example of a technique taken from [SV82] (Handling tall trees in “pointer graph”) Shortcuts [SV82]

  26. Handing tall trees in “pointer graph” (cont.) D A E B F C G H I Shortcuts ([SV82] Each child learns the pointer to its grandparent

  27. D A E B F C G H I Handing tall trees in “pointer graph” (cont.) Shortcuts

  28. Algorithm: finding who to merge with [SV82] D A E,E,F,G,H, I,C E B F C C G Add edges to knowledge graph: H node I, on becoming a child of root D, tells D about C I

  29. join join join 3 Algorithm (2) handling active star roots 5 12 24 Star roots ask to connect

  30. join join join 3 Algorithm (2) handling active star roots 5 12 24 -An undirected subgraph is created by join Messages. -Each root connects to local minima (ID) root On undirected subgraph.

  31. 3 Algorithm (2) handling active star roots 5 12 24 -An undirected subgraph is created by join Messages. -Each root connects to local minima (ID) root On undirected subgraph.

  32. join Algorithm (2) handling active star roots 12 24 7 (note differences from [SV82]) Non-stars do not join 9 20

  33. join 5 12 join join 3 Algorithm (2) handling active star roots Join smallest id- prevents cycles 5 joins 12!!! : smallest id neighbor 24 7 (only stars join, as opposed to [SV82]) 9 20

  34. join join join join 16 3 Algorithm (2) handling active star roots 5 12 24 7 9 20

  35. join join join join 16 3 Algorithm (2) handling active star roots 5 12 24 7 9 20

  36. join join join 16 3 Algorithm (2) handling active star roots 5 12 join 24 7 9 20

  37. join Algorithm handling passive star roots 5 24 9 Node 9 is “passive” since it does not know any Node  it cannot initiate any joining

  38. join join join 3 Algorithm handling passive star roots 8 5 12 Node 12 is passive: It does not know Any non-child. It cannot Initiate any progress. 24 7 9 20

  39. join join join 3 Algorithm handling passive star roots 8 5 12 24 7 Node 12 joins its lowest “suitor” 3. Suitors of 12 join 3 too. 9 20

  40. A Algorithm: An Example of a Techniquedifferent than [SV82](weakly connected directed graph) Without an additional technique, may take (n) time: Worst case (only A is active, its star grows by 1 per phase): Phase 1

  41. A (weakly connected diagraph) Without an additional technique, may take (n) time: Worst case (only A is active, its star grows by 1 per phase): phase 2

  42. A (weakly connected diagraph) Without an additional technique, may take (n) time: Worst case (only A is active, its star grows by 1 per phase): phase n-1

  43. A (weakly connected diagraph) Prevent (n) time, and still do not send too many messages: i Idea: star root A picks 2 neighbors (not just 1) in phase i. (In phaseithere areonly n/2 stars) i Example: i=2 2 =4 i

  44. i Idea: A picks 2 neighbors (not just 1) in phase i. A (weakly connected diagraph) Prevent (n) time: Some delicate points: resolving collisions, mixed topologies prove convergence in O(log n) phases

  45. Correctness 200 3 7 50 No connection  No cycle 30 14 Lemma: No cycles are created (hook on smaller + non-star does not connect) Assured by a new technique: 37 Possible because of Weak connectivity!! Lemma: always some active star

  46. Complexity Two kinds of progress in each phase: • A tall tree gets shallower by a constant factor. (2) Active star roots merge. Obstacles: • # of star trees may grow because of (1) • Trees may get taller because of(2) But (with the right combination) there is a progress in the combination.

  47. Conclusion • improved complexity in all measures (verifies • [HLL99] conjecture, forecasting a simple algorithm • with these complexities). • Deterministic. • Terminates (answers an open problem of Lipton). • Further research • Adaptive algorithms (vs. “one shot”) • Lower bounds? (in [CGK95] O(n) msgs for • undirected case).

More Related