1 / 30

Unstructured overlays: construction, optimization, applications

Unstructured overlays: construction, optimization, applications. Anne-Marie Kermarrec Joint work with Laurent Massoulié and Ayalvadi Ganesh. N nodes in a group. Each node gossips new messages to K other nodes chosen at random .

jaegar
Download Presentation

Unstructured overlays: construction, optimization, applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unstructured overlays: construction, optimization, applications Anne-Marie Kermarrec Joint work with Laurent Massoulié and Ayalvadi Ganesh

  2. N nodes in a group. Each node gossips new messages to K other nodes chosen at random. How large should K be so that ever node receive the message with high probability? Stronger than requiring that nearly 100% get the message with high probability Epidemic protocolsEpidemic multicast 3 4 0 1 9 8 5 2 6 7

  3. Epidemic protocolsPerformance • Modelled as a random graph • Erdos and Renyi result applies to connectivity of undirected graph. • Sharp threshold at log N. • Main results If K= log(N) + c, the probability that every node is reached is exp(−exp(−c)). Result applies if mean out-degree is log(N) + c, irrespective of the degree distribution Use of these results to parameterize protocols

  4. Epidemic protocolsPerformance 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Fanout 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Proportion of atomic multicast Proportion of infection in non atomic multicast

  5. Epidemic protocolsReliability 99.98 99.94 100 90 80 70 60 50 40 30 20 10 0 0% 10% 20% 30% 40% 50% Percentage of faulty nodes Proportion of atomic multicast Proportion of infection in non atomic multicast

  6. Epidemic protocolsResearch issues • Gossip-based algorithms • Scalable: load on each node grows logarithmically with group size • Highly Reliable: • Probabilistic guarantees • Proactive • Graceful degradation in the presence of failures • Major drawbacks • Non-scalable membership protocol • Oblivious to network topology • Generates a large number of messages in non faulty environments

  7. Partial membership Self-organizing membership protocols Decentralized Self-set fanout Topologically awareness Reducing Traffic Epidemic protocolsAgenda SCAMP LOCALISER TREE-BASED APPLICATION-LEVEL MULTICAST

  8. Epidemic protocolsSCAlable Membership Protocol • Partial knowledge: Each node has only a partial knowledge of the membership: local view • Adequate for reliability: O(log(n)) • Self-organizing and fully decentralized: size of local views converges to (c+1) log(N) • Membership management • Graph growth • Graph maintenance

  9. Epidemic protocolsJoin algorithm P=1/sizeof view (1-P) Join request forwarded contact new Join request to a random member

  10. Epidemic protocolsSubscription algorithm 76 Local view 1 4 5 6 4 0 6 7 2 36 6 6 1 2 6 0 8 3 1 7 0 1 56 6 6 5 8 7

  11. Epidemic protocolsAverage case analysis D(n) : Average size of local view with n nodes present. Subscription adds D(n)+1 directed arcs, so (n+1) D(n+1) = n D(n) + D(n)+1 Solution of this recursion is D(N) = D(1) + 1/2 + 1/3 + …+ 1/N  log(N)

  12. Epidemic protocolsGraph maintenance: Redirection • Analysis assumes that new nodes subscribe to a random pre-existing node. • Redirection • Use of weights reflecting the connectivity of the graph • A node receiving a new subscription request may redirect it to a member of its local view . • Subscription request performs random walk on membership until it is eventually kept at some node. • Stopping rule: random walk is close to uniform on all nodes.

  13. Epidemic protocolsPerformance • Convergence of view size • Confirms theoretical analysis • Impact of redirection • Impact of lease • Reliability • Comparison with traditional gossip • Attests to the “good” quality (uniformity) of views

  14. Epidemic protocolsOut-degrees

  15. Epidemic protocolsReliability 1 0.9 0.8 Full membership 0.7 SCAMP Proportion of nodes reached by the multicast 0.6 0.5 0.4 0.3 0.2 0.1 0 0% 10% 20% 30% 40% 50% 60% 70% Percentage of node failures

  16. Unstructured overlays

  17. Loosely structured overlays

  18. Degree balancing in Scamp Mean = 18

  19. Rewiring • Balanced number of neighbours • Topology-aware • Minimize the cost function di: degree of node I (neighbours) c(i,j): cost of transmission i→j (e.g. distance) • Keeping the number • of edges fixed • Local knowledge

  20. Select “open triangle” i—j—k at random; Evaluate locally cost of rewiring to i—k—j : Change to i—k—j with probability Distributed rewiring rule k k j j i i

  21. Experiments • Simulations • GT Topologies • Overlay created by Scamp • Metrics • Mean distance to neighbors • Maximum and distribution of degree • Graph connectivity • Average on 100 simulations

  22. Impact on the degree Scamp-GT Topology, 50,000 Nodes-mean degree = 18

  23. Degree distribution

  24. Impact on the distance to neighbours 50,000 Nodes

  25. Application-level multicast Good quality underlying overlay Tree-based multicast Source initiates the tree building by flooding A node takes as a parent the first node it hears from Small-world optimization Diameter (in hops) Failure resilience

  26. Delay penalty

  27. Relative delay penalty

  28. Tree shape

  29. Node load

  30. Conclusion • Reshaping unstructured into loosely structured overlays: degree balancing and locality • Support for efficient application-level multicast • More work on network load/overhead • Others reshaping metrics

More Related