1 / 9

Informed Content Delivery Across Adaptive Overlay Networks

Informed Content Delivery Across Adaptive Overlay Networks.

peterc
Download Presentation

Informed Content Delivery Across Adaptive Overlay Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Informed Content Delivery Across Adaptive Overlay Networks Overlay networks have emerged as a powerful and highly flexible method for delivering content. We study how to optimize throughput of large, multipoint transfers across richly connected overlay networks, focusing on the question of what to put in each transmitted packet. We first make the case for transmitting encoded content in this scenario, arguing for the digital fountain approach which enables end-hosts to efficiently restitute the original content of size n from a subset of any n symbols from a large universe of encoded symbols. Such an approach affords reliability and a substantial degree of application-level flexibility, as it seamlessly tolerates packet loss, connection migration, and parallel transfers. However, since the sets of symbols acquired by peers are likely to overlap substantially, care must be taken to enable them to collaborate effectively.We provide a collection of useful algorithmic tools for efficient estimation, summarization, and approximate reconciliation of sets of symbols between pairs of collaborating peers, all of which keep messaging complexity and computation to a minimum. Through simulations and experiments on a prototype implementation, we demonstrate the performance benefits of our informed content delivery mechanisms and how they complement existing overlay network architectures. Poster 46 John Byers, Jeffrey Considine,Michael Mitzenmacher and Stan Rost ShapeShifter Project

  2. Premises of Our Work • Working to improve content distribution • Current schemes are limited • Topologies limited to trees (servers at roots) • Other overlay links might have residual bandwidth • Plenty of room for improvement • Why suffer through slow download from server? • If last hop is not bottleneck, we can use residual bandwidth. • Can we make use of partial content? (most schemes assume otherwise)

  3. A Motivating Example • Improving a Tree Topology initial multicast tree parallel downloads collaborative transfers • Careful use of additional links can yield significant speedups. • Not Trivial: • Parallel Downloads: Orchestration across servers. • Collaborative Transfers: Need to avoid spurious transmissions.

  4. Digital Fountain to the Rescue Each yj = xj1Å xj2Åxj3 … • Erasure encodings allow packet loss to be handled without feedback. • Memory-less encodings enable unsynchronized downloads from multiple servers. • Large symbol space provides flexibility.

  5. Collaborative Transfers are Much Harder • Need to summarize working sets • Working sets often overlap • Correlation between spatially proximate peers can be very high • Transmitting shared packets is a waste • Must be done concisely since • Space of codewords can be large • Enumerating all codewords is prohibitively expensive • Our contributions • Estimating overlap in 1 packet • Working set in 10 packets • Recoding over received symbols (erasure coding by non-servers)

  6. Peer Assessment • How many symbols does A have that I don’t? • Assessing the amount of shared content of a peer allows evaluation of the peer. • Random sampling • Min-wise summaries (right) • Both fit in one packet! • Benefits of min-wise • Multiple peers can be compared to each other. • Similarity is discarded or utilized for parallel speedups.

  7. Set Reconciliation • Reconciling the differences in content between peers facilitates efficient transfer. • Exact set reconciliation methods are expensive. • We use faster approximate methods. • Bloom filters: efficient summaries for large universes • Approximate reconciliation trees: faster for small differences • Large stretch factor makes this feasible. In this example, if only 10 symbols are necessary, only 2 of the 4 differences need to be reconciled.

  8. Experiments Naïve methods suffer degradation from correlation. Two smart servers are better than four naïve ones. 1 full server and 1 partial server: Multiple partial servers: • Connecting to peers with partial content speeds up transfers with or without servers.

  9. Conclusions • Peers ignored in current distribution schemes are an untapped resource • Allow overlay manager to use almost any link • Concise summaries handle previous problems. • Overlay changes don’t matter • Reconfiguration is expected • More freedom to pick good links • Perfect for ad-hoc wireless • Performance closer to maximum-flow • Often better than any single path

More Related