1 / 12

Redundancy Elimination As A Network-Wide Service

Shuchi Chawla Ashok Anand Chitra Muthukrishnan UW-Madison. Redundancy Elimination As A Network-Wide Service. Srinivasan Seshan Vyas Sekar CMU. Scott Shenker UC-Berkeley. Ram Ramjee MSR-India. Aditya Akella UW-Madison. Growing traffic vs. network performance. Other svcs (backup).

allie
Download Presentation

Redundancy Elimination As A Network-Wide Service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ShuchiChawla Ashok Anand ChitraMuthukrishnanUW-Madison Redundancy Elimination As A Network-Wide Service SrinivasanSeshanVyasSekarCMU Scott Shenker UC-Berkeley Ram RamjeeMSR-India Aditya Akella UW-Madison

  2. Growing traffic vs. network performance Other svcs(backup) Data centers • Network traffic volumes growing rapidly • Annual growth: overall (45%), enterprise (50%), mobile (125%)* • Growing strain on installed capacity everywhere • Core (Asian ISPs – 80-90% core utilization), enterprise access, data center, cellular, wireless… • How to sustain robust network performance? Web content Video Strain on installed link capacities ISP core Enterprises Mobile users Home users * Interview with Cisco CEO, Aug 2007, Network world

  3. Scale link capacities by suppressing duplicates Other svcs(backup) Data centers • A key idea:suppress duplicates • Popular objects, partial content matches, backups, app headers • Effective capacity improves ~ 2X • Many approaches • Application-layer caches • Protocol-independent schemes • Below app-layer • WAN accelerators, de-duplication • Content distribution • CDNs like Akamai, CORAL • Bittorrent • Point solutions  apply to specific link, protocol, or app Web content Video Dedup/archival Wan Opt CDN Wan Opt ISP HTTP cache Dedup/archival Enterprises Mobile users Home users

  4. Universal need to scale capacities Point solutions inadequate Architectural support to address universal need to scale capacities? Implications? • RE: A primitive operation supported inherently in the network • Applies to all links, flows (long/short), apps, unicast/multicast • Transparent network service; optional end-point modifications • How? Implications? Dedup/archival Wan Opt Bittorrent Network RedundancyElimination Service ✗ Point solutions: Other links must re-implement specific RE mechanisms • Point solutions: • Little or no benefit in the core ✗ Point solutions:Only benefit system/app attached Wan Opt Dedup/archival ISP HTTP cache

  5. How? Ideas from WAN optimization 5 • Network must examine byte streams, remove duplicates, reinsert • Building blocks from WAN optimizers: RE agnostic to application, ports or flow semantics • Upstream cache = content table + fingerprint index • Fingerprint index: content-based names for chunks of bytes in payload • Fingerprints computed for content, looked up to identify redundant byte-strings • Downstream cache: content table WAN link Cache Cache Enterprise Data center

  6. From WAN acceleration torouter packet caches Wisconsin Packet cache at every router Router upstream removes redundant bytes Router downstream reconstructs full packet Network RE service: apply protocol-indep RE at the packet-level on network links IP-layer RE service Internet2 (Hop-by-hop works for slow links Alternate approaches to scale to faster links…) Berkeley CMU

  7. Implications overview: Performance and architectural benefits • Improved performance everywhere even if partially enabled • Generalizes point deployments and app-specific approaches • Benefits all network end-points, applications • Crucially, benefits ISPs • Improved switching capacity, responsiveness to sudden overload • Architectural benefits • Enables new protocols and apps • Min-entropy routing, RE-aware traffic engineering (intra- and inter-domain) • Anomaly detection, in-network filtering of unwanted traffic • Simplifies/improves apps: need not worry about using network efficiently • Application control messages & headers can be verbose  better diagnostics • Controlling duplicate transmission in app-layer multicast is a non-issue

  8. Implications example: Performance benefits Wisconsin Generalizes pointdeployments 62 packets Network RE  12 pkts (ignoring tiny packets) Internet2 Benefits ISPs: improve effective switching capacity Without RE  18 pkts 33% lower Berkeley CMU 32 packets 32 packets

  9. Implications example: New protocols Wisconsin 9 • Minimum-entropy routing • New, flexible traffic engineering mechanisms • Inter-domain protocols Simple RE  12 pkts • Internet2 • Redundancy-based anomaly • detectors • ✓ Network-assisted spam filtering • ✓ New content distribution mechanisms • RE + routing • 10 pkts Berkeley CMU

  10. Network RE service: Quantitative results • Analysis of 12 enterprises: traffic 15-60% redundant [SIGMETRICS 09] • ~1GB of cache sufficient to identify redundancies • DRAM or PCM (PRAM) on routers • Network RE benefits both ISPs and end-networks [SIGCOMM 08] • Upto 15-50% better util, responsive TE, control inter-domain traffic impact • Centralized algorithm for min-entropy routing (using “redundancy profiles”) • Reduces utilization by a further 10-25% in intra-domain case • Inter-domain min-entropy routing: gains much more significant (50-80%) • Is network RE viable at high speeds?Not in its current form… • Compression is slow: limits hop-by-hop speed at each hop to 2.5Gbps • Acceptable for access, wireless, cellular links, not for the core • Also, wastes memory on multiple routers  limits effectiveness

  11. SmartRE: Concerted network-wide RE • Toss out link-by-link view; treat RE as a network-wide problem per ISP [Current work] • Memory usage: each packet compressed/un-compressed once • Throughput: allow reconstruction multiple hops away from compression • Stand-alone reconstruction much faster when freed from dependence on compression immediately upstream • Reconstructor can reconstruct a lot more, from multiple different compression agents • Resource-awareness: carefully account for network and device resources, and traffic • Compression/reconstruction/caching locations decided based on memory capacity and memory operations • Also consider global TE objectives • Just 4% from ideal RE (no memory or processing constraints)

  12. Summary and future directions • RE service to scale link capacities everywhere • Architectural niceties and performance benefits • High speed router RE seems feasible • Future directions • End-host participation • Role of different memory technologies – DRAM, flash and PCM • Theoretical issues – pricing and economics, routing policy, network design • Network coding as an alternative to compression

More Related