1 / 45

Muhammad Shahbaz Lalith Suresh, Jennifer Rexford, Nick Feamster ,

Enhance cloud communication with source-routed multicast. Scale efficiently to millions of groups without control, CPU, or traffic overhead. Effectively encode and process multicast policies for improved performance.

tshaw
Download Presentation

Muhammad Shahbaz Lalith Suresh, Jennifer Rexford, Nick Feamster ,

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Elmo Muhammad Shahbaz Lalith Suresh, Jennifer Rexford, Nick Feamster, Ori Rottenstreich, and Mukesh Hira

  2. Elmo: Source Routed Multicast for Public Clouds Muhammad Shahbaz Lalith Suresh, Jennifer Rexford, Nick Feamster, Ori Rottenstreich, and Mukesh Hira

  3. 1-to-Many Communication in Cloud

  4. 1-to-Many Communication in Cloud Distributed Programming Frameworks Publish-Subscribe Systems State Replication Infrastructure Applications Streaming Telemetry and more …

  5. 1-to-Many Communication in Cloud 10,000s of tenants 100s of workloads Millions of groups

  6. 1-to-Many Communication in Cloud 10,000s of tenants 100s of workloads Millions of groups Multicast

  7. 1-to-Many Communication in Cloud 10,000s of tenants 100s of workloads Millions of groups Multicast

  8. Limitations of Native Multicast Controller

  9. Limitations of Native Multicast Processing overhead Controller Excessive control churn due to membership and topology changes Limited group entries

  10. Restricted to Unicast-based Alternatives Controller Processing overhead S R R

  11. Restricted to Unicast-based Alternatives Controller Traffic overhead Processing overhead S R R

  12. 1-to-Many Communication in the Cloud Controller S R R

  13. 1-to-Many Communication in the Cloud Controller Traffic overhead Processing overhead S R R

  14. 1-to-Many Communication in the Cloud Processing overhead Controller Excessive control churn due to membership and topology changes Traffic overhead Limited group entries Processing overhead S R R

  15. 1-to-Many Communication in the Cloud Processing overhead Controller Excessive control churn due to membership and topology changes Need a scheme that scales to millions of groups without excessive control, end-host CPU, and traffic overheads! Traffic overhead Limited group entries Processing overhead S R R

  16. Proposal: Source Routed Multicast Controller S R R

  17. Proposal: Source Routed Multicast Controller S R R

  18. Proposal: Source Routed Multicast Little processing overhead Controller Minimal control churn No group entries needed No traffic overhead Negligible processing overhead S R R

  19. A Naïve Source Routed Multicast • For a data center with: • 1000 switches • 48 ports per switch A multicast group encoded as a list of (Switch, Ports) pairs Switch 1: [Ports ] O(30) bytes per switch Switch 2: [.. .. ..] Switch 3: [.. .. ..] Switch 4: [.. .. .x ..] O(30,000)bytes header for a group spanning 1000 switches Switch 5: [.x .. .. ..] 20x the packet size!

  20. Enabling Source Routed Multicast in Public Clouds Key attributes: • Efficiently encode multicast forwarding policy inside packets • Process this encoding at hardware speed in the switches • Execute tenants’ applications without modification

  21. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switch 1: [Ports ] Switch 2: [.. .. ..] Switch 3: [.. .. ..] Switch 4: [.. .. .x ..] Switch 5: [.x .. .. ..]

  22. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Encode switch ports as a bitmap 1 Switch 1: [Bitmap] Switch 2: [.. .. ..] Bitmap is the internal data structure that switches use for replicating packets Switch 3: [.. .. ..] Switch 4: [.. .. .x ..] Switch 5: [.x .. .. ..]

  23. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Group switches into layers 2 Switch 1: [Bitmap] Switch 2: [.. .. ..] Switch 3: [.. .. ..] Switch 4: [.. .. .x ..] Switch 5: [.x .. .. ..]

  24. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Group switches into layers 2 Core Switch 1: [Bitmap] Core Switch 2: [.. .. ..] Spine Spine Switch 3: [.. .. ..] Switch 4: [.. .. .x ..] Leaf Leaf Switch 5: [.x .. .. ..]

  25. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with same ports share a bitmap 3 Switch 1: [Bitmap] Switch 2: [.. .. ..] Switch 3: [.. .. ..] Switch 4: [.. .. .x ..] Switch 5: [.x .. .. ..]

  26. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with same ports share a bitmap 3 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] Spine Switch 4: [.. .. .x ..] Leaf Switch 5: [.x .. .. ..]

  27. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with same ports share a bitmap 3 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] • For a data center with: • 628 switches • 325 bytes header space • Supports 890,000 groups! Spine Modern commodity switches can parse packet headers of 512 bytes No. of groups Switch 4: [.. .. .x ..] Leaf Switch 5: [.x .. .. ..]

  28. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with same ports share a bitmap 3 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] Spine Switch 4: [.. .. .x ..] Leaf Switch 5: [.x .. .. ..]

  29. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with N different ports share a bitmap 4 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] Spine Switch 4: [.. .. .x ..] Leaf Switch 5: [.x .. .. ..]

  30. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with N different ports share a bitmap 4 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] Spine Switch 4,5: [.x .. .x ..] Leaf

  31. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with N different ports share a bitmap 4 Switch 1: [Bitmap] Core • For a data center with: • 628 switches • 325 bytes header space • Supports 980,000 groups! Switch 2,3: [.. .. ..] Spine No. of groups Switch 4,5: [.x .. .x ..] Leaf Difference in ports

  32. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Switches within a layer with N different ports share a bitmap 4 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] Spine Fixed Header Size Switch 4,5: [.x .. .x ..] Leaf

  33. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Use switch entries and a default bitmap for larger groups 5 Switch 1: [Bitmap] Core Switch 2,3: [.. .. ..] Spine Fixed Header Size Switch 4,5: [.x .. .x ..] Leaf Default Bitmap Switch Table Entries

  34. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Use switch entries and a default bitmap for larger groups 5 Switch 1: [Bitmap] Core • For a data center with: • 628 switches • 325 bytes header space Switch entries Switch 2,3: [.. .. ..] Spine Fixed Header Size Switch 4,5: [.x .. .x ..] Leaf Traffic overhead Default Bitmap Switch Table Entries Difference in ports

  35. Encoding a Multicast Policy in Elmo A multicast group encoded as a list of (Switch, Ports) pairs Encode switch ports as a bitmap Group switches into layers Switches within a layer with: - same ports share a bitmap - N different ports share a bitmap Use switch entries and a default bitmap for larger groups 1 2 3 Switch 1: [Bitmap] Core 4 Switch 2,3: [.. .. ..] Spine 5 Fixed Header Size Switch 4,5: [.x .. .x ..] • For a data center with: • 628 switches • 325 bytes header space • Supports a Million groups! Leaf Default Bitmap Switch Table Entries

  36. Processing a Multicast Policy in Elmo 2. Computes the multicast policy 1. API Controller • 3. Installs entries in programmable • virtual switches to push Elmo headers on packets • hardware switches • More flow entries and higher update rates than hardware switches • No changes to the tenant application Virtual Switch S R R

  37. Processing a Multicast Policy in Elmo Controller Switch looks for: Matching bitmap or Table entry or Default bitmap S R R

  38. Applications Run Without Performance Overhead Subscriber Publisher

  39. Conclusion • Designed for multi-tenant data centers • Compactly encodes multicast policy inside packets • Operates at hardware speedusing programmable data planes Elmo Source Routed Multicast for Public Clouds Learn more here: https://elmo-mcast.github.io

  40. Backup Slides

  41. Control Plane Scalability For a multi-rooted Clos topology with 27K hosts and p-rule header of 325 bytes: * min (max) * https://conferences.sigcomm.org/co-next/2013/program/p61.pdf

  42. Elmo Adds Negligible Overheads to Soft. Switches

  43. Elmo Operates within the Header Size Limit of Switch ASICs For a 256-port, 200 mm2 baseline switching ASIC that can parse a 512-byte packet header: 190 bytes for other protocols (e.g., datacenter protocols take about 90 bytes)

  44. Elmo’s Primitives are Inexpensive to Implement in Switch ASICs For a 256-port, 200 mm2 baseline switching ASIC that can parse a 512-byte packet header: As a comparison, Conga consumes 2% of area and Banzai consumes 12% of area.

More Related