1 / 39

Elmo: Source-Routed Multicast for Public Clouds

Elmo: Source-Routed Multicast for Public Clouds. Muhammad Shahbaz Lalith Suresh, Jen Rexford, Nick Feamster , Ori Rottenstreich , and Mukesh Hira. Cloud Workloads Exhibit 1-to-Many Comm. Patterns. vSwitch. vSwitch. vSwitch. vSwitch. Cloud Workloads Exhibit 1-to-Many Comm. Patterns.

paul2
Download Presentation

Elmo: Source-Routed Multicast for Public Clouds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Elmo: Source-Routed Multicast for Public Clouds Muhammad Shahbaz Lalith Suresh, Jen Rexford, Nick Feamster, Ori Rottenstreich, and Mukesh Hira

  2. Cloud Workloads Exhibit 1-to-Many Comm. Patterns vSwitch vSwitch vSwitch vSwitch

  3. Cloud Workloads Exhibit 1-to-Many Comm. Patterns

  4. Cloud Workloads Exhibit 1-to-Many Comm. Patterns Tenants: … and more

  5. Cloud Workloads Exhibit 1-to-Many Comm. Patterns Distributed Programming Frameworks (e.g., Hadoop and Spark) Publish-Subscribe Systems (e.g., ZeroMQ and RabbitMQ) Replication (e.g., for Databases and state machines) Infrastructure Applications (e.g., VMware NSX and OpenStack) Streaming Telemetry (e.g., Ganglia Monitoring System) and more …

  6. Cloud Workloads Exhibit 1-to-Many Comm. Patterns Multicast vSwitch vSwitch vSwitch vSwitch

  7. Cloud Workloads Exhibit 1-to-Many Comm. Patterns Multicast vSwitch vSwitch vSwitch vSwitch

  8. Approaches to Multicast Good Bad Tenant Isolation Bisection Bandwidth Group Size Control Overhead Traffic Overhead Network Size Endhost Overhead Run at Line Rate No. of Groups IP Multicast SDN-based Multicast with rule aggregation App. Layer Multicast Src-Routed Multicast - SGM - BIER

  9. Approaches to Multicast Good Bad Tenant Isolation Bisection Bandwidth Group Size Control Overhead Traffic Overhead Network Size Endhost Overhead Run at Line Rate No. of Groups IP Multicast SDN-based Multicast with rule aggregation App. Layer Multicast Src-Routed Multicast - SGM - BIER Elmo

  10. Elmo: Source-Routed Multicast for Cloud Services

  11. Elmo: Source-Routed Multicast for Cloud Services vSwitch vSwitch vSwitch vSwitch

  12. Elmo: Source-Routed Multicast for Cloud Services Key challenges: • How to efficiently encode multicast forwarding policy inside packets? • How to process this encoding at line rate?

  13. Exploiting DC Characteristics for Efficient Encoding Core - Symmetric - Short Paths Spine - Co-located Placement Leaf Hypervisor vSwitch vSwitch vSwitch vSwitch Processes: VMs, containers, etc.

  14. Programmable Switches for Line Rate Processing Barefoot Tofino or Cavium XPliant PISCES PISCES PISCES PISCES

  15. Encoding a Multicast Tree in Elmo Key design decisions: • Encoding switch output ports in a bitmap • Encoding on the logical topology • Sharing bitmap across switches • Dealing with limited header space using default p-rules • Reducing traffic overhead using s-rules

  16. Encoding a Multicast Tree in Elmo C0 C1 C3 C2 Multicast Tree S6 S7 S0 S1 S2 S3 S4 S5 L6 L7 L0 L1 L2 L3 L4 L5 PISCES PISCES PISCES PISCES

  17. 1. Encoding Switch Output Ports in a Bitmap C0 C1 C3 C2 1011 1011 1011 1011 Downstream S6 S7 S0 S1 S2 S3 S4 S5 01 11 10 11 01 10 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 1. Bitmap is an internal data structurein switches for replicating packets 2. Switches typically have many ports participating in a multicast tree or none at all PISCES PISCES PISCES PISCES

  18. 1. Encoding Switch Output Ports in a Bitmap C0 C1 C3 C2 1011 1011 1011 1011 Downstream Upstream S6 S7 S0 S1 S2 S3 S4 S5 Bitmap + Flag Bit 01 11 10 11 01 10 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES

  19. 2. Encoding on the Logical Topology Core C0 C1 C3 C2 1011 1011 1011 1011 Spine S6 S7 S0 S1 S2 S3 S4 S5 01 11 10 11 01 10 Leaf L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES

  20. 2. Encoding on the Logical Topology Logical Core C0 C1 C3 C2 1011 P2 P0 P1 P3 Logical Spine S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 Leaf L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES

  21. 3. Sharing Bitmap Across Switches Logical Core C0 C1 C3 C2 1011 P2 P0 P1 P3 Logical Spine S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 Leaf L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 Clustering Algorithm (a greedy version of Min-K-Union) to determine which switches should share a bitmap while minimizing extra transmissions. PISCES PISCES PISCES PISCES 001111 L5, L7 p-rule

  22. 4. Dealing with Limited Header Space using Default p-Rule C0 C1 C3 C2 1011 P2 P0 P1 P3 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES 101111 L0, L6 001111 L5, L7 default p-rule p-rule

  23. 5. Reducing Traffic Overhead using s-Rules C0 C1 C3 C2 1011 P2 P0 P1 P3 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES 101111 L0, L6 001111 L5, L7 Broadcast default p-rule p-rule

  24. 5. Reducing Traffic Overhead using s-Rules C0 C1 C3 C2 1011 P2 P0 P1 P3 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES 001111 L5, L7 s-rule entry each p-rule

  25. Encoding a Multicast Tree in Elmo C0 C1 C3 C2 1011 P2 P0 P1 P3 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 PISCES PISCES PISCES PISCES

  26. Encoding a Multicast Tree in Elmo 1011 C0 C1 C3 C2 1011 P2 P0 P1 P3 P2,P3:11 P0:10 00-1 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 100101-1 L5,L7:001111 L6 L7 L0 L1 L2 L3 L4 L5 L0,L6:101111 001110 101101 001011 100111 Sender PISCES PISCES PISCES PISCES Parameters: 1 p-rule per downstream layer (2 extra transmissions) and no s-rules Default p-rule

  27. Encoding a Multicast Tree in Elmo 1011 C0 C1 C3 C2 1011 P2 P0 P1 P3 P2,P3:11 P0:10 00-1 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 100101-1 L5,L7:001111 L6 L7 L0 L1 L2 L3 L4 L5 L0,L6:101111 001110 101101 001011 100111 Sender PISCES PISCES PISCES PISCES Parameters: 1 p-rule per downstream layer (2 extra transmissions) and 1 s-rule per switch Default p-rule

  28. Encoding a Multicast Tree in Elmo 1011 C0 C1 C3 C2 1011 P2 P0 P1 P3 P2,P3:11 00-1 S6 S7 S0 S1 S2 S3 S4 S5 10 11 01 100101-1 L5,L7:001111 L6 L7 L0 L1 L2 L3 L4 L5 001110 101101 001011 100111 s-rule entry each Sender PISCES PISCES PISCES PISCES Parameters: 1 p-rule per downstream layer (2 extra transmissions) and 1 s-rule per switch

  29. Computes the encoding Encoding a Multicast Tree Controller 1011 C0 C1 C3 C2 P2 P0 P1 P3 P2,P3:11 00-1 S6 S7 S0 S1 S2 S3 S4 S5 100101-1 L5,L7:001111 L6 L7 L0 L1 L2 L3 L4 L5 s-rule entry each PISCES PISCES PISCES vSwitch Elmo Header

  30. Computes the encoding Encoding a Multicast Tree Controller s-rules C0 C1 C3 C2 P2 P0 P1 P3 S6 S7 S0 S1 S2 S3 S4 S5 L6 L7 L0 L1 L2 L3 L4 L5 PISCES PISCES PISCES PISCES flow rules specifying: output ports and the Elmo header to push on the packet 00-1 1011 P2,P3:11 L5,L7:001111 100101-1 Elmo Header

  31. Computes the encoding Encoding a Multicast Tree Controller s-rules C0 C1 C3 C2 P2 P0 P1 P3 S6 S7 S0 S1 S2 S3 S4 S5 L6 L7 L0 L1 L2 L3 L4 L5 PISCES PISCES PISCES PISCES flow rules specifying: output ports and the Elmo header to push on the packet 00-1 1011 P2,P3:11 L5,L7:001111 100101-1 Elmo Header

  32. Evaluation We evaluated three main aspects of Elmo: • Data- and control-plane scalability: Elmo scales to millions of multicast groups with minimal flow-table usage and control overhead on switches. • End-to-end applications: Applications run unmodified and benefits from reduced CPU and bandwidth utilization. • Hardware resource requirements: Elmo is inexpensive to implement in switching ASICs. • Load on hypervisor switches: Elmo imposes negligible overheads at hypervisor switches

  33. Elmo Scales to Millions of Groups For a multi-rooted Clos topology with 27K hosts (having 48 hosts per rack), 1M multicast groups (with groups sizes based on IBM’s WVE trace), and a p-rule header of 325 bytes:

  34. Elmo Scales to Millions of Groups For a multi-rooted Clos topology with 27K hosts (having 48 hosts per rack), 1M multicast groups (with groups sizes based on IBM’s WVE trace), and a p-rule header of 325 bytes: * min (max) * https://conferences.sigcomm.org/co-next/2013/program/p61.pdf

  35. Applications Run Unmodified with No Overhead Subscriber Publisher

  36. Elmo Operates within the Header Size Limit of Switch ASICs 190 bytes for other protocols (e.g., datacenter protocols take about 90 bytes)

  37. Elmo Adds Negligible Overheads to Soft. Switches Separate write call for each p-rule

  38. Conclusion • Designed for multi-tenant data centers • Compactly encodes multicast policy inside packets • Operates at line rate using programmable data planes Elmo A Scalable Multicast Service

  39. Questions? Please visit our project webpage at: https://elmo-mcast.github.io Muhammad Shahbaz http://cs.stanford.edu/~mshahbaz

More Related