1 / 19

A Scalable, Commodity Data Center Network Architecture

A Scalable, Commodity Data Center Network Architecture. Mohammad Al-Fares, Alexander Loukissas , Amin Vahdat Presented by Gregory Peaker and Tyler Maclean. Overview. Structure and Properties of a Data Center Desired properties in a DC Architecture Fat tree based solution

callia
Download Presentation

A Scalable, Commodity Data Center Network Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A Scalable, Commodity Data Center Network Architecture Mohammad Al-Fares, Alexander Loukissas, AminVahdat Presented by Gregory Peaker and Tyler Maclean

  2. Overview • Structure and Properties of a Data Center • Desired properties in a DC Architecture • Fat tree based solution • Evaluation of fat tree approach

  3. Common data center topology

  4. Problem With common DC topology • Single point of failure • Over subscription of links higher up in the topology • Typical over subscription is between 2.5:1 and 8:1 • Trade off between cost and provisioning

  5. Properties of solutions • Compatible with Ethernet and TCP/IP • Cost effective • Low power consumption & heat emission • Cheap infrastructure • Commodity hardware • Allows host communication at line speed • Over subscription of 1:1

  6. Cost of maintaining switches

  7. Review of Layer 2 & Layer 3 • Layer 2 • Data Link Layer • Ethernet • MAC address • One spanning tree for entire network • Prevents looping • Ignores alternate paths • Layer 3 • Transport Layer • TCP/IP • Shortest path routing between source and destination • Best-effort delivery

  8. FAT Tree based Solution • Connect end-host together using a fat tree topology • Infrastructure consist of cheap devices • Every port is the same speed • All devices can transmit at line speed if packets are distributed along existing paths • A k-port fat tree can support k3/4 hosts

  9. Fat-Tree Topology

  10. Problems with a vanilla Fat-tree • Layer 3 will only use one of the existing equal cost paths • Packet re-ordering occurs if layer 3 blindly takes advantage of path diversity • Creates overhead at host as TCP must order the packets

  11. FAT-tree Modified • Enforce special addressing scheme in DC • Allows host attached to same switch to route only through switch • Allows inter-pod traffic to stay within pod • unused.PodNumber.switchnumber.Endhost • Use two level look-ups to distribute traffic and maintain packet ordering.

  12. 2 Level look-ups • First level is prefix lookup • Used to route down the topology to endhost • Second level is a suffix lookup • Used to route up towards core • Diffuses and spreads out traffic • Maintains packet ordering by using the same ports for the same endhost

  13. Diffusion Optimizations • Flow classification • Eliminates local congestion • Assign to traffic to ports on a per-flow basis instead of a per-host basis • Flow scheduling • Eliminates global congestion • Prevent long lived flows from sharing the same links • Assign long lived flows to different links

  14. Results: Heat & Power Consumption

  15. Implementation • NetFPGA: • 4 Gigabit Ports, 36 Mb SRAM • 64MB DDR2, 3GB SATA Port • Implemented elements in Click Router Software • Two Level Table • Initialized with preconfigured information • Flow Classifier • Distributes output evenly across local ports • Flow Report + Flow Schedule • Communicates with central schedule

  16. Evaluation • Purpose: measure bisection bandwidth • Fat-Tree: 10 machines connected to 48 port switch • Hierarchical: 8 machines connected to 48 port switch

  17. Results

  18. Related Work • Myrinet – popular for cluster based supercomputers • Benefit: low latency • Cost: proprietary, host responsible for load balancing • Infiniband – used in high-performance computing environments • Benefit: proven to scale and high bandwidth • Cost: imposes its own layer 1-4 protocol • Uses Fat Tree • Many massively parallel computers such as Thinking Machines & SGI use fat-trees

  19. Conclusion • The Good: cost per gigabit, energy per gigabit is going down • The Bad: Datacenters are growing faster than commodity Ethernet devices • Our fat-tree solution • Is better: technically infeasible 27k node cluster using 10 GigE, we do it in $690M • Is faster: equal or faster bandwidth in tests • Increases fault tolerance • Is Cheaper: 20k hosts costs $37M for hierarchical and $8.67M for fat-tree (1 GigE) • KO’s the competing data center’s

More Related