FlexPass: Flexible Credit-based Transport for Datacenter Networks

1. FlexPass: A Case for Flexible Credit-based Transport for Datacenter Networks Hwijoon Lim (KAIST), Jaehong Kim (KAIST), Inho Cho (MIT CSAIL), Keon Jang (Rubrik), Wei Bai (Microsoft Research), Dongsu Han (KAIST)

2. 2 Reactive Congestion Control in Datacenter Network • Sender reacts to congestion signal to reduce its sending rate / window • ECN-based (e.g., DCTCP, DCQCN) / Delay-based (e.g., Swift, TIMELY) Switch Sender Receiver p4 p3 p6 p5 p2 p1 Window c1 c2 c3 c4 c5 c6 Reactive ACK Reactive data packet Bandwidth probing is inadequate for DCNs with faster links and low latency requirements.

3. 3 Proactive Congestion Control in Datacenter Network • Proactively allocates bandwidth to each sender (e.g., ExpressPass, HOMA) • Limits queue buildup  Low queueing delay, near-zero packet loss Credit Request Credit Stop Switch Receiver Sender p5 p4 p2 p1 p6 p3 c1 c2 c3 c4 c5 c6 c7 c8 c9 Throttle Credit Rate Proactive Credit Proactive data packet Proactive transport delivers high performance with low queueing delay and high throughput.

4. 4 Deploying new transport protocol • Gradual deployment is a common practice to roll out new transports in production  Environment that heterogeneous transports co-exist Host with legacy reactive congestion control Host with proactive congestion control … Core … Aggr. … ToR 6 6 6 6 6 6 6 6 … Host Cluster

5. 5 Deploying new transport protocol • Proactive transports are incompatible with the heterogeneous environment • Presence of legacy reactive traffic causes fluctuations in available bandwidth for proactive transports Starvation Starvation Naïve deployment of proactive transport causes starvation Gradual deployment in DCNs makes it challenging to deploy proactive transport.

6. 6 Limitations of Existing Proactive Transports • Problem: Proactive transports require the prior knowledge (i.e., link bandwidth) of the entire network, but this is hard to obtain in practice. • Existing Approaches Proactive Proactive 3 2 1 5 3 1 Reactive Reactive 7 6 4 5 7 6 9 4 8 2 9 8 Reactive Proactive Weighted Fair Queueing Link under-utilization Strict Priority Queueing Deprioritizes all legacy traffic Network Separation Operation cost increase Can we enjoy the high-performance of proactive transport without harm on legacy traffic?

7. 7 Limitations of Existing Proactive Transports • Problem: Proactive transports require the prior knowledge (i.e., link bandwidth) of the entire network, but this is hard to obtain in practice. • Existing Approaches FlexPass: Flexible Credit-based Transport for Datacenter Networks Proactive Proactive Reactive Reactive Reactive Proactive Weighted Fair Queueing Link under-utilization Strict Priority Queueing Deprioritizes all legacy traffic Network Separation Operation cost increase Can we enjoy the high-performance of proactive transport without harm on legacy traffic?

8. FlexPass Design

9. 9 FlexPass: Design Rationale • How much bandwidth is available for proactive transports? 10 Gbps? 10 Gbps! Sender Sender Receiver Receiver Without legacy traffic With legacy traffic Credit allocation without considering the legacy traffic may lead to queue buildup.

10. 10 FlexPass: Design Rationale • Queue buildup caused by legacy traffic may slow down proactive transports Switch Buffer Switch Buffer Proactive Proactive Legacy Without legacy traffic With legacy traffic Adopt queue Isolation + Weighted Fair Queueing (WFQ)  WFQ provides minimum guaranteed bandwidth

11. 11 FlexPass: Design Rationale (cont.) • Let proactive transports use up to the minimum guaranteed bandwidth • How can we leverage the spare bandwidth? Split a FlexPass flow into proactive and reactive sub-flows  Opportunistically probe available bandwidth using reactive sub-flow FlexPass Proactive Proactive sub-flow uses up to minimum guaranteed bandwidth Assign APP FlexPass Reactive Reactive sub-flow probes spare bandwidth in an opportunistic manner

12. 12 FlexPass: Design Rationale (cont.) FlexPass Proactive Proactive sub-flow uses up to minimum guaranteed bandwidth Assign APP FlexPass Reactive Reactive sub-flow probes spare bandwidth in an opportunistic manner FlexPass (Reactive) Legacy traffic Legacy traffic Link BW Minimum guaranteed bandwidth FlexPass (Proactive) FlexPass (Proactive) When FlexPass and legacy traffic compete Legacy traffic only FlexPass traffic only

13. 13 FlexPass Design Overview (1) Sender – Splits a flow into proactive and reactive sub-flows FlexPass Proactive Credit FlexPass Sender App. Data FlexPass Proactive FlexPass proactive sub-flow (internally uses ExpressPass) Assign APP FlexPass Reactive FlexPass reactive sub-flow (internally uses DCTCP) FlexPass Reactive Window FlexPass Proactive Data FlexPass Reactive Data FlexPass Proactive Credit

14. 14 FlexPass Design Overview (2) Switch – Use separate queues for FlexPass and legacy traffic for co-existence Switch Tx Queue FlexPass proactive credit Q0 FlexPass proactive sub-flow FlexPass proactive/reactive Q1 FlexPass reactive sub-flow Legacy Traffic Q2 Legacy Traffic (e.g., TCP) Rx Queue FlexPass Proactive Data FlexPass Reactive Data Legacy Data FlexPass Proactive Credit

15. 15 FlexPass Design Overview (3) Receiver – Reassemble packets from two sub-flows FlexPass Receiver FlexPass Proactive App. Data Reassembly APP FlexPass proactive / reactive FlexPass Reactive FlexPass Proactive Data FlexPass Reactive Data FlexPass Proactive Credit

16. 16 FlexPass Design Overview Switch FlexPass Sender FlexPass Receiver Tx Queue credit credit Q0 FlexPass Proactive App. Data FlexPass Proactive Reassembly Assign APP APP FlexPass Reactive FlexPass Reactive Q1 Q2 Rx Queue Legacy Traffic (e.g., TCP) Legacy Traffic FlexPass Proactive Data FlexPass Reactive Data Legacy Data FlexPass Proactive Credit

17. 17 Switch: Why place both sub-flows in the same queue? • Use Separated Queue for FlexPass and Legacy Traffic Candidate 2. Candidate 1. Q0 Proactive Credit Q0 Proactive Credit Q1 FlexPass Proactive sub-flows Q1 FlexPass Proactive sub-flows Q2 FlexPass Reactive sub-flows Q2 FlexPass Reactive / Legacy Q3 Legacy Flows (DCTCP, etc.) • • Legacy traffic may “trap” the reactive sub-flow  Slowdown of FlexPass flows, latency ↑ Out-of-order arrivals  Makes loss recovery challenging

18. 18 Switch: Why place both sub-flows in the same queue? • FlexPass assigns both proactive and reactive sub-flows into the same queue. FlexPass Q0 Proactive Credit Q1 FlexPass Proactive/Reactive sub-flows Q2 Legacy Traffic • No out-of-order arrivals, not affected by the legacy traffic We need to achieve desired bandwidth sharing and queue bound inside Q1.

19. 19 Switch: Keeping FlexPass Queue Short • ECN marking: Gives loss signal to reactive transport to reduce window • Selective dropping: Selectively drop reactive packets inside the switch Reactive Threshold = 3 Switch Egress Queue Packet Ingress Proactive Drop Proactive packet Reactive packet FlexPass effectively prevents the queue buildup on Q1 with ECN and selective dropping.

20. 20 Host: Per-packet State Machine at the Sender • Only leverage proactive sub-flow for the loss recovery Sent as reactive On Reactive window open-up Receive ACK Pending Never been transmitted yet Detect Loss New data segment from application Proactive retransmission Sent as proactive Sent as proactive sub-flow Sent as reactive Sent as reactive sub-flow ACKed Lost Pending Lost Detected packet loss Detect Loss On credit update ACKed Received ACK On credit update Receive ACK Sent as proactive This loss recovery mechanism mitigates congestions when competing with legacy reactive flows.

21. Evaluation

22. 22 Evaluation Setup • Testbed Setup • Implement prototype of FlexPass on BESS (Berkeley Extensible Software Switch) • 9 hosts with 10 Gbps NIC connected to a single switch (Broadcom Tomahawk) • Microbenchmark with long flows / incast with short flows • Simulation Setup • Large-scale simulation with 192 hosts (40 Gbps link) • Evaluate realistic workloads

23. 23 Experiments – Testbed with 10 Gbps link • Set the minimum guaranteed bandwidth to half of the link bandwidth Tail Flow Completion Time (FCT) of 64kB Flows on incast Throughput of ExpressPass and FlexPass flows competing for 10Gbps link DCTCP vs. ExpressPass DCTCP vs. FlexPass FlexPass co-exists well with legacy transport and achieves high performance.

24. 24 Simulation – Gradual deployment • Large-scale simulation of gradual deployment (DCTCP new transport) Naïve △ Naïve deployment of ExpressPass (No queue isolation) Oracle Weighted Fair (Not applicable in practice) – Leverage prior knowledge of the total fraction of ExpressPass traffic ◊ oWF FlexPass ○ FlexPass Flow Completion Time (FCT) during the transition (load=0.5)

25. 25 Simulation – Gradual deployment 99%-ile FCT (Flow Completion Time) (<100kB) Standard deviation of FCT of small flows (<100kB) FlexPass improves the tail FCT while posing minimal harm to legacy traffic.

26. 26 Conclusion • FlexPass is a practical proactive transport designed to co-exist with legacy traffic. • FlexPass is gradually deployable and achieves fair co-existence while preserving the high performance. • https://ina.kaist.ac.kr/projects/flexpass Receiver Sender Application Application FlexPass FlexPass Reassembly Assign Reactive Data Reactive Data DCTCP Proactive Data Proactive Data ExpressPass DCTCP ExpressPass Proactive Credit Rx Queue Q0 Q1 Reactive ACK Reactive ACK Q2 Tx Queue Q0 Q1 Legacy Q2 Reactive Flows