QCN with Delay-based Congestion Detection for Limited Queue Fluctuation in Data Center Networks

1. QCN with Delay-based Congestion Detection forLimited Queue Fluctuation in Data Center Networks • Y. TanisawaM. Yamamoto • Kansai University, Japan

2. Outline of Presentation • QCN • QCN with large number of flows • Performance Evaluation • Our proposed QCN/DC • Overview • Performance Evaluation • Conclusions

3. QCN（Quantized Congestion Notification) Switch Sending Receiving device device Feedback frame Source dynamics Switch dynamics • Switch calculates feedback value • On receiving a feedback frame, source decreases sending rate when data frame arrives • Detection of congestion by • When source receives no feedback, it increases sending rate feedback value • Switch returns feedback frame to source with a certain probability

4. QCN（Quantized Congestion Notification) Switch Sending Receiving device device Feedback frame Source dynamics(2) Whenever Byte Counter RPsendsframesofBC_LIMIT or (150 KB) Time Counter Timer spends 15 ms is incremented by 1, sending rate is increased

5. Switch sends feedback to maintain queue close to the target queue Calculating feedback Queue How much smallerqueue length is than Qeq The increase of queue Qeq Q Qold When CP receives a data and its Fb is negative, Switch Dynamics Feedback propability CP sends feedback value with a certain probability 10% 1% Reducing the control overhead 0

6. Source Dynamics ・Rate Decrease Whenever a feedback frame is received, Rate Decrease Active Increase Hyper-Active Increase The source reduces sending rate Fast Recovery ・Fast Recovery Rate CR is increased rapidly, if no feedback is received ・Active Increase Rate increase is slower, because Source Dynamics CR is close to the previous rate at which congestion occurred ・Hyper-Active Increase Without congestion detection for long time time, CR and TR is increased rapidly Feedback message received TR（Target Rate） CR（Current Rate）

7. Simulation parameter • Simulation tool：NS2 • RTT：100[us] • # of flows ：10,50,70 • Qeq(Target queue)：22[pkts] • Bandwidth：10[Gbps] • Queue length：100[pkts] • Simulation time：1[s] • 10[us] • 10[us] Sender 1 Receiver 1 Switch1 Switch 2 • 30[us] • 10[Gbps] Sender 2 Receiver 2 QCN Performance with Large# of Flows … … Sender N Receiver N We preliminary evaluate QCN performance in the situation that many flows share a same bottleneck link

8. 60 Target queue 40 Queue length[pkts] 20 0 0.3 0.4 0.3 0.4 0.3 0.4 0.5 0.5 0.5 Time[sec] Time[sec] Time[sec] Queue Length Characteristics (a)10 flows (b)50 flows (c)70flows When the number of flows through a bottleneck link grows, queue length behavior becomes unstable.

9. Feedback received Byte Counter :congestion Time Counter HyperActiveIncrease :non congestion Queuelength[pkts] # of Counter CR[Gbps] Simulation Result in Case of 70 Flows Time[sec] Time[sec] Probabilistic feedback transmission causes no feedback reception Increase of transmission rate queue fluctuation even in a congested time period

10. Aim of the paper QCN suffers queue length fluctuation in the case of large number of flows Another congestion detection is required Loss-based congestion detection Original QCN is loss-less Our proposal QCN + Delay-based Congestion Detection QCN with Delay-based Congestion detection,QCN/DC

11. CRTT: Round trip time of each transmitted frame(measured at sender) TRTT: Threshold for RTT CRTT<TRTT: CRTT≧TRTT: Delay-based Feedback-based • QCN/DC work as original QCN Congestion is detected at each sender by Delay-based control Rate is controlled by fb CRTT < TRTT CRTT ≧ TRTT CRTT < TRTT SR:delay-based transmission rate β:decreased factor CR SR Overview of QCN/DC FB received ①　CR is continuously calculatedby received feedback even when transmission rate Transmission rate control is switched to delay-based one ②　When CRTT become less than TRTT, ② ① transmission rate is switched from SR to CR Time

12. Queue Length Characteristics 60 Target queue 40 QCN Queue length[pkts] 20 0 60 QCN 40 + Queue length[pkts] Delay-based 20 0 0.3 0.3 0.4 0.5 0.4 0.5 0.4 0.3 0.5 Time[sec] Time[sec] Time[sec] (c)70 flows (b)50 flows (a)10 flows • TRTT：130[us] • β：0.99 Some large spikes for queue length are newly observed

13. Cause of Spike CRTT < TRTT CRTT ≧ TRTT 20 FBreceived delay-based control phase CR 60 Byte counter SR Time Counter 40 Hyper-Active Increase Queue length[pkts] # of Counter 10 Target queue Timing control Transmission rate 20 converted 0 0 0.4 0.3 Time[sec] (b)50 flows Time During delay-based control phase delay-based control is switched to QCN control No feedback frame is received at a focus sender, and Byte Counter and Time Counter continuously increase Some spikes are observed After both counters reach 5, HAI starts

14. During delay-based control phase large increase of transmission rate is not reasonable Still in congestion Hyper-Active Increase (HAI) control • CRTT ≧ TRTT Byte Counter＝0 (in delay-based phase) Time Counter＝0 Hyper-ActiveIncrease(HAI)control Byte Counterreaches5 To prevent rapid increase of CR caused by HAI phase and Time Counterreaches5

15. Queue Length Characteristics 60 Target queue 40 QCN Queue length[pkts] 20 0 60 QCN + 40 Queue length[pkts] Delay-based 20 without HAI control 0 60 QCN + 40 Delay-based Queue length[pkts] 20 with HAI control 0 0.3 0.4 0.5 0.3 0.4 0.5 0.3 0.4 0.5 • TRTT：130[us] Time[sec] Time[sec] Time[sec] • β：0.99 (b)50flows (a)10flows (c)70flows

16. Simulation parameter • # of flows ：10,50,70 • Simulation tool：NS2 • RTT：100[us] • Qeq：22[pkts] • Bandwidth：10[Gbps] • TRTT：130[μs] • Queue Length：100[pkts] • Simulation Time：1[s] • β：0.99 0.2s N Sender 1 Receiver 1 Switch1 Switch 2 N-1 N-2 # of flows Sender 2 Receiver 2 Dynamic Situation … … N-3 N-4 Sender N Receiver N 0 1 2 Time[s] We evaluate queue length behavior in the case of a new flow arrival and withdraw of a flow

17. Dynamic Situation 100 QCN Queue length[pkts] 50 0 100 QCN + 50 Delay-based Queue length[pkts] with HAI control 0 1 2 0 0 2 1 1 2 0 Link Utilization Time[sec] Time[sec] Time[sec] (a)10flows (b)50flows (c)70flows Queue behavior of QCN/DC shows undershoot

18. A Cause ofUndershoot CR SR Switched to Feedback-based Rate[Gbps] Queue length[pkts] Fbvalue Fbvalue Bumpy Switching FB received Time[sec] Time[sec] In Delay-based phase When New flow arrives … Feedback is ignored In QCN, queue length temporally grows Transmission rate is gentlydecreased with MD (β=0.99) With feedback reception, transmission rate is rapidly adjusted Bumpy Switching

19. Smooth Switching Smooth Switching (SS) β * SR < CR β * SR > CR CR is too much regulated by FB CR is not adequately by Feedback-based When CC is operated in Delay-based phase, undershoot might happen Feedback-based (original QCN) cannot work well With SS Delay-based Feedback-based

20. Dynamic Situation 100 Queue length[pkts] QCN 50 0 100 QCN + Queue length[pkts] Delay-based 50 without SS 0 100 QCN + Delay-based Queue length[pkts] 50 with SS Link Utilization 0 0.3 0.3 0.5 0.4 0.5 0.4 0.3 0.4 0.5 Time[sec] Time[sec] Time[sec] (c)70flows (b)50flows (a)10flows

21. Conclusions • In QCN, we show that queue length fluctuates with large number of flows in congested link • We reveal that reason for queue fluctuation is HAI increase in some flows receiving no feedbacks even in congestion time period • We propose QCN/DC in which delay-based congestion detection isadditionally used • QCN/DC realizes stable and small queue occupancy with high utilization of bottleneck link Future works Detailed investigation about adaptive adjustment of TRTT is our future work