Student: Shih-Chiang Tsao Advisor: Ying-Dar Lin Date: 2007/12/12

1. Dissertation Fairness Controls for TCP-equivalence at Endpoint and Request-Response Scheduling at Gateway Student: Shih-Chiang Tsao Advisor: Ying-Dar Lin Date: 2007/12/12 Where are bottlenecks? What kind of fairness? How and where to control it?

2. Bottlenecks for the Internet Traffic EDU2 D2 EDU1 Private Fairness S D1 ER ER Intranet ER ISP2 ER H1 ISP1 ER ER R GI GU ER: edge router GI: ISP-side gateway GU:User-side gateway Hi, S, D1, D2: End Points Public Fairness Hn Internet

3. E R1 Data S1 TCP R R E S2 E R2 Streaming ??? Internet Public Fairness Control at End Point Motivation: • New rate control for Streaming Objective: • Smoothness • Fairness with TCP Seq. No. received playing Late packets Because of the oscillatory rate of TCP Time Start-up Latency Classification of Internet Streaming

4. Private Fairness Control at Gateway • Motivation: • Scheduling packets fails to allocate downlink bandwidth at GI • Scheduling requests to manage the responses • Objectives: • Weighted fairness • Shared bandwidth • Full link utilization • Short transmission time the IP addresses of Hi’s are hidden in packets Favorable place by enterprises user-side access gateway ISP-side edge gateway H1 W1 Internet Uplink requests -> GU GI G W2 <- Downlink responses Hn G W3 Queuing packets

5. Related Work and Research Road Map High-speed TCP Utilization of high BW*delay path Bandwidth • HS-TCP [SF03] • FAST [JLH07,TWH05] • XCP [KHR02] • VCP [XSS05] Load Balance On-the-fly TCP Path Selection (Computer Comm.) Fairness Versions of TCP Public Fairness Private Fairness • Vegas [BP95] • Improvement [YCC06] • TCP-friendliness [FF99, BCC98] • TCP throughput eq [PFT98, AAB05] • Internet Conditions [ZDP01][JID04] • Weighted Fairness [PG93]- Class-based [FJ95] TCP-friendly AQM • Survey [CLB04] • WARD [YCC07] TCP-friendly Schemes Packet Scheduling Request Scheduling • web server [PBB98, BBK00, CP99] • web-side gateway [CCC02, CC01, LGC01] - user-side gateway 3. Minimum-service first request scheduling (Submitted to Computer Networks) • WFQ [PG93] • WRR/ DRR [SV96] • SCFQ [GOL94] • SFQ [GVC96] Pre-order DRR (Computer Networks) Evaluation • GAIMD [YL00] • TFRC [FHP00] • TEAR [ROY00] • SQRT, IIAD [BB01] • SIMD [JGM03] 2. WARC (To be submitted to IEEE Trans. on Computer) - Survey [WDM01] - Dynamic Cond. [BBF01] • TFRC’s analysis [VB05] 1. Taxonomy & Evaluation (IEEE Network, Nov. 2007) 802.11e, 802.11s Wireless Protocol DCCP [KHF06]

6. Taxonomy and Evaluation of TCP-friendly Rate-Control Schemeson Fairness, Aggressiveness, and Responsiveness Why do TCP-friendly Schemes have throughput unequal to TCP’s?

7. 3 Criteria for 8 TCP-friendly Schemes Influenced by AQM Taxonomy Evaluation More realistic

8. Fairness PolicyIn steady-statehow a scheme control a flow to use the equivalent bandwidth as a TCP flow? • Update CWND by a set of control parameters • Specific relations between the parameters • GAIMD, SQRT, IIAD, SIMD, AIAD/H • Long-term Fair • E[TWB]=f(p, RTT, a,b,k,l..) • E[TTCP]=f(p, RTT,α=1, β=0.5) • E[TWB]=E[TTCP] Rate-based (RB) Window-based (WB) • Rate=1/(the time between packets) • Estimates the recent TCP throughput during the connection. • Repeatedly adjust the sending rate by the estimation • TFRC,TEAR, TFRCP long-term TCP’s mean rate Cong. Window (CWND) short-term TCP’s mean rate Rate x x x x x x x x x x x x x x Time (s) Time (RTT) packet loss event

9. Aggressiveness Policy How a scheme increases the throughput of a flow before encountering the next loss Slow at beginning and Fast if no loss occurs for long time Tradeoff between aggr. & smoothness Sub-linear Super-linear Tradeoff CWND CWND CWND Aggressive but not smooth Smooth but not aggressive Time Time Time

10. Fixed Variable Tx. Rate Tx. Rate Time Time Loss rate Loss rate Time Time Responsiveness Policy How a a scheme decreases the throughput of a flow when the loss condition becomes severe Tradeoff between resp. & smoothness DiscardOut-of-bound history Tradeoff CWND Smooth but not responsive Responsive but not smooth Time • Historical scheme is adaptive for wider network conditions • Fixed-history scheme have fast responsive behavior

11. D1 S1 100 Mbps20ms on average n TCP senders Dn Sn 2n Mbps10ms R1 R2 D’1 S’1 Drop-Tail n TCP-friendly senders 100 Mbps20ms on average D’n S’n loss loss loss X X Time X Inter-loss 100Mbps2ms 100Mbps2ms 100Mbps30 ms S R1 R2 D Discarding packets by math model Evaluations Dumbbell topology Artificial loss link

12. Fairness Test for TCP-Equivalence:Different Variances of Inter-loss Time Observation 1: Non-periodic losses should be considered in adopting WB/RB fairness policies Different trend Different trend TCP-equivalence as CV[T]=0 Losses in the Internet Losses in the Internet [ZDP01] T: the time between two losses CV[T]: the coefficient-of-variance of T

13. IIAD SIMD GAIMD SQRT TFRCP, TFRC, TEAR Fairness Test for TCP Equal-share: Low-multiplexing Traffic Observation 2: RB fairness policy wins and RTT-heterogeneity matters for TCP equal-share Drop-Tail, N=8 Rate-based fairness policy wins (b) n = 8 n=8 RTT-heterogeneity matters for TCP equal-share

14. Aggr. and Resp. Test For TCP Equal-share Observation 3: Historical/super-linearly aggressive and fixed-history responsive policies are satisfactory (a) Normalized Loss Ratio TFRC TCPSIMD GAIMD SQRT IIAD History/super-linear aggressive policy TEAR Non-history aggressive policy Fixed-history responsiveness policy: fewer losses AIAD/H TFRCP

15. Summary: Strategies in Eight Schemes • TCP-compatibility is not enough • Fairness at steady-state and fast agg/resp at transient-state

16. Summary: Comparison among Schemes O: Satisfactory Δ: Acceptable X: Unacceptable Rate-based fairness policy Historical/super-linear aggressiveness policy Fixed history responsiveness policy

17. A Fast-Converging TCP-Equivalent Window-Averaging Rate Control Scheme Perform better in terms of fairness, smoothness, aggressiveness, and responsivness

18. Window-Averaging Rate Control Major rate control: • Real-time estimation (RTE) control model (Rate-based fairness policy) • Fairness even under non-periodic losses (variant inter-loss time) • Faster aggressiveness Three complemental rate controls: • History-reset (HR) mechanism: (Fixed-history responsiveness policy) • For fast responsiveness • Fluid-based timeout mechanism: • For fairness under heavy-losses • One-RTT reduction procedure: • For FIFO-managed link

19. How to Calculate TCP’s Mean Rate? • WARC adjusts the rate per RTT. • WARC averages the latest s CWNDs of a potential TCP flow. Lower rate under variant inter-loss time TEAR and TFRC: Fixed # of Epoches WARC: Fixed # ofCWNDs CWND CWND Time Time Fixed # of CWNDs Epoch, inter-loss time Real-Time Estimation (RTE) Control Model

20. Fast Aggressiveness WARC TFRC,TEAR -8 -7 -6 -4 -3 -2 0 -5 -1

21. History-reset Mechanism for Fast Responsiveness Ifthen remove CWNDs before the nHRth lastloss from rate computing the 2th last loss the 9th last loss the last loss X(-1) X(-N) X(-2) CWND X(-N+1) R(t,s) Rate (pkt/RTT) s rounds Rounds T T-S(N)

22. Analysis of Fairness [Definition]in the steady state a scheme can control a flow to have the same mean rate as TCP does when both perceived the same network conditions Loss conditions Scheme

23. 1/Aggressiveness(RTTs) IIAD(1,2/3) GAIMD(1/5,1/8) WARC(160) SIMD(1/16) TCP Analysis on Aggressiveness [Definition] [JGM03] 1/Aggr(m)= the time taken by a scheme to increase its rate with a factor of m. rate m 1 ??? time Last loss Fast as the most aggressive scheme

24. Better Tradeoff between Smoothness and Aggressiveness 1080 445 150

25. WARC(160) SIMD(1/16) GAIMD(1/5,1/8) IIAD(1,2/3) 1/Responsiveness(#loss events) WARC Smoothness (CV[w]) Analysis on Responsiveness [Definition][JGM03] 1/Resp(m) =the number of loss events required by a scheme to decrease the rate with a factor of m. more losses for smoothness .

26. Assume X(-j) is an i.i.d. exponential distribution forms a gamma distribution (n, λ) Probability of False-Positive Enabling HR Invoked when the mean of inter-loss time does not change P=10-3 ->False Positive per 1000 losses 35 mins when W=5~30, RTT=50~300ms [JID04]

27. Fairness Test for TCP-Equivalence: Under the Variant-Losses Network WARC:Average fixed # of CWND

28. Fairness Test for TCP Equal-Share 15 Mbps-link TEAR TimeoutMechanism WARC GAIMD SIMD 60 Mbps-link Equal share WARC

29. Fast Aggressiveness & Responsiveness GAIMD TCP WARC w/oOne-RTT reduction TFRC WARC decreases rate with fewer losses TEAR 20sec Fast aggressiveness: WARC and SIMD

30. Smoothness over Different Time Scale SQRT Smoother rate than TCP IIAD GAIMD SIMD TEAR WARC is smooth as TFRC (0.1 sec) Better smoothness

31. Low Start-up Latency for Constrained Streaming (e.g. video conference) TCP WARC late packets WARC TCP WARC has low ratio of late packets

32. Applicability of TCP-equivalent Smooth Rate Controls Kernel-layer Solution (RFC4340, S. Floyd) Layered/Base Protocols APP APP Socket Socket Rate Control TCP DCCP IP IP Datagram CongestionControl Protocol (DCCP) Supported in Linux Kernel User-layer Solution (IETF Draft) APP APP RTP/RTCP Rate Control RTP/RTCP Socket Socket A possible solution in MS Windows UDP UDP LiveMedia Library (LGPL),DirectShow RTP Filter IP IP

33. Summary • WARC • RTE control model + Fixed number of CWNDs • Fairness, Aggressivness, • History-reset mechanism • Responsiveness • TCP-equivalence and TCP equal-share • Fairness under stationary loss condition. • For non-periodic loss conditions • Fast Aggr. & Rspo. for drastic change • Smoothness

34. Problems on Applying Fair Queuing Discipline to Schedule Requests at Access Gateway for Downlink Differential QoS No-monthly fee solution for downlink differential service

35. Where to Schedule Packets? ISP-side gateway User-side gateway • User-side gateway (GU) or ISP-side gateway (GI) ? • GU is bought by the user’s specification and easy to be managed • GI is owned by ISP. Additional charge may requrie. • Packets are not queued at GU • GI cannot see the IPs of H1~Hn • Scheduling uplink requests at GU to managing downlink responses • Class-based Fair Queuing Internet H1 W1 Uplink requests -> GI GU G W2 access link <- Downlink responses Hn G W3 Queuing packets

36. monopolizes the link bandwidth sending one-by-one Responses share the downlink neither is appropriate sending reqs one-by-one sending a request right after getting a response 1. Time to Release the Next ... Request Packet requests S packets S responses monopolize simultaneous

37. Selecting in the order of service-completion time Known packet size Fairness should rely on response size Response size is unknown until it returns 2. From Which Queue to Release the Next .. Request Packet requests Q1 packets ? ? ? Q1 6 2 1 requests S Q2 ? ? ? S Q2 8 4 7 Q3 ? ? ? Q3 9 5 3 response known packet length response size is only available in 1st packet of response

38. 3. User-based Weighted Fairness • Class-based • Between different types of traffic: e.g. voice or ftp • Admission Control • User-level Differentiation • High-class users get more bandwidth than low-class users

39. Minimum-Service First Request Scheduling Internet H1 GI GU Hn Minimum-service order arbiter (MOA) Window-basedrate controller (WRC) Requests Q1 Cq requestselector requestreleaser Q2 requestreceiver Qn Wmax W SC1 w1 w3 U UC1 UCn SCn End of RI End of Rsp. Response Cr Minimum-Service First Request Scheduling (MSF-RS) A is a variable A SC: Service Counter UC: User Counter w: Weight Data flow A changes B A B A is referenced by B A B

40. Minimum-service Order Arbiter (MOA) 2. Select from the class with the min SC Minimum-service order arbiter (MOA) Window-basedrate controller (WRC) Requests Q1 Cq requestselector requestreleaser Q2 requestreceiver Qn Wmax W SC1 w1 wn U UC1 UCn SCn End of RI End of Rsp. Cr Response 1. Log the amount of received service the length of the received response k in bytes

41. Minimum-service order arbiter (MOA) Q1 Cq requestselector Q2 requestreceiver Qn SC1 w1 w3 UC1 UCn SCn Window-based Rate Controller (WRC) Window-basedrate controller (WRC) Requests requestreleaser Release requests if W<W+ W+ W U End of RI End of Rsp. Response Cr W : the number of outstanding requests T : the time interval between two updates Si : the responses in bytes received during T C : the link capacity. K: a constant. Ui: the link utilization. U+: upper bound of U

42. W+=80 W+=40 W+=20 W+=10 TaMSF-RS/Taordinary Analysis of User-perceived Latency Long queuing time of request+ Short transmission time of response = Short user-perceived latency Example 2 0 1 2 1 2 1 2 1 2 1 (4*1+4*2)/8=1.5 Time User-perceived latency Ta=Tq+Ts Tq Ts Client send request Gateway get request Gateway get response Client get response Gateway send request

43. Analysis for Worst-case Fairness of MSF-RS: Fairness Parameter defined by Golestani for analysis of SCFQ Di(t1,t2): the responses receved by Class i in bytes between t1 and t2 wi: the weight of Class i W+ : # of sub-links L+ : Resp. of max. size Qi Normalized Service S Class j Qj Class i Rspi,1 sublink1 0 sublinkW+ t0 t1 t2 Time Rspi,W+

44. Weighted Fairness and Sharing User-based Class-based Bandwidth per user (Kbps) Weighted Fairness Bandwidth Sharing Users in Class 1 Users in Class 1 BW1<BW2 or BW3

45. User-Perceived Latency • Lower congestion • Lower transmission time Higher # of concurrent connections Higher loss rate Client send request Gateway get request Gateway get response Client get response Gateway send request

46. Experimental Results • User-perceived Latency • Percentage on CPU utilization • Lower CPU loading due to fewer concurrent transactions in MSF-RS

47. Summary for MSF-RS • Scheduling uplink requests -> Control Downlink Responses • MSF-RS= Minimum-service Order Arbiter (MOA) + Window-based Rate Control (WRC) • User-based weighted fairness • Bandwidth Sharing among classes • Reduce 20~30% of user-perceived Latency • Reduce 25% of CPU loading • Low congestion • Fewer concurrent transactions

48. Dissertation Conclusions Public Fairness: • Taxonomy and evaluation of 8 TCP-friendly schemes • TCP-equivalence and TCP equal-share • Rate-based fairness + historical/super-linear aggressiveness + fixed history responsiveness • TFRC: if meeting TCP-compatibility is the major concern • SIMD: if fast aggressiveness is favorable • The design of WARC • RTE control -> Non-periodic Fairness, Fast aggressiveness as SIMD • History-reset procedure -> Fast Responsiveness as TFRC • Better Meeting TCP-equivalence and TCP equal-share • Smoothness in short-term for interactive constrained streaming Private Fairness: • The design of MSF-RS • Scheduling Uplink Requests to Manage Downlink Responses • User-based Weighted Fairness • High utilization while reducing 30% of user-perceived latency • Reducing 25% of CPU loading

49. References [FF99] Floyd, S., and Fall, K., “Promoting the Use of End-to-End Congestion Control in the Internet.” IEEE/ACM Transactions on Networking, August 1999. [PFT98] J. Padhye, V. Firoiu, D. Towsley, and J. Kurose, “Modeling TCP throughput: A simple model and its Empirical Validation,” in Proc. of ACM SIGCOMM’98, Sep. 1998, pp. 303-314. [YL00] Y. Yang and S. Lam, “General AIMD Congestion Control,” in Proc. of IEEE ICNP 2000, Nov 2000, pp. 187-198. [FHP00] S. Floyd, M. Handley, J. Padhye, and J. Widmer, “Equation-based Congestion Control for Unicast Applications,” in Proc. of ACM SIGCOMM’00, Aug 2000, pp. 43-56. [ROY00] I. Rhee, V. Ozdemir, and Y. Yi, “TEAR: TCP Emulation at Receivers-Flow Control for Multimedia Streaming,” Tech. Rep., Department of Computer Science, NCSU, Apr. 2000. [JGM03] Shudong Jin, Liang Guo, Ibrahim Matta, and Azer Bestavros, "A Spectrum of TCP-friendly Window-based Congestion Control Algorithms," IEEE/ACMTransactions on Networking, vol. 11, no. 3, pp. 341-355, June 2003. [BB01] D. Bansal and H. Balakrishnan., “Binomial Congestion Control Algorithms,” In Proc.of IEEE INFOCOM, April 2001, pp. 631-640. [PKT99] J. Padhye, J. Kurose, D. Towsley, and R. Koodli., “A Model Based TCP-friendly Rate Control Protocol,” In Proc.of NOSSDAV, June 1999. [BCC98] B. Braden, D. Clark and J. Crowcroft, “Recommendations on Queue Management and Congestion Avoidance in the Internet,” RFC 2309, Apr 1998, Available at http://www.ietf.org. [BBF01] D. Bansal, H. Balakrishnan, S. Floyd, S. Shenker, “Dynamic Behavior of Slowly-responsive Congestion Control Algorithms,” in Proc. of ACM SIGCOMM’01, Aug. 2001, pp. 263-274. [LT03] Lahanas A and Tsaoussidis V, “Exploiting the efficiency and fairness potential of AIMD-based congestion avoidance and control,” Computer Networks, vol 43, no 2, pp. 227-245, Oct. 7 2003. [VB05] Milan Vojnovic and J.-Y. Le Boudec, "On the Long-Run Behavior of Equation-Based Rate Control," IEEE/ACM Transactions on Networking, vol. 13, no 3, pp. 568-581, June 2005. [ZDP01] Y. Zhang, N. Duffield, V. Paxson, and S. Shenker, “On the Constancy of Internet Path Properties, “ in Proc. of ACM SIGCOMM Internet Measurement Workshop, Nov. 2001, pp 197-211. [CLB04] Gwyn Chatranon, Miguel Labrador, and Sujata Banerjee, "Fairness of AQM Schemes for TCP-friendly Traffic" in Proc. of IEEE Globecom, Dallas, Dec. 2004, pp 721-731.

50. [AAB05] E. Altman, K. Avrachenkov, and C. Barakat, “A Stochastic Model of TCP/IP with Stationary Random Losses,” IEEE/ACM Transactions on Networking, vol. 13, no 2, pp. 356-369, April 2005. [NS06] The Network Simulator - ns-2, 2.1b9, http://www.isi.edu/nsnam/ns/. [RHE99] R. Rejaie, M. Handley, and D. Estrin, “Rap: An End-to-end Rate-based Congestion Control Mechanism for Real-time Streams in the Internet,” In Proc. of IEEE INFOCOM, Mar. 1999, pp. 1337-1345. [SW00] D. Sisalem and A. Wolisz, “Lda+ tcp-friendly adaptation: A measurement and comparison study,” in Proc. of NOSSDAV, June 2000. [WDM01] J. Widmer, R. Denda and M. Mauve, “A survey on TCP-friendly congestion control,” Special Issue of the IEEE Network ``Control of Best Effort Traffic”, vol. 15, no 3, pp. 28-37, May/June 2001. [FJ95] Floyd, S., and Jacobson, V., “Link-sharing and Resource Management Models for Packet Networks,”IEEE/ACM Transactions on Networking, Vol. 3 No. 4, pp. 365-386, August 1995. [WTL04] H. Y. Wei, S. C. Tsao, and Y. D. Lin, “Assessing and Improving TCP Rate Shaping Over Edge Gateways,” IEEE Trans. on Computer, vol. 53, issue 3, pp. 259-275, March 2004. [CKD02] M. Conti, M. Kumar, S. K. Das, and B. A. Shirazi, “Quality of Service Issues in Internet Web Services,” IEEE Trans. on Computers, vol. 51, no. 6, June 2002. [PBB98] R. Pandey, J. Fritz Barnes, and R. Fritz Barnes, “Supporting Quality of Service in HTTP Servers,” Proceedings of the Seventeenth Annual ACM Symposium on Principles of Distributed Computing, pp. 247-256, 1998. [BBK00] N. Bhatti, A. Bouch, and A. Kuchinsky, “Integrating User-Perceived Quality into Web Server Design,” Proceedings of the 9th International World Wide Web Conference, 2000. [CP99] L. Cherkasova and P. Phaa, “Session Based Admission Control: a Mechanism for Web QoS,” Proceedings of the International Workshop on Quality of Service, 1999. [CCC02] V. Cardellini, E. Casalicchio, M. Colajanni, and M. Mambelli, “Enhancing a Web-Server Cluster with Quality of Service Mechanisms,” Proceedings of IEEE International Performance Computing and Communications Conference, 2002. [CC01] E. Casalicchio and M. Colajanni, “A Client-Aware Dispatching Algorithm for Web Clusters Providing Multiple Services,” Proceedings of the 10th International World Wide Web Conference, 2001. [LGC01] C. Li, G. Peng, K. Gopalan, and T. Chiuch, “Performance Guarantee for Cluster-Based Internet Services,” State University of Stony Brook, May 2001. [PG93] A. K. Parekh and R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Single-Node case,” IEEE/ACM Trans. on Networking, pp. 344-357, June 1993. [SQI06] Squid Web Proxy Cache, http://www.squid-cache.org/. [SV96] M. Shreedhar and G. Varghese, “Efficient Fair Queuing Using Deficit Round-Robin,” IEEE/ACM transactions on networking vol. 4, no. 3, June 1996. [GOL94] J. Golestani, “A Self-Clocked Fair Queueing Scheme for Broadband Applications,” Proceedings of the IEEE INFOCOM, Toronto, June 1994.