Quality of Service Support in Packet Networks

1. Quality of Service Supportin Packet Networks Tiziana Ferrari ferrari@cnaf.infn.it Italian National Institute for Nuclear Physics INFN - CNAF Tiziana Ferrari Quality of Service Support in Packet Networks

2. Overview • Problem statement • technical solutions: • ATM • RSVP and RSVP to ATM SVC mapping • differentiated services (diffserv) • Diffeserv in detail • Diffserv: a case study • Diffserv test activities (TF-TANT) • comments Tiziana Ferrari Quality of Service Support in Packet Networks

3. Quality of Service • Heterogeneous networks • capacity • transmission technology • bottlenecks • congestion and lack of transmission guarantees • heterogeneous application requirements • interactive: telnet, remote X sessions, web browsing • non-interactive and packet loss tolerant: ftp, mailing • delay sensitive: real time applications (e.g. remote control) • delay variation sensitive and packet loss: voice over IP, videoconferencing • ... Quality of Service Tiziana Ferrari Quality of Service Support in Packet Networks

4. Quality of Service: basic concepts • Service: a pre-defined type of packet treatment during transmission across the network • qualitative • quantitative • delay • instantaneous delay variation • packet loss probability • throughput • MTU (Maximum Transfer Unit) • priority (e.g. for congestion treatment) • Class: set of packets to which a given service applies. Classification is based on traffic filters. A filter defines a set of packet matching rules. Matching is based on the content of packet fields. E.g. filter := (pack(src) = SRC) && (pack(dest) = DEST) && (pack(pro) = TCP) Tiziana Ferrari Quality of Service Support in Packet Networks

5. Quality of Service: benefits • Benefits: • network. congestion management, congestion independent QoS parameters, traffic engineering • differentiated allocation of expensive network resources e.g. over intercontinental connections • multiple services • application: • within a single application: differentiated treatment of streams according to their requirements and priority • multiple applications: applications hierarchy according to priorities • today: single service, best-effort Tiziana Ferrari Quality of Service Support in Packet Networks

6. Solutions • Cell based networks: • ATM (permanent and/or switched connections) • IP based networks: • RSVP (resource ReSerVation Protocol) and integrated services • heterogeneous networks: • RSVP to ATM SVC mapping (IP and ATM) • differentiated services Tiziana Ferrari Quality of Service Support in Packet Networks

7. ATM (Asynchronous Transfer Mode) • Cell based • pre-defined classes of services: • Constant Bit Rate (peak cell rate - PCR-) • Variable Bit Rate (sustainable cell rate, PCR, maximum burst size -MBS-) • real time • non real time • Available Bit Rate (minimum cell rate -MCR-, PCR, rate increase/decrease factors) • Unspecified bit rate (PCR) • today: deployed as backbone technology (GARR, European national research networks, TEN-155, ESnet), not common as LAN technology. Only permanent connections are commonly deployed to support traffic engineering. Tiziana Ferrari Quality of Service Support in Packet Networks

8. ATM (cont) • Permanent and/or dynamic connections (PVC, SVC) • 1-to-1 or 1-to-many • end-to-end signalling protocol for ATM connection set-up • traffic engineering • Disadvantages: • not widely deployed as LAN technology (no end-to-end connectivity) • no native ATM applications • IP over ATM: overhead • few applications supporting traffic profile definition • addressing scheme not compatible with IP • signalling only in few backbones -> lack of interoperability Tiziana Ferrari Quality of Service Support in Packet Networks

9. RSVP • RSVP: signalling protocol for IP based applications • traffic profile definition - source - • reservation profile specification - destination - • 3 classes of service: best-effort, controlled load, guaranteed • reservation: (soft state) • QoS support in heterogeneous network • multicast is supported • QoS support to the application • Advantage: IP compatible • Disadvantages: • requires RSVP support on each router on the path from tx to rx • lack of scalability • admission control and policy management Tiziana Ferrari Quality of Service Support in Packet Networks

10. RSVP to ATM SVC mapping • ATM signalling in the core • RSVP signalling at the edge (access networks) • reservation parameter mapping at the boundary • Advantages: • deploys ATM features in the backbones • QoS support in heterogeneous environments • QoS to the application • interim solution -> intserv - diffserv (scalability) • Disadvantages: • application: RSVP capable, traffic profile specification • RSVP support at both edges • connection set-up latency: not convenient for delay sensitive applications transmitting small chunks of data • still relays on ATM signalling in the core Tiziana Ferrari Quality of Service Support in Packet Networks

11. RSVP to ATM mapping: features • Translation of intserv classes of services and parameters into ATM classes of services and parameters • deployment of best-effort connections (UBR in the ATM core) for initial transmission of RSVP messages (PATH, RESV) • combination of RSVP and ATM admission control • ATM: tx initiated signalling vs RSVP: rx initiated reservation request Tiziana Ferrari Quality of Service Support in Packet Networks

12. Scenarios • Mapping in the end-system • mapping in the router LIS 2 LIS 3 LIS 1 ATM ATM ATM RSVP -> ATM NRN - TEN-155 - NRN RSVP RSVP 1. PATH ATM 4. SVC 2. RESV 3. RSVP -> ATM Tiziana Ferrari Quality of Service Support in Packet Networks

13. Differentiated Services:Architecture Tiziana Ferrari Quality of Service Support in Packet Networks

14. Differentiated services cont. • Why diffserv? • new technology • independent of layer 2 technologies • interoperability between independent national research networks (different requirements, infrastructures, policies and management) • traffic aggregation • scalability: no reservation state maintained in the routers • nosignalling • QoS for networks not ATM based • RFC 2474: Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 headers • RFC 2475: An Architecture for Differentiated Services • RFC 2598: An Expedited Forwarding PHB • RFC 2597: Assured Forwarding PHB Group Tiziana Ferrari Quality of Service Support in Packet Networks

15. Diffserv architecture: building blocks • Label: DS field (1 byte), DS Code Point (6 bits) • packet classification • packet scheduling • traffic conditioning: • metering • marking • policing • shaping 0 6 7 CU DSCP Tiziana Ferrari Quality of Service Support in Packet Networks

16. DS building blocks: logical view meter Shaper/ policer scheduler marker classifier Tiziana Ferrari Quality of Service Support in Packet Networks

17. Diffserv: traffic aggregation and (re)marking aggregation and re-marking aggregation aggregation re-marking marking Tiziana Ferrari Quality of Service Support in Packet Networks

18. Terminology • Per Hop Behaviour (PHB): the externally observable forwarding behaviour applied to a DS-compliant node to packets with same label (DS codepoint) • PHB Group: set of PHBs which can only meaningfully specified and implemented simultaneously (e.g. with common constraints on queue servicing and queue management). E.g. 4 PHB each associated to a different drop priority. A single PHB is a special case of PHB Group. • Service: quantitative or statistical definition of significant characteristics of packet transmission in one direction across the network in terms of throughput, delay, jitter, loss, priority in access to network resources. Services are implemented through PHBs. The service describes the overall treatment end-to-end. Tiziana Ferrari Quality of Service Support in Packet Networks

19. Terminology (cont) • DS codepoint: specific value of the DSCP field • DS behaviour aggregate: packets with same code point • DS domain: contiguous set of nodes with same service provisioning policies and same code point numbering scheme • DS region: set of contiguous DS domains • DS ingress/egress node: DS node handling packets entering/leaving the DS domain it belongs to • classifier: entity selecting packets according to the content of packet headers according to a defined rule • BA classifier: a classifier which only takes into account the DS field content Interior/Ingress/Egress Node DS Region DS Domain Tiziana Ferrari Quality of Service Support in Packet Networks

20. Terminology (cont) • Conditioning: metering, marking • Policing: packet discard according to the state of a corresponding meter enforcing a traffic profile • Metering: the process of measuring the temporal properties of a traffic stream selected by a classifier • Marking: the process of setting the DS codepoint in a packet based on defined rules • Service Level Agreement: traffic contract between a customer and service provider specifying the forwarding service the customer’s traffic should receive • Service Provisioning Policy: specification of • microflow mapping into a DS Behaviour Aggregate • conditioning configuration Tiziana Ferrari Quality of Service Support in Packet Networks

21. Diffserv architecture: network model for TEN-155 DS domain NRN DS domain MPLS Non DS capable domain DS domain NRN Marking policing scheduling TEN-155 marking shaping DS domain DS domain Tiziana Ferrari Quality of Service Support in Packet Networks

22. Diffserv architecture: PHBs • Standardised PHBs: • Expedited Forwarding (low delay, low delay variation, guaranteed bandwidth) • Assured Forwarding (Behaviour Aggregate, 4 classes, 3 drop priorities per class) • Experimental PHBs • PHB class selectors 3 7 0 precedence Tiziana Ferrari Quality of Service Support in Packet Networks

23. PHB: Expedited Forwarding • For the implementation of services requiring a reservation profile like: low loss, low latency, low jitter, assured bandwidth • loss, latency and jitter  queue management (small queues) in order to prevent a queue from building up, in the EF queue of each transit node, the aggregate maximum arrival rate < departure rate • EF implementation based on • scheduling (for traffic isolation and support of bandwidth guarantees) • policing • shaping • EF traffic can preempt other classes, for this reason the maximum EF rate has to be limited through policing • EF codepoint: 101|110 Tiziana Ferrari Quality of Service Support in Packet Networks

24. PHB: Assured Forwarding • 4 independent AF forwarding classes • 3 drop priorities in each AF class • given two packets in node with drop precedence p and q respectively, with p < q, pack(p) is always transmitted BEFORE pack(q) • at the boundary between two AF domains, traffic conditioning can apply: shaping, per class discarding, drop precedence remarking and AF class reassignment. Traffic conditioning has to avoid REORDERING -> performance gain on the rx side Tiziana Ferrari Quality of Service Support in Packet Networks

25. PHB: Assured Forwarding (cont) • In case of long term congestion AF packets are dropped • drop algorithm: WRED (Weighted Random Early Discard) for the implementation of a gradual discard mechanism based on congestion levels and proportional to the drop precedence of a given microflow • Codepoints: class 1 class2 class3 class4 low drop prec 001|010 010|010 011|010 100|010 medium drop prec 001|100 010|100 011|100 100|010 high drop prec 001|110 010|110 110|110 100|110 Tiziana Ferrari Quality of Service Support in Packet Networks

26. PHB: Class Selectors and Experimental PHBs • Class selectors: • XXX000 where x = 1 or 0 • for backward compatibility with precedence field of the old so called TOS (Type Of Service) byte. TOS was replaced by the DS field. Precedence = [0, 7] • if pred(pack1) < pred(pack2) then p_drop(pack1) > p_drop(pack2) • Experimental PHB: • not standardised codepoint and packet forwarding behaviour • definition up to the ISP • requires PHB mapping at the boundary precedence TOS byte 3 7 0 Tiziana Ferrari Quality of Service Support in Packet Networks

27. Diffserv architecture:schedulingpolicing and classificationtraffic metering Tiziana Ferrari Quality of Service Support in Packet Networks

28. Scheduling • Scheduling: queue service policy for differentiated treatment of packets among queues • Examples of packet scheduling algorithms: • simple priority queue (high priority queues have the highest priority, arrival rate < departure rate, low priority traffic starvation is possible) • weighted round robin queuing (queues serviced in round robin fashion, service time proportional to the weight) • weighted fair queuing (minimum rate guaranteed per class, service time of each packet in each queue is a function of the packet size and of the queue weight. Current service time is updated every time a packet is sent) • class based queuing (maximum rate per class is configured) Tiziana Ferrari Quality of Service Support in Packet Networks

29. Congestion management and service enforcement: policing Policing: traffic which exceeds a given rate threshold is treated differently from conforming traffic. E.g. Exceeding packets can be dropped, re-marked, transmitted as best-effort etc. Policing is deployed for service level agreement enforcement: - to limit the input rate at the edge - at the boundary between domains to guarantee a fair deployment of the service among different domains Network Edge Packet Classifier and Policer CustomerPremise Policy Specification Tiziana Ferrari Quality of Service Support in Packet Networks

30. Policing: token bucket • Policing is based on traffic metering. A typical algorithm is called token bucket. Input pack stream drop probability = 1 Exceed burst 0 < drop probability < 1 Normal burst (number of tokens available), drop probability = 0 R: departure rate at which tokens are replenished output pack stream Tiziana Ferrari Quality of Service Support in Packet Networks

31. Token bucket: algorithm • tokens replenished at regular intervals • Normal burst: max number of tokens which can be in the bucket (in bytes) • Exceed burst: to avoid tail drop in favour of gradual drop • Actual debt AD =  ADi where is the number of borrowed tokens, • ADi decreases of R tokens per time unit • Compounded debt CD =  ADi where ADi CD = 0 after a packet drop packi is dropped if CB > exceed burst Tiziana Ferrari Quality of Service Support in Packet Networks

32. Token bucket: algorithm (cont) Example (by BoB Olsen) token rate = 1 data_unit/time_unit normal burst size = 2 data_units (DUs) extended burst = 4 DUs. rate = 2 DUs arrive per time unit. After 2 time units, the stream has used up its normal burst and must begin borrowing one DU per time unit, beginning at time unit 3. Time tocken available DU arrivals Actual Debt Compounded Debt 0 2 1 2-2+1=1 2 0 0 2 1-2+1=0 2 0 0 3 0-2+1=-1 2 1 1 4 -1-2+1=-2 2 2 3 5 -2-2+1=-3 2 3 (temp) 6 (temp) > 4 Tiziana Ferrari Quality of Service Support in Packet Networks

33. Token bucket: algorithm (cont) • At this time a packet is dropped because the new compounded debt (6) would exceed the extended burst limit (4). This causes CD to effectively become 0, and lowers AD back down to 2. The values 3 and 6 were only temporary and do not remain valid in the case where a packet is dropped. The final values for time unit 5 are given below. • Time tocken available DU arrivals Actual Debt Compounded Debt 5 -2-2+1=-3 2 3 (temp) 6 (temp) > 4 drop 5 -2 2 2 (*) 0 6 -2-2+1=-3 2 3 3 7 -3-2+1=-4 2 4 (temp) 7 (temp) drop 7 -3 2 3 0 Tiziana Ferrari Quality of Service Support in Packet Networks

34. Congestion Avoidance:Weighted Random Early Discard (WRED) • Goal: to avoid congestion before it occurs by dropping isolated packets among different streams. Packets are dropped according to the current amount of data into a buffer. If buffer_level < thr1 p(drop) = 0 case 1 thr1 < buffer_level < thr2 0 < p(drop) < 1 case 2 otherwise p(drop) = 1 case 3 • (W)RED is a congestion avoidance algorithm for TCP traffic based on the TCP flow control features (TCP reduces the output rate when a single packet is dropped before real congestion occurs) • WRED: like RED but p(drop) in case 2 depends on the DS codepoint of the packet. Packets with low priority experience packet drop before packets with higher priority • with both RED and WRED packet drop is randomly distributed among several flows Tiziana Ferrari Quality of Service Support in Packet Networks

35. Weighted Random Early Discard (WRED) - cont - • Class 6 gets 50% minimum • Class 4 gets 30% minimum • Class 2 gets 20% minimum • The remaining traffic gets 10% Bronze Packet Drop Probability Packet Drop Probability Standard Silver Gold 70% 60% 40% 90% Queue Length Max • GoldClass 6 will never get drop unless extreme congestion : 90%of queue depth • Silver Class 4 will not get dropped unless severe congestion :70%of queue depth • Bronze Class 2 will start drop at 60%queue occupancy • Standard The remaining traffic will start drop at 40% of queue occupancy Tiziana Ferrari Quality of Service Support in Packet Networks

36. Differentiated Services:Case Study Tiziana Ferrari Quality of Service Support in Packet Networks

37. Characterisation of the application Characterisation needed to identify the requirements of the application: service definition • 1. Hardware of the trigger: remote control- ROBIN - • few transactions (low bandwidth), TCP traffic on a limited well-known set of TCP ports, IP address of the server known • client - server, one connection to a server at a time  low paket loss, delay sensitive application, reliability, burst tolerance • 2. Monitoring of quality and correctness of the results of the analysis - ROOT - • exchange of analysis results (root object) • low bandwidth consumption • client - server, IP address of the server is known  bandwidth guarantee, more tolerance to packet loss Tiziana Ferrari Quality of Service Support in Packet Networks

38. Models of network deployment 1. Trigger hardware control Power PC VME Client 1 Server 1 Client 2 VME Server 2 Client 3 ... ... VME Client n Server m bottlenecks 1. Monitoring of analysis Browser 1 server Browser 2 Browser 3 ... Browser n Tiziana Ferrari Quality of Service Support in Packet Networks

39. Service for trigger control packets • SERVICE 1: • low drop probability • delay bound (minimum queue size) • precedence: highest precedence, higher than monitoring packets precedence • Capacity: a minimum network capacity guaranteed, in case of spare capacity, more bandwidth can be allocated. • No upper limit in rate for maximum burst tolerance Tiziana Ferrari Quality of Service Support in Packet Networks

40. VME Client 1 Server 1 Client 2 VME Server 2 Client 3 ... VME Client 4 Service 1: implementation • If pack(src/dest) = (s1 || s2 || … || s m) &&pack(TCP_port) in [x, y] then • pack(label) = max precedence • policing: always transmit • delay: buffer size of 2*MTU • minimum service rate = m * r * Nclient or rate = m * R * Nclient • m: overbooking factor • r: estimated rate consumed by 1 client, R: estimated rate consumed by 1 server • Nclient : number of clients downstream r R 2*m*r r 4*m*r R r R 2*m*r r Server 8 Marking client -> server Marking server -> client Scheduling client -> server Scheduling server -> client Tiziana Ferrari Quality of Service Support in Packet Networks

41. Service for monitoring traffic • SERVICE 2: • precedence: higher than best effort, lower than service 1 packets • drop probability: packets can be dropped in case of congestion. • Label: lower precedence • capacity: minimum guaranteed bandwidth, more bandwidth can be allocated if available • maximum upper rate threshold: for fair bandwidth allocation between several clients Tiziana Ferrari Quality of Service Support in Packet Networks

42. Service 2: implementation • If pack(src/dest) = s1 then • if rate < max, pack(label) = medium prcedence (marking) • else drop (policing) • shaping: buffer size > server or client burst size • minimum service rate = r * Nclient (client), or rate = R * Nclient (server) • r: estimated rate needed for 1 client, • R: estimated rate needed by the server to support m clients • Nclient : number of clients downstream • precedence: < precedence(service 1) Client 1 r 2*r r Client 2 4*r R Server r Client 3 ... 2*r r Policing server -> client Policing client -> server Client 4 Marking server -> client Marking client -> server Scheduling server -> client Scheduling client -> server Tiziana Ferrari Quality of Service Support in Packet Networks

43. Diffserv testing and QoS measurement Tiziana Ferrari Quality of Service Support in Packet Networks

44. Test network Tiziana Ferrari Quality of Service Support in Packet Networks

45. LAN layout (example) Tiziana Ferrari Quality of Service Support in Packet Networks

46. Equipment • Test workstations • traffic generators (3 SmartBits, Netcom System loan) • ATM switches in the test sites • cabletron ethernet switch • DS capable platforms: • CISCO: C7200 or C7500 (partial CISCO loan) • IBM: IBM 2212 and IBM 2216 (2 routers in 5 sites, IBM donation) • Linux • Cabletron (LAN switch) • (Nortel, Torrent) Tiziana Ferrari Quality of Service Support in Packet Networks

47. CISCO: traffic policing (CAR) • CAR: Committed Access Rate • Multi-field classification: OK • packet marking (precedence setting): OK • exceed action testing: OK • policing (at a configurable rate): two parameters token bucket for TCP performance optimisation • normal burst • exceed burst --> parameter tuning Tiziana Ferrari Quality of Service Support in Packet Networks

48. CAR: test of exceed actions Throughput: SWITCH: 1.20 Mbps DANTE: 0.38 Mbps Tiziana Ferrari Quality of Service Support in Packet Networks

49. CAR: TCP burst tolerance • Normal and exceed burst tuning with TCP traffic • single and multiple TCP streams optimum values are functions of the rate R at which traffic is policed, in particular: normal burst = 0.5 sec * R exceed burst = 2 * normal burst Tiziana Ferrari Quality of Service Support in Packet Networks

50. CAR: TCP burst tolerance (cont) Tiziana Ferrari Quality of Service Support in Packet Networks