Quality of Service for Remote Control in the High Energy Physics Experiments: a Case Study

1. Quality of Service for Remote Control in the High Energy Physics Experiments: a Case Study Tiziana Ferrari Tiziana.Ferrari@cnaf.infn.it INFN - CNAF Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

2. Overview • Quality of Service • differentiated services architecture • application characterization • service definition and implementation • future work Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

3. Quality of Service: problem statement • today: single service, best-effort • Service: type of “data” treatment during transmission across the network • qualitative • quantitative (delay, instantaneous delay variation, packet loss, throughput, MTU, congestion treatment) • Class: set of packets to which a given service applies • Benefits: • congestion management • differentiated allocation of expensive network resources (ISP) • application: differentiated treatment of streams according to their requirements and priority • guarantees of congestion independent parameters (e.g. delay variation) Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

4. Differentiated services • Why the differentiated services (diffserv)? • independent of layer 2 technologies • interoperability between independent national research networks (different requirements, infrastructures, policies and management) • traffic aggregation • scalability • nosignalling • QoS for networks not ATM based Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

5. Diffserv architecture components • DS field (1 byte) - IP header - • packet classification • packet scheduling • traffic conditioning: • metering • marking • policing • shaping 0 6 7 CU DS field Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

6. Diffserv architecture: PHBs • Per Hop Behaviours: description of the packet treatment into the network • Standardised PHBs: • Expedited Forwarding (low delay, low delay variation, guaranteed bandwidth) • Assured Forwarding (Behaviour Aggregate, 4 classes, 3 drop priorities per class) • Experimental PHBs • PHB class selectors: XXX000 (backward compatibility with TOS precedence) Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

7. Diffserv architecture: network model DS region DS domain DS domain ATM MPLS Non-DS capable domain DS domain Marking policing scheduling Core domain marking shaping DS domain DS domain Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

8. Application characterization Goal: identification of the application requirements for service definition • Trigger hardware remote control • Monitoring: quality and correctness of analysis results Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

9. Server 1 VME Client 1 Server 2 Client 2 VME R1 R2 R3 Client 3 ... ... VME Client n Server m bottlenecks Application 1: trigger hardwareremote control • ROBIN • client - server, duplex exchange of data, interactive • few transactions (low bandwidth) • TCP ports and IP address of the server known • one-way delay sensitiveness • high reliability (packet loss probability, bandwidth) • burst tolerance Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

10. server ... bottleneck Application 2: Analysis monitoring • exchange of analysis results (root object) • duplex, asymmetric traffic volume • low bandwidth consumption • client - server, • TCP, IP address of server and clients known • packet loss tolerant • minimum bandwidth guarantee per client • dynamic allocation of the maximum overall capacity allocated Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

11. Service Level Specification (SLS) - 1 • Hardware remote control: • packet classification and marking (at the edge): if (pack(src) = (s1 || s2 || … || sm) && pack(dst) = (d1 || d2 || … || dn) && protocol = TCP && pack(port)[x, y] ) then label(pack) = max precedence • bandwidth guarantee (through policing): max service rate = 64 Kbps * Nsrc 64 kbps: estimated rate consumed by 1 server/client Nsrc: number of servers (upstream) of clients (downstream) if (traffic <= max && burst <= 64 kby) then label = Expedited Forwarding else label = best-effort • one-way delay (through scheduling): priority queuing Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

12. Service Level - 1: implementation PQ VME Client 1 R PQ r Server 1 PQ Client 2 R PQ PQ VME PQ PQ r PQ Server 2 PQ PQ R Client 3 ... PQ VME r PQ Client 4 Server 8 PQ Client  server: scheduling marking traffic, max rate R Server client: policing scheduling marking traffic, max rate r Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

13. Service Level Specification -2 • Analysis monitoring: • server  client: • packet classification and marking (at the edge): if (pack(src) = s&& pack(dst)=(d1 || d2 || … || dn) && protocol = TCP) then label(pack) = PHB  Assured Forwarding(AF)class • bandwidth guarantee (through policing): max service rate per client = 256 Mbps min rate per client: 64 kbps for each client: if (traffic <= min && burst <= 16 Kby) then label = AF11; else if (traffic <= max) then label = AF12; else drop(packet) • scheduling: Weighted Fair Queuing • traffic differentiation: Weighted Random Early Discard Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

14. Service Level Specification -2 (cont) • Client  server: • packet classification and marking (at the edge): if (pack(src) = di && pack(dst) = s && protocol = TCP) then label(pack) = PHB  Assured Forwarding(AF)class • bandwidth guarantee (through policing): max service rate = 5 Mbps min rate per client: 64 kbps for each client: if (traffic <= min && burst <= 16 Kby) then label = AF11; else label = AF12; if (traffic(Af12) <= max) then OK else drop(packet); • scheduling: Weighted Fair Queuing • traffic differentiation: Weighted Random Early Discard Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

15. Service Level -2: implementation r WFQ Client 1 WFQ r WFQ Client 2 R WFQ WFQ WFQ r Server WFQ WFQ WFQ Client 3 r WFQ Client 4 WFQ scheduling marking Client  server: traffic, max rate R policing scheduling marking traffic, max rate r Server client: Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

16. Scheduling: PQ vs WFQ Priority Queuing: absolute precedence over any other queue one-way delay minimization WFQ: distribution of bandwidth between queues according to the weight associated to each of them Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000

17. Future work Fermilab Test site 1 ESnet diffserv Testbed European diffserv testbed or ATM p-2-p link Test site 1 ... Test site 1 • Testbed configuration • tuning of router configuration • validation of the EF : priority queuing, burstiness and scheduling • validation of the AF implementation: WRED configuration Quality of Service for Remote Control in the High Energy Physics Experiments CHEP, 07 Feb 2000