Design Principles and Architectures for End-to-End Quality of Service

1. Design Principles and Architectures for End-to-End Quality of Service Chris Gill cdgill@cs.wustl.edu Center for Distributed Object Computing Department of Computer Science Washington University, St. Louis, MO http://www.cs.wustl.edu/~cdgill/End_to_End.ppt

2. Theme: Layering in Distributed Systems • End-to-end layers • Application components, middleware services, and transport protocols distributed over multiple end-systems • Hop-to-hop layers • Network: internetworking e.g., IP routing between LANs • Data Link: e.g., frame management in LANs, bridges • Physical: EE aspects for a medium: voltages, timing • Different time scales • End-to-end vs. hop-to-hop, in-band vs. out-of-band mechanisms • Design Trade-offs • How to decide what functionality belongs where

3. The End-to-End Argument • J.H. Saltzer, D.P. Reed and D.D. Clark, “End-To-End Arguments in System Design”, ACM Transactions on Computer Systems, 4(4):277-288, November 1984. • Design Principle: “A lower level subsystem that supports a distributed application may be wasting its effort providing a function that must by nature be implemented at the application level anyway” - depends on other design forces • Examples • FTP correctness, message delivery, data encryption, duplicate message suppression, FIFO message ordering, transactions

4. The End-to-End Argument: FTP Correctness • Question: where to perform bit error detection and/or correction for file transfer in a distributed system? • End-to-end path: I/O device to application to OS to protocol stack to LAN to router to protocol stack to router to LAN to protocol stack to OS to application to I/O device • Hop-to-hop error detection/retransmit, or error correction is overly expensive unless link is highly unreliable, and will not solve the entire problem, as sender crashes, I/O bit errors, and other threats still require end-to-end solution • Application-specific solution is an end-to-end checksum every so many bytes and at EOF

5. The End-to-End Argument: Message Delivery • A message is much shorter than a byte stream (e.g., FTP) and is at less risk for corruption • However, there are still many threats along the path • E.g., packet dropping due to router congestion • Hop-to-hop ACK/retransmit may be useful for congestion control depending on link reliability and router capabilities, but still cannot ensure “perfect” reliability • E.g., receiver crashes right after receipt, unbounded priority inversion on receiver, etc. • Need an application-specific end-to-end solution such as a transactional commit protocol

6. The End-to-End Argument: Secure Transmission • If hop-to-hop encryption is performed, network nodes will need to securely manage encryption keys • Unless the application also performs decryption, the message will be vulnerable on the destination end-system • The application will also need to perform authentication of the messages it receives • Therefore, intermediate encryption is redundant here • However, intermediate encryption still might be useful to ensure no unauthorized exposure of information occurs, say outside a departmental firewall: additional requirement

7. The End-to-End Argument: Duplicate Messages • For efficiency, and possibly correctness, it is useful to discard duplicate messages • Even if the network discards all duplicate messages, it has no way to detect duplication at the application level • E.g., a repeated user login request due to slow system response • Since the application needs a way to suppress duplicates anyway, that function can also be used to suppress duplicates originating in lower layers • Therefore eliminating duplicates in lower layers is wasteful, unless motivated by another design force

8. The End-to-End Argument: FIFO Ordering • Independent paths are useful for robustness in the face of link failures (e.g., spanning tree re-convergence for routing), for priority-preserving virtual circuits, etc. • Latency differences (e.g., due to congestion, retransmission) between consecutive messages routed along independent paths may result in reordering • However, distributed applications over multiple end-systems are asynchronous, so for many application if the order of messages is important, then a higher level message synchronization protocol is needed anyway

9. The End-to-End Argument: Applied • Distributed reads and writes in the SWALLOW distributed data storage system - anticipates DOC applications (1984) • Object id plus message version is sufficient to detect duplicate writes at application layer of server, plus duplicate replies for reads are easily discarded at client • Writes on server are ACKed back to client after data is stored safely, and the read value response is the ACK • Performing duplicate elimination and acknowledgement at the application level simplifies the lower layers, and improves performance by reducing both in-band overhead and message traffic from lower level ACKs

10. The End-to-End Argument: Endpoints • The message delivery and secure transmission examples highlight a more fundamental issue: where a path ends • Different applications may apply requirements to different path segments, and may have different overall paths • Client-server application: two-way reply appears on the screen • Manufacturing application: remote robot arm rotates 15 clockwise • Different inherent semantics for various approaches • CORBA 2-way calls vs. transactional MOM • Different applications may produce different decisions WRT the end-to-end argument

11. QoS Architectures • Aurrecoechea, C., Campbell, A.T. and L. Hauw, "A Survey of QoS Architectures," ACM/Springer Verlag Multimedia Systems Journal , Special Issue on QoS Architecture, Vol. 6 No. 3, pg. 138-151, May 1998 • Presents a set of QoS design principles • Presents an architecture (they use the term framework) for QoS specification and enforcement • Surveys other QoS management architectures, circa 1998 • Terrific bibliography (modulo a few erroneous dates) - will highlight a few papers that proposed design principles

12. QoS Design Principles • Integration principle - QoS must be configurable, predictable, and maintainable over all layers • Kurose, J.F. “Open Issues and Challenges in Providing Quality of Service Guarantees in High Speed Networks”, ACM Computer Communications Review, vol. 23, no. 1, pp. 6-15, Jan 1993 • End-to-end QoS depends on all modules traversed in the end-to-end path, each of which must offer: • QoS configurability based on some specification • Resource guarantees and enforcement mechanisms • Maintenance of ongoing flows

13. QoS Design Principles, Continued • Separation principle - media transfer, control, and management are three functionally distinct activities • Lazar, A.A., “A Real-time Control, Management, and Information Transport Architecture for Broadband Networks”, Proc. International Zurich Seminar on Digital Communications, pp. 281-295, 1992 • Three activities at three scales of complexity • Media transfer - simple transmission: hop-to-hop forwarding • Control - more complex but related to media transfer: routing, queue scheduling, fine-grain monitoring etc. • Management - distributed, possibly end-to-end: reservations, reconfiguration, large-scale monitoring, etc.

14. QoS Design Principles, Continued • Transparency principle - applications should be shielded from the complexity of the underlying QoS policies and management mechanisms • Reduces accidental complexity by decoupling QoS aspects from application functionality • Delegates inherent complexity to a layer below the application • Combined with the end-to-end argument, this makes a good case for QoS management in middleware

15. QoS Design Principles, Continued • Multiple time scales principle - guides division of QoS control and management functionality between modules • (also from) Lazar, A.A., “A Real-time Control, Management, and Information Transport Architecture for Broadband Networks”, Proc. International Zurich Seminar on Digital Communications, pp. 281-295, 1992 • Three activities at three scales of time • Media transfer - rapid transmission • Control - additional overhead but still close to media time scale • Management - distributed, possibly end-to-end time scales • Balancing design force to the end-to-end argument

16. QoS Design Principles, Continued • Performance principle - a collection of widely agreed design idioms pertaining to overall system performance • The end-to-end argument • Avoiding multiplexing • Tennenhouse, D.L., “Layered Multiplexing Considered Harmful”, Protocols for High-Speed Networks, Elsevier Science Publishers (North-Holland), 1990 • Structuring communications protocols • Clark, D. and Tennenhouse, D.L., “Architectural Consideration for a New Generation of Protocols”, ACM SIGCOMM 1990 • Pushing protocol processing down into hardware

17. QoS Architecture: Specification • Each category captures a class of application level QoS requirements as declarative (what rather than how) policies that are enforced by layer-specific mechanisms • Flow synchronization - degree of frame tightness between flows • interestingly, applies to frame-based event-driven applications • Flow performance - throughput, delay, jitter, loss • Level of service - deterministic, predictive, best effort gurantees • QoS management policy - adaptation and scaling capabilities • Cost of service - competitive, collaborative, individual/social good

18. QoS Architecture: Mechanisms • Provisioning • Initial flow establishment and end-to-end re-negotiation • Setting up configurations of resources and functionality in support of media transfer and processing • End-to-end time scale • Control • In-band handling of flows on time-scales close to media transfer • Management • Out-of-band reconfiguration of flows to ensure contracts are met • Time scales between media transfer and end-to-end

19. QoS Mechanisms: Provisioning • QoS mapping • Transforms application QoS requirements into policies for configuring and operating mechanisms in systems layers • Admission control • Determines whether new requirements can be met given current reservations and resources • Resource reservation • Once admission control has been performed, allocates (binds) resources to the new flow(s)

20. QoS Mechanisms: Control • Flow shaping • In-band traffic pacing, smoothing, pruning to improve overall good • Flow scheduling • Forwarding/processing based on per-flow or aggregate policy • Flow policing • Detects misbehaving flows • Flow control • Sender pacing or receiver feedback adaptation • Flow synchronization • Control of flows based on aggregate policies

21. QoS Mechanisms: Management • QoS monitoring • Each layer may track QoS achieved by other modules (feedback) • QoS maintenance • Tunes operations based on perceived/expected QoS • QoS degradation • QoS cannot be corrected by lower layers, must handle or pass up • QoS availability • Performance monitoring/assessment/signaling intervals • QoS scalability • Filtering and adaptation over heterogeneous resources/end-systems

22. Survey of Related QoS Architectures • IBM European Networking Center (Heidelberg) - HeiRAT • Columbia U - eXtended integrated Resource Model (XRM) • U Pennsylvania - OMEGA • IETF - IntServ • Lancaster U, Alcatel - QoS-A • OSI - QoS Framework • UC Berkeley - Tenet Architecture • TINA-C - QoS Framework • U Pierre et Marie Curie - MASI End-to-End model • Washington U (Gopal) - End System QoS Framework

23. Conclusions • Application-specific QoS requirements must be considered • Timeliness, correctness, liveness, security • Application-specific design forces combine with the inherent complexities of distribution • Asynchrony, failures, latency, etc. • System design principles articulate key idioms/forces • Design patterns and pattern languages appear helpful • Identifying and codifying trade-offs driven by design forces • End-to-end argument supports the case for middleware • Need flexibility to plug in application-specific high-level policies • Encapsulation of common mechanisms in middleware reduces accidental complexity for the application