J.H.Saltzer, D.P.Reed, C.C.Clark End-to-End Arguments in System Design

1. J.H.Saltzer, D.P.Reed, C.C.Clark End-to-End Arguments in System Design Reading Group 19/11/03 Torsten Ackemann

2. Introduction • In a modular system, functions can often be put into different sub-systems • By network, client, joint venture, redundantly • Many functions require knowledge only the applications have • Providing the function in the network is impossible • This is known as the “end-to-end argument” • Reasoning against low-level function implementation

3. End-to-end Caretaking • Considering “Careful file transfer” • Host A linked via multi hop network to host B • Goal is to move file without damage • There is an file transfer application and the data communication system • Specific steps of transaction: • File transfer application on host A reads file • Applications asks comm system for transmission • Comm network transmits file from A to B • Comm system on host B reads packets and delivers them to file transfer application on host B • File transfer application on host B writes file

4. End-to-end Caretaking: Threats • Threats to the transaction: • Reading incorrect data from the disk • The software on both hosts might make a mistake in buffering and copying the data • Processor or memory on hosts A and B might be faulty • Comm system might drop or change bits in a packet, lose packets or deliver packets more than once • Either host may crash part way through the transaction • How to cope with these threats?

5. End-to-end Caretaking: Solutions • Reinforce each of the steps along the way? • using duplicate copies, timeout and retry, redundancy, crash recovery etc. • Uneconomical if all threats are low in probability • Alternative is “end-to-end check and retry” • Perform all steps, then check on receiving host • Additional step to read file back into memory on host B, then calculate and send checksum to host A • Retry from the beginning if checksums don’t match • Two or more failures on the same transfer indicate that the system is in need of repair

6. End-to-end Caretaking: Reliability • Proposal: Comm system guarantees reliable data transmission • For example by redundancy with packet checksums, sequence numbers, retry mechanism • Reliability of transmission can be increased to any level • The application still has to counter treats outside the comm system • Read and write failures, software failures etc. • The extra effort in the comm system has no effect on the correctness of the outcome • The application has to provide an end-end reliability guarantee anyway

7. Performance Aspects: Reliability • Application is not aware of packets, and retransmits whole file in case of errors • Comm system can retransmit on packet level • Effort on lower levels to increase reliability can increase performance significantly • But: No “perfect” reliability not necessary on lower levels • Tradeoff based on performance rather than a requirement based on correctness • Reliability measures in comm system lower performance, too!

8. Performance Aspects • Performing the function at the lower level may cost more! • Job can’t be done as efficiently since information is more limited • May be more efficient if it can be performed with minimal perturbartion • Applications that don’t need the function have to pay for it

9. Example: Delivery Guarantees • Acknowledgement of Delivery of a message • Knowing that a message got delivered is not important for an application • Knowing that the other host acted on the message is important • This can only be done by the application on the other host, not the comm system

10. Example: Secure Transmission of Data • Encryption in the network requires trust to manage encryption keys • Data is in the clear as it passes into the target node • Authenticity still has to be checked by application • Network encryption ensures that a node cannot deliberately transmit information that should not be exposed

11. Example: Duplicate Message Transmission • Some network designs allow(ed) a message to be delivered twice • The application might do that, too • Thus, suppression must be accomplished by the application anyway • Function can be omitted from lower levels

12. Example: Transaction Management • Distributed data storage system • Accessing data via identifier, version, type of access: read/write (, value to be written) • Network is simplified and does not suppress duplicate messages • Identifier plus version suffices to detect duplicate writes • Duplicate read only leads to duplicate response • Acknowledgement needed is that data is stored safely • Can only be provided by the higher levels • By eliminating ACKs, the number of messages is halved!

13. Identifying the Ends • Analysis of application requirements is essential • For voice traffic, unusually strong version of the argument applies • Bit-perfect communication leads to delays in packet delivery • It is better to accept slightly damaged but not delayed packets • However, this changes if the conversation is not real-time (voice mail) • Accuracy is more important than performance suddenly • The end-to-end argument is not an absolute rule, but a guideline • Carefully identify the end points!

14. Conclusions • End-to-end arguments are “Occam’s Razor” for networking • Comm systems are often specified before the applications • Designer may be tempted to “help” users by taking on more functions than necessary

15. Statement/Questions • End-to-end allows rapid deployment of new services by anybody • Reason for the success of the Internet? • Will history repeat with Wireless vs. UMTS? • Application level nets and overlay nets are end-to-end applications • Political Dimension • It is about control: “Walled gardens” of providers • Users want to give up control for comfort? • End-to-end is less secure? • More than two stakeholders? Governments etc. • End-to-end transparency is the key • Smart nodes in between just have to appear dumb