End-to-End Arguments in System Design

1. End-to-End Arguments in System Design Presentation by David Florey CS533 - Concepts of Operating Systems

2. Overview • The End-to-End Argument • Considerations • Guarantees • Thoughts CS533 - Concepts of Operating Systems

3. End-to-End Argument • Simple File Transfer from Host A to Host B • File can be corrupted in transit • File can be corrupted when read • Buffered file can be corrupted in memory • Adding checks during transit may work, but what if they don’t work in all cases? • The only thing that is going to know if everything went down correctly is the highest level – the File Transfer Application • Use checksums at Application Level CS533 - Concepts of Operating Systems

4. The End-to-End Argument CS533 - Concepts of Operating Systems

5. ConsiderationsReliability and Performance • We cannot guarantee total reliability at the low level • How much is enough? • Performance tradeoffs • Poor reliability due to little or no checking at low level leads to poor performance due to too many retries above • Too much checking at low level leads to poor performance because of duplication of effort, perhaps unnecessary work • Middle ground – reliable enough, but not perfect • Low level should make its “best effort” to improve performance without a complicated implementation CS533 - Concepts of Operating Systems

6. ConsiderationsPlacement of Things • Early checks can detect errors sooner, but is it the job of the application or of the subsystem? • Consider what applications will be using the subsystem • Are they all going to benefit from this? • Should each on write their own check? • We are not adding to the low level to improve overall reliability, we are adding to improve performance CS533 - Concepts of Operating Systems

7. Guarantees • Delivery Guarantees • We only care if the receiver did the job or not • Subsystem should not try to make this guarantee because only the application will know if the message made it or not • Secure transmissions • Responsibility of application: needs to check authenticity and ensure that message is encrypted the entire time CS533 - Concepts of Operating Systems

8. Guarantees • Duplicate message suppression • What’s your definition of a duplicate? • Details of an actual duplicate are unknown to subsystem as this is very application specific • Therefore, since subsystem can’t guarantee to suppress all types duplicate messages, and the app needs to detect these any way, the app should do all duplicate detection/supression CS533 - Concepts of Operating Systems

9. What is an “End”? • Consider the application • temporal accuracy vs logical accuracy • Real-time or streaming voice or video applications can tolerate some amount of error at the application level • Lossy compression algorithms do this • Perhaps I could say “Could you repeat that?” • Low level error checking can get in the way and slow it down CS533 - Concepts of Operating Systems

10. Summary of End-to-End • Application knows best • Applications are better suited for error recovery, duplicate suppression, encryption and so on • Subsystems need to be reliable within reason, but should not provide these “application” features CS533 - Concepts of Operating Systems

11. Thoughts • Larger concepts • “Open” Operating Systems • Applications know better how to manage resources: • Microkernels • Scheduler activations • Exokernel • Evolution of things • End-to-End seems to be the first step, the Neanderthals of arguments • Better to consider solutions that can eventually be standardized, automated and pushed to the next layer down or simply placed in a shared module CS533 - Concepts of Operating Systems

12. Thoughts • Parameterization • Providing functionality at the low level like duplicate suppression, while providing the ability to just turn it off • Consider potentially providing levels of functionality, chosen by the application CS533 - Concepts of Operating Systems

13. Thoughts • This seems to be the main argument behind layering • Functions layered, modularized • FTP checks only FTP, TCP doesn’t care about it and IP doesn’t care about TCP • No lower module performs jobs better suited in a higher module CS533 - Concepts of Operating Systems

14. Conclusion • The End-to-End Argument should be considered for every low level function or for each “problem solved” at the low level • Who’s going to use it? • What does it provide? Functionality? Performance? • Applications know best, but one should be aware of overheads and costs incurred when implementing functionality at ANY level CS533 - Concepts of Operating Systems