Random Thoughts on Application Performance and Network Characteristics

1. Random Thoughts on Application Performance and Network Characteristics Brian L. Tierney bltierney@lbl.gov Distributed Systems Department Lawrence Berkeley National Laboratory GGF

2. Questions • 1) Applications have different requirements of the various network attributes (ie bandwidth, packet loss, latency, jitter). How would you create classifications of applications such that network interfaces could be created to optimize performance for each classification? • 2) What can the network do better to create better performing applications for each of the classifications? • 3) What can the applications do to make better use of the network? • 4) Can standard applications interfaces (sockets) and/or protocols be created to enable the ideas in 2 and 3? GGF

3. Application Classifications • Types of applications • Loss sensitive applications • Jitter sensitive applications • Latency sensitive apps • All would benefit from more network knowledge • All require some form of QoS • High Bandwidth applications • “Fixing” TCP could make network monitoring unnecessary for these types of applications • Should need nothing more than next generation TCP and parallel streams • Higher-level services like LSL should also help • Important to define exactly what problems we are trying to solve • What applications are most important GGF

4. Ubiquitous Network Monitoring • Image an internet with the following ability: • A DNS query (e.g.: gethostbyname() ) returned all of the following: • IP address • Lat/long • Min/max/average delay over the past hour • Min/max/average available bandwidth over the past hour • packet loss metric • out of order packet metric • jitter metric GGF

5. Network Monitoring: Open Issues • Many Issues! • Use passive or active monitoring? • End-to-end? 1st router to last router? 1st router to end host? • What should be reported? min/max/ave/std dev/? Time series? Over what time interval? • Prediction? (“ask the network oracle”?) Over what time interval? • What about QoS capabilities? How does QoS effect the above results? • Exactly how will applications use this data? • Need to prioritize which application classes will gain the most benefit GGF

6. Make Applications Smarter? • Do applications need to be “network aware”, or should they use higher level services which are “network aware”? • Many simple things are not being done today • Long pipelines • Large TCP buffers • More application buffering • Need better services to hide network from the application • Examples: • Logistical Network Layer (M Swany) • Buffering Service (Kangaroo) • Forward error correction service GGF

7. Better Bulk Transfer • Assuming data is replicated in multiple locations (not always true…), always just download in parallel from multiple sources (bittorent model) • Example: • Copies of a file exist in 20 location • Divide the file into blocks (eg: 256K), and download in parallel from 5 servers • Servers chosen based on shortest RTT • If blocks from a given server are coming in too slow, try a different server • Should be easy to saturate the incoming host, while not overloading any server • Monitoring is all application level. • Save performance results to use as estimate for the next time GGF

8. New API’s • Higher level APIs are needed • but what should they look like? GGF

9. Conclusion • Need to determine exactly what network monitoring data applications (or network services) really need and will actually use • New GGF Network Monitoring for Applications Research Group (NMA-RG) has been formed to precisely deal with these issues • http://dsd.lbl.gov/NMA-RG/ GGF