1 / 21

Autotuning in Web100

Autotuning in Web100. John W. Heffner August 1, 2002 Boulder, CO. A Senior Thesis. Wrote senior honors thesis on autotuning. Since integrated the code into the Web100 kernel. Currently in CVS, not released. This talk is mainly based on that work.

Download Presentation

Autotuning in Web100

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Autotuning in Web100 John W. Heffner August 1, 2002 Boulder, CO

  2. A Senior Thesis • Wrote senior honors thesis on autotuning. • Since integrated the code into the Web100 kernel. Currently in CVS, not released. • This talk is mainly based on that work. • My thesis can be found (in PS or PDF) at http://www.psc.edu/~jheffner/papers/

  3. The Problem (in a nutshell) • Network bandwidth is increasing exponentially, TCP is not keeping up. • One reason: TCP queues require space proportional to bandwidth, buffers are statically sized. • Can use large static sizes, but not a general solution.

  4. Goal • Understand TCP queuing. • Implement network-based queue sizes. No arbitrary limits.

  5. TCP Window • A window is the amount of data TCP keeps “in flight” on a connection. • TCP sends one window of data per round-trip time. • Assume RTT constant. • Throughput proportional to window size.

  6. The BDP • If full path bandwidth is used, window = bandwidth * RTT. This is known as the Bandwidth-Delay Product (BDP). • Path bandwidths follow a variant of Moore’s Law, increasing exponentially with time. • Round-trip times are bounded by the speed of light. • Therefore, BDPs and window sizes increase exponentially with time.

  7. TCP Window Limits • Window size bounded by: • Sending application • Congestion window • Announced receive window (receiver problem) • Retransmit queue buffer size (sender problem) • Improper resource allocation may cause last two to improperly limit window.

  8. The Receiver Problem • The receive window is primarily a flow control device. • Standard sockets-based TCPs have a per-connection receive buffer of a fixed size. Uses receive window to ensure this buffer does not overflow. • When receive buffer is too small, throughput is limited.

  9. Example: rwin too small

  10. Large Receive Buffer • Why not just set the receive buffer very large? • Flow control will break!

  11. Re-think Receive Buffer • Do we need strict per-connection memory limits? Not really. • Note: implications for possible DOS attacks not fully explored. • Announce window based on observed network properties, not memory limits. • Don’t have to worry about protocol and queuing overhead space.

  12. Measuring Network from Receiver • Use the DRS algorithm [Wu Feng, Mike Fisk] • Measure window as bytes received in RTT, use twice this value in announcement calculation. • Bound RTT by measuring time taken to receive one window. • Additional RTT measurement using timestamps. • May be able to track RTT variations better.

  13. Announcing Receive Window • New variables: • rcv_space: twice the most recently measured window (DRS) • rcv_alloc: the amount of unread in-order data queued • ofo_alloc: the amount of out-of-order data queued • rwin = rcv_space – rcv_alloc + ofo_alloc

  14. Behavior Trace

  15. Receiver Test Results

  16. The Sender Problem • TCP must keep at least one window of data in the retransmit queue. • Why not just use a large send buffer? • Bulk transfer applications will fill any send buffer, and may waste a large amount of memory per connection.

  17. Autotuning ’98 • Semke, et. al. at PSC did original “Auto-tuning” in NetBSD, published in SIGCOMM ’98. • Automatically tunes sndbuf to at least 2*cwnd. • Implemented in Linux 2.4, though it still imposes a (small) per-connection limit.

  18. A Different Way • Do we need a per-connection send buffer limit? Yes. • Does the retransmit queue have to be in the send buffer? No. • Just separate the retransmit queue, and do not limit it.

  19. Benefits • Don’t have to worry about queuing overhead space • Saves some memory • Natural, simple implementation

  20. Sender Test Results

  21. Future Work • Understand and defend against possible attacks. • Demonstrate or improve robustness. • DRS fails when routing queue sizes vary due to other flows.

More Related