1 / 34

Reducing the Buffer Size in Backbone Routers

Reducing the Buffer Size in Backbone Routers. Yashar Ganjali High Performance Networking Group Stanford University. February 23, 2005 yganjali@stanford.edu http://www.stanford.edu/~yganjali. Motivation. Problem Internet traffic is doubled every year

kata
Download Presentation

Reducing the Buffer Size in Backbone Routers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reducing the Buffer Size in Backbone Routers Yashar Ganjali High Performance Networking Group Stanford University February 23, 2005 yganjali@stanford.edu http://www.stanford.edu/~yganjali

  2. Motivation • Problem • Internet traffic is doubled every year • Disparity between traffic and router growth (space, power, cost) • Possible solution • All-optical networking • Consequences • Large capacity  large traffic • Very small buffers High Performance Networking Group

  3. Outline of the Talk • Buffer sizes in today’s Internet • From huge to small (Guido’s results) • 2-3 orders of magnitude reduction • From small to tiny • Constant buffer sizes? High Performance Networking Group

  4. Backbone Router Buffers • Universally applied rule-of-thumb • A router needs a buffer size: B=2TxC • 2T is the two-way propagation delay • C is the capacity of the bottleneck link • Known to the inventors of TCP • Mandated in backbone routers • Appears in RFPs and IETF architectural guidelines Source Destination Router C 2T High Performance Networking Group

  5. Window size t Review: TCP Congestion Control Rule for adjusting W • If an ACK is received: W ← W+1/W • If a packet is lost: W ← W/2 Only Wpackets may be outstanding Source Dest High Performance Networking Group

  6. Probability Distribution Buffer Size Multiplexing Effect in the Core B 0 High Performance Networking Group

  7. Backbone router buffers • It turns out that • The rule of thumb is wrong for a core routers today • Required buffer is instead of High Performance Networking Group

  8. Required Buffer Size Simulation High Performance Networking Group

  9. Impact on Router Design • 10Gb/s linecard with 200,000 x 56kb/s flows • Rule-of-thumb: Buffer = 2.5Gbits • Requires external, slow DRAM • Becomes: Buffer = 6Mbits • Can use on-chip, fast SRAM • Completion time halved for short-flows • 40Gb/s linecard with 40,000 x 1Mb/s flows • Rule-of-thumb: Buffer = 10Gbits • Becomes: Buffer = 50Mbits High Performance Networking Group

  10. How small can buffers be? • Imagine you want to build an all-optical router for a backbone network… • …and you can build a few dozen packets in delay lines. • Conventional wisdom: It’s a routing problem (hence deflection routing, burst-switching, etc.) • Our belief: First, think about congestion control. High Performance Networking Group

  11. Utilization of bottleneck link = 75% TCP with ALMOST No Buffers High Performance Networking Group

  12. Problem Solved? • 75% utilization with only one unit of buffering • More flows  Less buffer • Therefore, one unit of buffering is enough High Performance Networking Group

  13. TCP Throughput withSmall Buffers High Performance Networking Group

  14. TCP Reno Performance High Performance Networking Group

  15. Two Concurrent TCP Flows High Performance Networking Group

  16. 1 RTT Simplified Model • Flow 1 sends W packets during each RTT • Bottleneck capacity = C packets per RTT • Example: C = 15, W = 5 • Flow 2 sends two consecutive packets during each RTT • Drop probability is increased with W High Performance Networking Group

  17. Simplified Model (Cont’d) • W(t+1) = p(t)x[W(t)/2] + [1-p(t)]x[W(t)+1] • But, p grows linearly with W • E[W] = O(C½) • Link utilization = W/C • As C increases, link utilization goes to zero. • Snow model!!! High Performance Networking Group

  18. Q&A Q. What happens if flow 2 never sends any consecutive packets? A. No packet drops unless utilization = 100%. Q. How much space we need between the two packets? A. At least the size of a packet. Q. What if we have more than two flows? High Performance Networking Group

  19. Per-flow Queueing • Let us assume we have a queue for each flow; and • Server those queues in a round robin manner. • Does this solve the problem? High Performance Networking Group

  20. Per-flow Buffering High Performance Networking Group

  21. Time t + RTT Per-Flow Buffering Temporarily Idle • Flow 3, does not have a packet at time t; Flows 1 and 2 do. • At time t+RTT we will see a drop. Time t High Performance Networking Group

  22. Ideal Solution • If packets are spaced out perfectly; and • The starting times of flows are chosen randomly; • We only need a small buffer for contention resolution. High Performance Networking Group

  23. M/M/1 r 1 X Buffer B P(Q > b) Packet Loss b Randomization • Mimic an M/M/1 queue • Under low load, queue size is small with high Probability • Loss can be bounded High Performance Networking Group

  24. TCP Pacing • Current TCP: • Send packets when ACK received. • Paced TCP: • Send one packet every W/RTT time units. • Update W, and RTT similar to TCP High Performance Networking Group

  25. CWND: Reno vs. Paced TCP High Performance Networking Group

  26. TCP Reno: Throughput vs. Buffer Size High Performance Networking Group

  27. Paced TCP: Throughput vs. Buffer Size High Performance Networking Group

  28. Early Results Congested core router with 10 packet buffers. Average offered load = 80% RTT = 100ms; each flow limited to 2.5Mb/s source source >10Gb/s >10Gb/s router 10Gb/s server High Performance Networking Group

  29. What We Know Arbitrary Injection Process Complete Centralized Control If Poisson Processwith load < 1 Theory Experiment Constant fractionthroughput with constant buffers [Leighton] Need buffersize of approx: O(logD + logW) i.e. 20-30 pkts D=#of hops W=window size [Goel 2004] Any rate > 0 need unboundedbuffers TCP Pacing: Results as goodor better than forPoisson High Performance Networking Group

  30. Limited Congestion Window PACING RENO Unlimited Window Limited Window High Performance Networking Group

  31. Slow Access Links Congested core router with 10 packet buffers. RTT = 100ms; each flow limited to 2.5Mb/s source source 5Mb/s 5Mb/s router 10Gb/s server High Performance Networking Group

  32. Conclusion • We can reduce 1,000,000 packet buffers to 10,000 today. • We can “probably” reduce to 10-20 packet buffers: • With many small flows, no change needed. • With some large flows, need pacing in the access routers or at the edge devices. • Need more work! High Performance Networking Group

  33. Extra Slides High Performance Networking Group

  34. S1 D 1 2 3 4 S2 Pathological Example • Flow 1: S1  D; Load = 50% • Flow 2: S2  D; Load = 50% • If S1 sends a packet at time t, S2 cannot send any packets at time t, and t+1. • To achieve 100% throughput we need at least one unit of buffering. High Performance Networking Group

More Related