Helping TCP Work at Gbps

1. Helping TCP Work at Gbps Cheng Jin the FAST project at Caltech http://netlab.caltech.edu/FAST

2. Talk Outline • TCP Reno does not perform well at Gbps • TCP protocol has stability problems • How FAST improves stability of TCP • Project status

3. High Energy Physics • Clear and present need for high bandwidth • Global cooperation • 2000 physicists from 150 institutions in the world • 300 - 400 physicists in US from > 30 universities and labs • Large file transfers ~ 1 TB • At 622 Mbps ~ 4 hrs • At 2.5 Gbps ~ 1 hr • At 10 Gbps ~ 15 minutes

4. Highspeed TCP Performance 622 Mbps 155 Mbps 2.5 Gbps 5 Gbps 10 Gbps ns-2: 100 sources, 100 ms round trip propagation delay J. Wang (Caltech)

5. TCP Window Evolution FAST TCP/RED ns-2: capacity = 10 Gbps J. Wang (Caltech)

6. Current TCP Protocol • Stability problems: • Slow-timescale oscillation as delay or capacity increases • Independent of packet-level AIMD dynamics • Independent of network noise

7. 800 700 600 500 400 300 200 100 0 0 8000 2000 4000 6000 10000 Stable: Small (20 ms) Delay 70 60 50 40 individual window average window 30 20 10 0 2000 4000 6000 8000 10000 0 Window Instantaneous Queue 50 identical FTP sources, single link 9 pkts/ms, RED marking

8. 800 700 600 500 400 300 200 100 0 0 2000 4000 6000 8000 10000 Unstable: Large (200ms) Delay 70 individual window 60 50 40 30 20 10 average window 0 2000 4000 6000 8000 0 10000 Congestion Window Instantaneous Queue 50 identical FTP sources, single link 9 pkts/ms, RED marking

9. mean RTT 16ms same RTT 20ms mean RTT 208ms Other Effects on Queue Length 30% noise 30% noise same RTT 200ms

10. Stability Regions 100 95 N = 60 Unstable for • Large delay • Large capacity • Small load 90 N = 50 85 80 N = 40 delay (ms) 75 N = 30 70 65 N = 20 60 55 50 8 9 10 11 12 13 14 15 capacity (pkts/ms)

11. TCP Reno uses loss as congestion measure Loss becomes noisy as capacity increases TCP’s increase and decrease of cwnd not adaptive to system response FAST can use either queueing delay or loss Queueing delay has the right scaling with respect to capacity FAST adapts to capacity or end-to-end delay Loss vs.Delay

12. FAST: Fast AQM Scalable TCP • If loss is used • Both sender TCP and router AQM need to be changed • If queueing delay is used • Only sender TCP needs to be changed • Injecting x ms of queueing delay into the network and change the send rate based on the observed queueing delay and its rate of change

13. Project Status • Designed improved TCP/AQM protocols with the right scaling • Compare FASTtoexisting approaches for highspeed TCP • Linux kernelimplementation of FAST • Router implementation of AQM • Experiments on “real” networks

14. Linux Kernel Implementation • RedHat 7.3 with 2.4.18 kernel • Modifications to the TCP layer • Monitoring tool as loadable kernel module • Incorporate features such as tunable socket buffer size and MTU

15. Floyd’s Highspeed TCP • Slow increase of congestion window requires extremely small loss probability • Tuning TCP’s AIMD window adjustment • More rapid increase of cwnd • Less aggressive reduction of cwnd • More simulations/experiments are needed

16. Equation-Based Approach • Remove packet-level AIMD effect • Estimate congestion signal (usually packet loss) and compute transmission rate • Needs the right scaling with respect to delay and capacity

17. Effect of Protocol Instability • Large jitters, bad for real-time traffic • Creating bursty queues, causing packet losses • Lower network utilization at high speed