Designing a Predictable Backbone Network with Valiant Load Balancing

1. Designing a Predictable Backbone Network with Valiant Load Balancing Nick McKeown Stanford University All the hard stuff was done by: Rui Zhang-Shen Clean Slate Design for the Internet http://cleanslate.stanford.edu NSF 100 x 100 Clean Slate Program http://100x100network.org

2. 2 1 3 5 4 Backbone network Backbone network design using VLB

3. US Backbone Networks: Observations • ~50 nodes interconnected by long-haul optical links • Increasingly rich mesh topology • Built over mesh of WDM or TDM circuits and switches • Reduce hop count and delay • Fault tolerance • Load balancing • Low utilization—links over-provisioned • Uncertainty in traffic matrix the network is designed for • Headroom for future growth • Prepare to take over when links or routers fail • Minimize congestion and delay variation Backbone network design using VLB

4. Back-bone 2 1 Regional Node i 3 ri 5 ri 4 ri is easier to predict and has to be predicted anyway Traffic Matrices To From Traffic matrix is hard to predict Backbone network design using VLB

5. 80% utilization: 0.0003% 67% utilization: 1.1% 80% utilization: 0.026% 67% utilization: 0.66% 80% utilization: 0.00008% 67% utilization: 0.09% 80% utilization: 0.0009% 67% utilization: 0.026% What fraction of traffic matrices can they support? Verio Abilene Sprint AT&T Verio, AT&T, and Sprint topologies courtesy of RocketFuel Backbone network design using VLB

6. Desired Characteristics • Dependable • Continues to operate when traffic patterns change in the short and long term • Continues to operate under failure • Recovers quickly • Efficient • And at no extra cost Backbone network design using VLB

7. r r r r r Why is this hard? r r 1 2 r 3 N r 4 … r r Backbone network design using VLB

8. r r r r r Nr Why is this hard? r r 1 2 r 3 N r 4 … r r Backbone network design using VLB

9. Our Approach • The operator already estimates ri • Requires only local knowledge of users and market estimates • Use Valiant Load Balancing (VLB) • Supports all traffic matrices • History • L. G. Valiant, G. Brebner, 1981-82 • Parallel communication • Statistical delay guarantee • C.-S. Chang, etc.; I. Keslassy etc., 2001-05 • Switch scheduling • Throughput guarantee • Optimality Backbone network design using VLB

10. Valiant Load-Balancing r r 1 2 r 3 N r 4 … r r Backbone network design using VLB

11. 1 2 3 N 4 … Valiant Load-Balancing 2r/N r r r r 4 2 • In practice: • The mesh could be a mesh of lambdas or TDM circuits • Send on direct path, and only spread when network is congested. r r Backbone network design using VLB

12. Aside: Routers based on VLB Can you build a router switched backplane based on VLB? Appealing possibilities 100% throughput for any arrival pattern No per-packet arbitration and scheduling Passive switch fabric consumes almost zero power Switch Rack < 100W Linecards Linecards Linecards 40 x 40 MEMS 1 2 55 56 “Scaling Routers using Optics” Sigcomm 2003 Backbone network design using VLB

13. 1 2 3 N 4 … Failures • Node failures • Takes away corresponding links and traffic • Still a full mesh network • Links failures • Asymmetric network • Many scenarios Backbone network design using VLB

14. Fault Tolerance • Load balance traffic over available paths • To tolerate any k link or router failures, sufficient to increase the capacity each link by • Example: A 50 node network requires 11% more capacity to withstand any 5 failures. Backbone network design using VLB

15. 1 2 3 N cij rirj 4 … Heterogeneous Network R= iri r2 r1 Homogeneous: c = 2r/N rN r3 cij = 2rirj /R r4 ri Gravity Configuration Backbone network design using VLB

16. Heterogeneous Network • As before, the total capacity we need with VLB is twice what we’d need if we knew the traffic matrix (and it was static). • With oblivious routing we need an extra capacity. Backbone network design using VLB

17. Is VLB efficient? • Not knowing the traffic matrix means we need a total capacity 2-times larger than if we did. • But we never know the traffic matrix, and it changes. So the cost is surprisingly small. • Anecdotally, a network that can support all traffic matrices and behaves predictably on failure requires less capacity than existing networks. Backbone network design using VLB

18. Rp q2 q3 2 1 2 1 q4 N 3 N 3 4 … 4 … Interconnecting Backbones • Peering parameters: • Rp is maximum peering traffic • qi¸0 for peering nodes, qi=0 for non-peering nodes, iqi=1 • Peering link capacity: Rpqi Backbone network design using VLB

19. Within a VLB Network • Assume peering condition is fixed • Given: Rp ; qi • Variables: pi • Spread traffic over the peering links Backbone network design using VLB

20. 1 2 2 1 3 N N 3 … 4 … 4 Spread over peering links 1 1 1 cij = ri pj + rj pi + min(ri,Rp)(max(pj,qj)-pj) + min(rj,Rp)(max(pi,qi)-pi) • If Rp >ri, optimal solution: pi = qi ;cij = ri qj + rj qi • Efficient use of peering links • Supports all traffic matrices as before Backbone network design using VLB

21. Other questions • Delay-sensitive applications • How much does it matter? • It may matter for interactive voice, video, gaming • Dealing with it: “Express paths” ,Adaptive load-balancing Backbone network design using VLB