Topic 3 Network Layer - Adapting Routing to the Traffic Part C2

1. Topic 3Network Layer - Adapting Routing to the TrafficPart C2 The majority of the slides in this course are adapted from the accompanying slides to the books by Larry Peterson and Bruce Davie and by Jim Kurose and Keith Ross. Additional slides and/or figures from Jennifer Rexford are also included in this presentation. Network Layer

2. Goals of Today’s Lecture • Challenges • Reacting quickly to alleviate congestion • Avoiding over-reacting and causing oscillations • Limiting bandwidth & CPU overhead on routers • Load-sensitive routing • Routers adapt to link load in a distributed fashion • At the packet level, or on “group of packets” • Traffic engineering • Centralized computation of routing parameters • Network-wide measurements of offered traffic Network Layer

3. TCP congestion control Senders react to congestion Decrease sending rate But… the TCP sessions receive lower throughput IP routing protocols Routers react to failures Compute new paths But… the new paths may be congested Do IP Networks Manage Themselves? 2 2 1 1 4 4 1 1 3 3 2 2 1 1 5 5 4 4 3 3 Network Layer

4. Do IP Networks Manage Themselves? • In some sense, yes: • TCP senders send less traffic during congestion • Routing protocols adapt to topology changes • But, does the network run efficiently? • Congested link when idle paths exist? • High-delay path when a low-delay path exists? 2 2 1 1 4 4 1 1 3 3 2 2 1 1 5 5 4 4 3 3 Network Layer

5. Adapting the Routing to the Traffic • Goal: modify the routes to steer traffic through the network in most effective way • Approach #1: load-sensitive protocols • Distribute traffic & performance measurements • Routers compute paths based on load • Approach #2: adaptive management system • Collect measurements of traffic and topology • Management system optimizes the parameters Network Layer

6. Outline: Three Alternatives • Load-sensitive routing at packet level • Routers receive feedback on load and delay • Routers re-compute their forwarding tables • Fundamental problems with oscillation • Load-sensitive routing at circuit level • Routers receive feedback on load and delay • Router compute a path for the next circuit • Less oscillation, as long as circuits last for a while • Traffic engineering as a management problem • Routers compute paths based on “static” values • Network management system sets the parameters • Acting on network-wide view of traffic and topology Network Layer

7. Load-Sensitive Routing Protocols • Advantages • Efficient use of network resources • Satisfying the performance needs of end users • Self-managing network takes care of itself • Disadvantages • Higher overhead on the routers • Long alternate paths consume extra resources • Instability from out-of-date feedback information Network Layer

8. Packet-Based Load-Sensitive Routing • Packet-based routing • Forward packets based on forwarding table • Load-sensitive • Compute table entries based on load or delay • Questions • What link metrics to use? • How frequently to update the metrics? • How to propagate the metrics? • How to compute the paths based on metrics? Network Layer

9. Original ARPANET Algorithm (1969) • Routing algorithm • Shortest-path routing based on link metrics • Instantaneous queue length plus a constant • Distributed shortest-path algorithm (Bellman-Ford) 2 1 3 1 3 2 1 5 20 congested link Network Layer

10. Performance of ARPANET Algorithm • Light load • Delay dominated by transmission & propagation • So, link metrics don’t fluctuate much • Medium load • Queuing delay is no longer negligible • Moderate traffic shifts to avoid congestion • Heavy load • Very high metrics on congested links • Busy links look bad to all of the routers • All routers avoid the busy links • Routers may send packets on longer paths Network Layer

11. Lincoln Tunnel NJ NYC Holland Tunnel “Backup at Lincoln” on radio triggers congestion at Holland Problem: Out-of-Date Information • Routers make decisions based on old information • Propagation delay in flooding link metrics • Thresholds applied to limit number of updates • Old information leads to bad decisions • All routers avoid the congested links • … leading to congestion on other links • … and the whole things repeats Network Layer

12. Problem: Frequent Updates • Update messages • Link keeps track of its metric (e.g., queuing delay) • Link transmits updates when the metric changes • Frequency of updates • Frequent changes to the metric lead to frequent updates • Significantly increases the overhead of the protocol • Oscillation makes the problem worse • Oscillation leads to wild swings in the link metrics • Forcing very frequent update messages • … that add to the load on the links in the network Network Layer

13. Second ARPANET Algorithm (1979) • Link-state protocol • Old: Distributed path computation leads to loops • New: Better to flood metrics and have each router compute the shortest paths • Averaging of the link metric over time • Old: Instantaneous delay fluctuates a lot • New: Averaging reduces the fluctuations • Reduce frequency of updates • Old: Sending updates on each change is too much • New: Send updates if change passes a threshold Network Layer

14. Problem of Long Alternate Paths • Picking alternate paths • Long path chosen by one router consumes resource that other packets could have used • Leads other routers to pick other alternate paths • Solution: limit path length • Bound the value of the link metric • “This link is busy enough to go two extra hops” • Extreme case • Limit path selection to the shortest paths • Pick least-loaded shortest path in the network Network Layer

15. Load-Sensitive Routing • Timescales • What timescale of routing decisions? • What timescale of feedback about link loads? • Load-sensitive routing at packet level • Routers receive feedback on load and delay • Routers re-compute their forwarding tables • Fundamental problems with oscillation • Load-sensitive routing for groups of packets • Routers receive feedback on load and delay • Router compute a path for the next flow or circuit • Less oscillation, as long as circuits last for a while Network Layer

16. Reducing Effects of Out-of-Date Info • Send link metrics more often • But, leads to higher overhead • But, propagation delay is a fundamental limit • Make the traffic last longer • Route on groups of packets, rather than packets • Fewer routing decisions, and more accurate feedback • Groups of packets • Telephone network: phone call (3-minutes long) • Internet: TCP connection (10-packets long) • Internet: all traffic between a pair of hosts, or routers, … Network Layer

17. Quality-of-Service Routing on Circuits Network Layer

18. Quality-of-Service Routing With Circuit Switching • Traffic performance requirement • Guaranteed bandwidth b per connection • Link resource reservation • Reserved bandwidth rion link I • Capacity ci on link i • Signaling: admission control on path P • Reserve bandwidth b on each link i on path P • Block: if (ri+b>ci) then reject (or try again) • Accept: else ri = ri + b • Routing: ingress router selects the path Network Layer

19. Source-Directed QoS Routing • New connection with b =3 • Routing: select path with available resources • Signaling: reserve bandwidth along the path (r = r +3) • Forward data packets along the selected path • Teardown: free the link bandwidth (r =r -3) r=8, c=10 r=6, c=7 b=3 r=1, c=5 r=15, c=20 Network Layer

20. QoS Routing: Path Selection • Link-state advertisements • Advertise available bandwidth (ci –ri ) on link i • E.g., every T seconds, independent of changes • E.g., when metric changes beyond threshold • Each router constructs view of topology • Path computation at each router • E.g., Shortest widest path • Consider paths with largest value of mini(ci-ri) • Tie-break on smallest number of hops • E.g., Widest shortest path • Consider only paths with minimum hops • Tie-break on largest value of mini(ci-ri) over paths Network Layer

21. How To Get IP Packets on to Circuits? • Who initiates the circuit? • End system application or operating system? • Edge router? • Edge router can infer the need for a circuit • Match on packet header bits • E.g., source, destination, port numbers, etc. • Apply policy for picking bandwidth parameters • E.g., Web connections get 10 Kbps, video gets 2 Mbps • Trigger establishment of circuit for the traffic • Select path based on load and requirements • Signal creation of the circuit • Tear down circuit after an idle period Network Layer

22. Grouping IP Packets Into Flows • Group packets with the “same” end points • Application level: single TCP connection • Host level: single source-destination pair • Subnet level: single source prefix and dest prefix • Group packets that are close together in time • E.g., 60-sec spacing between consecutive packets flow 2 flow 4 flow 3 flow 1 Network Layer

23. But, Staleness Can Still Be a Problem… • Link state updates • High update rate leads to high overhead • Low update rate leads to oscillation • Connections are too short • Average Web transfer is just 10 packets • Requires high update rates to ensure stability • Idea: QoS routing only for long transfers! • Small fraction of transfers are very large • … and these few transfers carry a lot of traffic • Forward most transfers on static routes • … and compute dynamic routes for long transfers Network Layer

24. Identifying the Long Transfers • A nice property of transfer sizes • Most transfers are short, but a few are very long • Distribution of transfer sizes is “heavy tailed” • A nice property of heavy tails • After you see 10 packets, it is likely a long transfer • Even the remainder of the transfer is long • Routing policy • Forward initial packets on the static default route • After seeing 10 packets, try to signal a circuit • Forward the remaining packets on the circuit • Avoids oscillation even for small update rates • http://www.cs.princeton.edu/~jrex/papers/sigcomm99.ps Network Layer

25. Traffic Engineering as a Network-Management Problem: Case Study Network Layer

26. 2 1 3 1 3 2 1 5 4 3 Using Traditional Routing Protocols • Routers flood information to learn topology • Determine “next hop” to reach other routers… • Compute shortest paths based on link weights • Link weights configured by network operator Network Layer

27. Approaches for Setting the Link Weights • Conventional static heuristics • Proportional to physical distance • Cross-country links have higher weights • Minimizes end-to-end propagation delay • Inversely proportional to link capacity • Smaller weights for higher-bandwidth links • Attracts more traffic to links with more capacity • Tune the weights based on the offered traffic • Network-wide optimization of the link weights • Directly minimizes metrics like max link utilization Network Layer

28. Example of Tuning the Link Weights • Problem: congestion along the pink path • Second or third link on the path is overloaded • Solution: move some traffic to the bottom path • E.g., by decreasing the weight of the second link 2 1 3 1 3 2 3 1 5 4 3 Network Layer

29. Measure, Model, and Control Network-wide “what if” model Offered traffic Changes to the network Topology/ Configuration measure control Operational network Network Layer

30. Traffic Engineering Problem • Topology • Connectivity and capacity of routers and links • Traffic matrix • Offered load between points in the network • Link weights • Configurable parameters for routing protocol • Performance objective • Balanced load, low latency, service level agreements … • Question: Given the topology and traffic matrix, which link weights should be used? Network Layer

31. Key Ingredients of the Approach • Instrumentation • Topology: monitoring of the routing protocols • Traffic matrix: fine-grained traffic measurement • Network-wide models • Representations of topology and traffic • “What-if” models of shortest-path routing • Network optimization • Efficient algorithms to find good configurations • Operational experience to identify key constraints Network Layer

32. i j Formalizing the Optimization Problem • Input: graph G(R,L) • R is the set of routers • L is the set of unidirectional links • cl is the capacity of link l • Input: traffic matrix • Mi,j is load from router i to j • Output: setting of the link weights • wlis weight on unidirectional link l • Pi,j,lis fraction of traffic from i to j traversing link l Network Layer

33. 0.25 0.25 0.5 1.0 1.0 0.25 0.25 0.5 0.5 0.5 Multiple Shortest Paths: Even Splitting Values of Pi,j,l Network Layer

34. f(x) x 1 Defining the Objective Function • Computing the link utilization • Link load: ul = Si,j Mi,j Pi,j,l • Utilization: ul/cl • Objective functions • min (maxl(ul/cl)) • min(Slf(ul/cl)) Network Layer

35. Complexity of the Optimization Problem • Computationally intractable problem • No efficient algorithm to find the link weights • Even for simple objective functions • What are the implications? • Must resort to searching through weight settings Network Layer

36. Optimization Based on Local Search • Start with an initial setting of the link weights • E.g., same integer weight on every link • E.g., weights inversely proportional to capacity • E.g., existing weights in the operational network • Compute the objective function • Compute the all-pairs shortest paths to get Pi,j,l • Apply the traffic matrix Mi,j to get link loads ul • Evaluate the objective function from the ul/cl • Generate a new setting of the link weights repeat Network Layer

37. Making the Search Efficient • Avoid repeating the same weight setting • Keep track of past values of the weight setting • … or keep a small signature of past values • Do not evaluate setting if signatures match • Avoid computing shortest paths from scratch • Explore settings that changes just one weight • Apply fast incremental shortest-path algorithms • Limit number of unique values of link weights • Do not explore 216 possible values for each weight • Stop early, before exploring all settings Network Layer

38. Incorporating Operational Realities • Minimize number of changes to the network • Changing just 1 or 2 link weights is often enough • Tolerate failure of network equipment • Weights usually remain good after failure • … or can be fixed by changing 1-2 weights • Limit effects of measurement accuracy • Good weights remain good, despite noise • Limit frequency of changes to the weights • Joint optimization for day & night traffic matrices Network Layer

39. Application to AT&T’s Backbone • Performance of the optimized weights • Search finds a good solution within a few minutes • Much better than link capacity or physical distance • Competitive with multi-commodity flow solution • How AT&T changes the link weights • Maintenance every night from midnight to 6am • Predict effects of removing link(s) from network • Reoptimize the link weights to avoid congestion • Configure new weights before disabling equipment Network Layer

40. Example from AT&T’s Operations Center • Amtrak repairing/moving part of train track • Need to move some of the fiber optic cables • Or, heightened risk of the cables being cut • Amtrak notifies AT&T the timework will be done • AT&T engineers model the effects • Determine which IP links go over affected fiber • Pretend the network no longer has these links • Evaluate the new shortest paths and traffic flow • Identify whether link loads will be too high Network Layer

41. Example Continued • If load will be too high • Reoptimize the weights on the remaining links • Schedule time for new weights to be configured • Roll back to old weights when Amtrak is done • Same process applied to other cases • Assessing the network’s risk to possible failures • Planning for maintenance of existing equipment • Adapting link weights to installation of new links • Adapting link weights in response to traffic shifts Network Layer

42. Conclusions • Adapting routing to the traffic • To alleviate congestion • To minimize propagation delay • To be robust to future failures • Two main approaches • Load-sensitive routing protocol • Optimization of configurable parameters Network Layer