The Impact of False Sharing on Shared Congestion Management

1. The Impact of False Sharing on Shared Congestion Management Srinivasa Aditya Akella Joint work with Srini Seshan and Hari Balakrishnan 28 Feb, 2001

2. Introduction • Predominant model for congestion control • Slow-start • AIMD • Not always optimal • Multiple concurrent flows from Src to Dest may share a bottleneck • Compete for resources rather than co-operate • Especially visible in the context of Web transfers

3. Sharing Congestion Information... • Solution - share congestion information • Granularity of sharing • Common destination host (network interface) • All destination hosts on the same IP subnet • Set of flows sharing congestion info - macroflow

4. False Sharing • Flows sharing congestion state might not share the same bottleneck • Sender has no knowledge • False sharing in the Internet • Flows are treated differently- Service Differentiation • Flows take different paths - Path Diversity

5. False Sharing • Service Differentiation • Integrated Services • Differentiated Services (DiffServ) • Path Diversity • Network Load Balancers • Network Address translators (NATs)

6. Questions... • Impact on performance and correctness • Compromise to end-to-end congestion control? • Degradation in performance of individual flows? • Detection • Under what conditions can false-sharing be detected? • Response • How should congestion sharing systems be modified? • What effect do these modifications have? • What should be the default behavior?

7. Quantifying the Penalty XXX needs to be fixed • Analysis • False sharing reduces observed flow throughput • l_share = l_1 l_2 / (l_1 + l_2) • False sharing increases observed flow loss rate • r_noshare = sqrt(r_1 r_2) • r_share = (r_1 + r_2)/2

8. Service Differentiation • Network treats different flows differently • Bandwidth allocation and buffer resources • IETF DiffServ architecture • Three PHBs : Assured Forwarding, Expedited Forwarding, Best Effort • Nortel's implementation of Diffserv • Experiments with two traffic classes : AF and BE • WRR for bandwidth sharing • RIO (for AF) and RED (for BE) for buffer management • Styles of buffer management • Shared and unshared

9. Topology for Diffserv

10. Results... • Predicted throughput = XXX need to fill • The faster connection is slowed down by the slower one • Slower connection is never persistently overloaded • Loss rate for the slower connection does not increase appreciably with sharing

11. Path Diversity • Two flows taking different routes may not share a bottleneck • Two scenarios where path diversity leads to false sharing • Dispersity Routing • NATs • Three distinct categories • Unshared bottleneck • No shared bottleneck link • Semi-shared bottleneck • One of the unshared paths has a bottleneck • Fully shared bottleneck • No bottlenecks in the unshared portions • RTTs would be different

12. Topology for Unshared Bottleneck

13. Results for Unshared-Bottleneck • Bandwidth is close to the prediction • Loss rates followed similar pattern as with the DiffServ case

14. Delays and Losses... • Delays vary independently of each other • Losses are uncorrelated • Variations and delays in losses in one flow are more correlated than those across flows

15. Path Diversity, Other Cases

16. Fully Shared Bottleneck - How is it Different? • Variations in delay seem correlated • The two flows share a common point of congestion • The flows should not share congection information

17. Detection • Test description • Rubenstein's Delay and Loss Correlation tests • Need modifications to be a part of the architecture • Flows might undergo false-sharing if even one of their bottlenecks is unshared • Two differentially served flows might observe statistically dependent delays • Scheduler at the sender might apportion bandwidths non-uniformly • Congestion control schemes depend on RTTs • Aggregating flows with different RTTs would lead to false sharing

18. Loss-correlation Test • Idea -- Losses are likely to come in bursts • This should hold across flows from the same source when a bottleneck is shared • Rubenstein's tests compare the auto and cross correlation metrics for pairs of flows • Does not detect unshared bottlenecks • Need a test to detect all if all bottlenecks are shared • New test - Symmetric Loss Correlation • Loss and cross correlation metrics defined in a manner independent of the flows solves the problem • However, packets across flows are assumed to be spaced closer than those within a flow -- Not always true • A fix -- Schedule transmmissions appropriately

19. Delay-correlation Test • Delay = f(propagation time, queueing delay) • Queueing delay (Q)can vary significantly with time • Current Q is strongly related to recently values • Challanges with measuring delay • Clocks cannot be easily synchronized • Use change in delay or the relative delay • Methodology of the tests • Use timestamps to compute delays • Compute correlations • Correlation is independent of constant differences

20. Out-of-Order Test • Flows might have fundamentally different delays • DelayCorr does not identify this • Loss and Delay tests might help detect false-sharing • MultiPath Routing where bottleneck is shared • Out-of-Order test handles this well • Look at packet reordering from a source • Reordering by more than 3 packets => No sharing • Limitation: Packets must be delivered to the same physical destination • Cannot be applied to situations like NAT • Rely on RTTs in such situations

21. Genuine Sharing is Harder to Detect

22. Evaluation of the Tests • Two metrics for each tests • Detection time • Probability of correct decision • Which test is the best? • Out-of-order tests are mostly accurate • Loss tests are neither timely nor accurate • Delay tests are timely but not as accurate • Symmetric Loss test ouputs correct result much more often than the asymmetric test

23. Response to False Sharing • Design Issues • Default behavior: share information and detect false-sharing • Scheduling • False sharing detected more easily than genuine sharing • Default of no-sharing makes no sense with out-of-order tests • Upon detection, stop sharing • In CM, associate the different flows to different macroflows • Relatively small confidence intervals can be used • No significant penalty due to an incorrect decision

24. Performance • How good can restoration possibly be? • False sharing may penalize flows significantly • It might take time to restore performance • However, the greater the penalty, the easier it is to detect • Approach to performance evaluation -- multiple, de-randomized, offline runs • Performance restored in less then a factor of 3 of time taken to detect