1 / 20

Evaluating the Running Time of a Communication Round over the Internet

Evaluating the Running Time of a Communication Round over the Internet. Omar Bakr Idit Keidar MIT MIT/Technion PODC 2002. Communication Round. Exchange of information from all hosts to all hosts Part of many distributed algorithms, systems

cachez
Download Presentation

Evaluating the Running Time of a Communication Round over the Internet

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Evaluating the Running Time of a Communication Round over the Internet Omar Bakr Idit Keidar MIT MIT/Technion PODC 2002

  2. Communication Round • Exchange of information from all hosts to all hosts • Part of many distributed algorithms, systems • consensus, atomic commit, replication, ...

  3. Common Metric for Evaluating Algorithms • Number of rounds (or steps) they require

  4. Questions • What is the best way to implement a communication round over the Internet • decentralized vs. centralized • How long is a communication round over the Internet?

  5. Prediction is Hard • Internet is unpredictable, diverse, … • Different answers for different topologies, different times • Different performance metrics • local runningtimeone host is engaged in algorithm • overall runningtimefrom when first host starts to when last host finishes

  6. “Communication Round” Primitive • Initiated by some host • Propagates data from every host to every other host connected to it

  7. All-to-all Leader Secondary Leader Example Implementations

  8. Experiment I • 10 hosts: Taiwan, Korea, US academia, ISPs • TCP/IP (connections always up) • Algorithms: • All-to-all • Leader (initiator) • Secondary leader (not initiator) periodically initiated at each host - 650 times over 3.5 days

  9. Computing Overall Running Time • Elapsed time from initiation (at initiator) until all hosts terminate • Requires estimating clock differences • Clocks not synchronized, drift • We compute difference over short intervals • Compute 3 different ways • Achieve accuracy within 20 ms. on 90% of runs

  10. All-to-all: 2 Leader: 3 Secondary Leader: 4 Teaser: Comparing Performance Based on Number of Steps

  11. 150+240 = 390 150+150+150 = 450 Predicting Overall Runnig Times From MIT • Ping-measured latencies (IP): • Longest link latency 240 milliseconds • Longest link to MIT 150 milliseconds

  12. Measured Running Times Runs Initiated at MIT Running times in milliseconds

  13. What’s Going On? • Loss rates on two links are very high • 42% and 37% • Taiwan to two ISPs in the US • Loss rates on other links up to 8% • Upon loss, TCP’s timeout is big • More than round-trip-time • All-to-all sends messages on lossy links • Often delayed by loss

  14. Distribution of Running Times Up to 1.3 sec. at MIT

  15. All-to-all overall local Leader overall local Sec. Leader overall local Average (runs under 2 sec) 866 645 1120 844 679 607 % runs over 2 seconds 54% 24% 64% 43% 13% 7% Running Times Runs Initiated at Taiwan Running times in milliseconds

  16. Distribution of Running Timesin Taiwan

  17. Taiwan MIT Hosts with bad links to Taiwan Leader Secondary Leader What’s Going On? Good link Lossy link Other Hosts All-to-all

  18. Experiment II: Removing Taiwan • Overall running times much better • For every initiator and algorithm, less than 10% over 2 seconds (as opposed to 55% previously) • All-to-all overall still worse than others! • either Leader or Secondary Leader best, depending on initiator • loss rates of 2% - 8% are not negligible • all-to-all sends O(n2) messages; suffers • But, all-to-all has best local running times

  19. Probability of Delay due to Loss • If all links would have same latency • assume 1% loss on all links; 10 hosts (n=10) • Leader sends 3(n-1) = 27 messages • probability of at least one loss: 1 -.9927 »24% • All-2-all sends n(n-1) = 90 messages • probability of at least one loss: 1 -.9990 »60% • In reality, links don’t have same latency • only loss on long links matters

  20. Conclusions • Message loss causes high variation in TCP link latencies • latency distribution has high variance, heavy tail • Latency distribution determines expected time for receiving O(n) concurrent messages • Secondary leader helps • No triangle inequality, especially for loss • Different for overall vs. local running times • Number of rounds/steps not sufficient metric • One-to-all and all-to-all have different costs

More Related