1 / 43

Internet Inter-Domain Traffic

Internet Inter-Domain Traffic. Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, Farnam Jahanian. Presented by: Kaushik Choudhary. Outline. Introduction Data Collection Methodology ASN Traffic Analysis Application Traffic Analysis Internet Size Estimates Conclusion.

lelia
Download Presentation

Internet Inter-Domain Traffic

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Inter-Domain Traffic Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, Farnam Jahanian Presented by: Kaushik Choudhary

  2. Outline • Introduction • Data Collection Methodology • ASN Traffic Analysis • Application Traffic Analysis • Internet Size Estimates • Conclusion

  3. Introduction • “The Internet has changed dramatically over the last five years” – Cliché. • Internet traffic has gone over the roof due to:

  4. The changes • Content providers (like Google) build their own global backbones. • Cable internet service providers (ISPs) offer wholesale national transit. • Transit ISPs offer content distribution networks (CDNs).

  5. The changes Transition from: Fig 1: Traditional Internet logical topology.

  6. The changes to: Fig 2: Emerging new Internet logical topology.

  7. What is new in this paper • Most studies about traffic have focused on BGP route advertisements, DNS probing, industry surveys, private CDN statistics etc. • In this paper, the authors studied over 3000 peering edge routers of 110 participating internet providers over two years.

  8. Outline • Introduction • Data Collection Methodology • ASN Traffic Analysis • Application Traffic Analysis • Internet Size Estimates • Conclusion

  9. Data Collection Methodology • Data collected by probes of a commercial security and traffic monitoring platform instrumenting BGP edge routers. • Each probe exports traffic flow statistics that includes traffic per BGP AS, ASPath, network and transport layer protocols, countries, etc. independently to form different datasets.

  10. Data Collection Methodology Fig 3: Netflow network probes.

  11. Challenges in Data Collection • Commercial privacy concerns. • Misconfiguration of probes (resulting in wild fluctuations in daily traffic). • Decommissioning of edge routers! • Decommissioning of older probes and addition of new ones!

  12. Need for Data Aggregation • Wild fluctuations in the data. • Heterogeneity of the providers. • Ratios and percentages were consistent despite the fluctuations.

  13. Data Aggregation • The probes calculated average traffic volume every five minutes for all members of all datasets throughout every 24 hour period. • They also calculated the average volume of total inter-domain network traffic. • Finally, the daily traffic volume per item and network total were used to calculate the daily percentage.

  14. Data Aggregation • Calculation of weights: where, = Weight of participant i on day d. = Router count for participant i on day d. N = All study participants.

  15. Data Aggregation • Calculation of weighted average percentage: where, = Weighted average percent share of traffic for A on day d. (𝐴) = Each provider’s measured average traffic volume for A on day d. = Total average inter-domain traffic for day d. A = Traffic attribute like ASN, TCP port, country of origin, etc.

  16. Data Defects and Validation • The probes did not detect traffic over peering adjacencies between enterprise business partners or in cases of similar agreements. • The data collected was validated through private discussions with large content providers, transit ISPs and regional networks.

  17. Outline • Introduction • Data Collection Methodology • ASN Traffic Analysis • Application Traffic Analysis • Internet Size Estimates • Conclusion

  18. ASN Traffic Analysis Table 1: Traffic contributors by weighted average percentage in July 2007.

  19. ASN Traffic Analysis Table 2: Traffic contributors by weighted average percentage in July 2009.

  20. Trends • Data from July 2007 is compliant with textbook diagrams of internet topology. • Changes in commercial policy and traffic engineering have drastically impacted shares in Internet inter-domain traffic.

  21. Trends Table 3: Providers with most significant inter-domain traffic share growth in 2007-2009.

  22. Google Trends • Google, a content provider, now rivals global transit networks and enjoyed the highest growth. • Providers and the data collected suggest that Google’s huge growth may be ascribed to the acquisition of Youtube.

  23. Google Trends Fig 4: Google inter-domain traffic contribution.

  24. Comcast Trends Consumer Content Fig 5: Comcast inter-domain traffic contribution.

  25. Comcast Trends • Majority of the traffic growth in Comcast came from transit traffic. • What did Comcast do? • Consolidated regional backbones into a single nationwide network. • Rolled out a consumer product called “triple play” (voice, video, data). • Began offering wholesale transit, cellular backhaul and IP video distribution!

  26. Outline • Introduction • Data Collection Methodology • ASN Traffic Analysis • Application Traffic Analysis • Internet Size Estimates • Conclusion

  27. Application Traffic Analysis • Applications could be identified from TCP/UDP port numbers but: • Applications could use non-standard ports. • Port-based classification only accounts the control traffic and not the often random port numbers associated with subsequent data transfer. • The authors obtained validation data from five cooperating provider deployments in Asia, Europe and North America.

  28. Largest Applications Table 4: Top applications by weighted average percentage.

  29. Largest Applications Table 5: Average application breakdown in July 2009 across five consumer providers.

  30. Application Traffic Changes Fig 6: Distribution of weighted average percentage of traffic from well known ports (July).

  31. Application Traffic Changes • TCP and UDP combined account for more than 95% of all inter-domain traffic. • In July 2007, 52 ports contributed 60% of traffic. • In July 2009, 25 ports contributed 60% of traffic. • What’s happening here?

  32. Application Traffic Changes • Internet application traffic is getting migrated to a smaller set of ports and protocols. • Microsoft migrated all Xbox Live traffic to use port 80 on June 16, 2009. • One reason for this is that majority of firewalls allow HTTP traffic. • The other reason for this trend is the dominance of

  33. Applications Exhibiting Growth • Much of the growth in web data is due to video (Youtube uses progressive downloading). • Growth in video traffic is due to the introduction of services like Hulu, Youtube, Veoh, etc.

  34. Applications Exhibiting Growth Obama! Fig 7: Change in weighted average percentage of video protocols.

  35. Applications Exhibiting Decline • P2P dropped by 2.8% between 2007-2009. • Possible reasons • Migration to tunnelled overlays. • Use of P2P encryption. • Stealthier P2P clients and algorithms. • Network operators however, suggest P2P traffic may have migrated to alternatives like direct download and streaming video.

  36. Outline • Introduction • Data Collection Methodology • ASN Traffic Analysis • Application Traffic Analysis • Internet Size Estimates • Conclusion

  37. Internet Size Estimates Fig 8: Independent inter-domain traffic volumes vs calculated aggregate ASN share..

  38. Internet Size Estimates • The authors solicited verification data from twelve large providers and content sites. • The slope of the above curve means that a 2.51% of share of all traffic means 1 Tbps of traffic. Extrapolating this way, • This was in July 2009 (before the release of iPad!).

  39. Internet Size Estimates • In other analyses, the authors report the annual growth rate of the internet to be about 44.5% and the monthly traffic volume to be about 9 exabytes ( = 9 million terabytes!)

  40. Outline • Introduction • Data Collection Methodology • ASN Traffic Analysis • Application Traffic Analysis • Internet Size Estimates • Conclusion

  41. Conclusion • This paper is the first longitudinal study of Internet inter-domain traffic. • It identifies a significant ongoing evolution of provider interconnection strategies from a hierarchical to a more densely connected model. • Most internet content is migrating to a relatively small number of hosting, cloud and content providers.

  42. Conclusion • Google is the largest contributor to Internet traffic growth. • Majority of inter-domain traffic has migrated to a relatively small number of ports and protocols. • As of July 2009, the inter-domain traffic peaks exceeded 39 Tbps and were growing at 44.5% annually.

  43. Questions?

More Related