1 / 21

Hierarchy-Aware Distributed Overlays in Data Centers using DC2

Hierarchy-Aware Distributed Overlays in Data Centers using DC2. Karthik Nagaraj , Hitesh Khandelwal , Charles Killian, Ramana Rao Kompella Computer Science, Purdue University. Today’s Data Center Networks. Up to 240:1 Over-subscription. Core. Aggregation. ... . . ... . . ToR.

kohana
Download Presentation

Hierarchy-Aware Distributed Overlays in Data Centers using DC2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hierarchy-Aware Distributed Overlays in Data Centers using DC2 Karthik Nagaraj, Hitesh Khandelwal, Charles Killian, RamanaRaoKompella Computer Science, Purdue University

  2. Today’s Data Center Networks Up to 240:1 Over-subscription Core Aggregation ... ... ... ... ToR #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl #incl Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  3. Today’s Data Center Networks Internet • Canonical applications • Group communication systems: Multicast • (Social networks) • Distributed key-value stores • (E-commerce) ... ... ... ... ... ... ... ... ... ... ... ... • Wide-area links • Latency high • Bandwidth low Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  4. Latency Sensitive Routing Scribe overlay multicast (latency-aware) Latency ≠ Location Nearby subscribers unknown No low latency next hop in overlay ... ... ... ... Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  5. Latency Sensitive Routing Ideal overlay tree ... ... ... ... Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  6. Overview • DC2 design • Data Center aware Distributed Communication • Two distributed applications: • DC2-Multicast • DC2-Store • Prototype evaluation • Conclusion Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  7. DC2 Design • Goals • Limit traffic within close proximity - hierarchy • Provide scalable overlay properties • Components: • Location identifiers • Cluster Coordinators • Overlay routing Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  8. Rendezvous Tree Routing • Hash topic name  group key • Route to group key on overlay • Pastry, Chord Rendezvous node Key: SHA160/122a4a9fd2… Intermediate nodes automatically added Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  9. Location Identifiers • Hierarchical Internet Datacenter ID Rack ID Host ID • How we get them • How we use them • Location bit sequence • Prefix node’s hash ... ... ... ... ... ... ... ... ... ... ... ... Rack ID Host ID Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  10. Cluster Coordinators Rendezvous node • Random rendezvous points • Each cluster 0x122a… 0x522a… Rack cluster 0x512a… Location-aware routing Datacenter cluster Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  11. DC2-Multicast • Location-aware group multicast • Two key ideas: • Contain content distribution within cluster • Avoid forwarder nodes • Forwarder nodes forced into tree • Stress on nodes high Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  12. DC2-Multicast Rendezvous node • Coordinators provide group lookup • Basic random tree protocol Rack cluster • Rack: 20-40 machines • Datacenter: 100s of racks Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  13. DC2-Store • Reverse path caching Rendezvous node Traditional Caching DHT Cache Rack coordinator cache Increasing distance Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  14. Evaluation • Deployable Prototype implementation • Using Mace • LOC: 1.1k Mace C++; 22k generated C++ • ModelNet emulated data center topology • 700 nodes • 15 physical machines • Intermediate multicast solution • DC2-Scribe Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  15. DC2-Multicast Performance • Workload: zipf-ian group sizes, 10 pkts/sec • Latency: Application perceived performance Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  16. DC2-Multicast Performance • Link Stress: Datacenter network congestion Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  17. DC2-Store Performance • Key-Value lookup latency Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  18. Conclusions • Data center networks over-subscribed • DC2: Explicitly, transparently leverage location • Framework for Distributed communication • Scalable • High throughput • Low latency Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  19. Backup Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  20. Oversubscription • DC2 is a software approach to alleviate condition • Previous hardware approaches • Fat-tree [Alfares08], VL2 [Greenberg09] • Requires changes to existing topologies • Not yet widely used in data centers • Across DCs still an issue Hierarchy-Aware Distributed Overlays in Data Centers using DC2

  21. RandTree protocol • Simple tree management • Efficient on small groups • Assumes lookup service Root Hierarchy-Aware Distributed Overlays in Data Centers using DC2

More Related