1 / 57

Affinity in Distributed Systems

Affinity in Distributed Systems. Ph.D . thesis d efense . August 21, 2009. Ý mir Vigfússon. Joint work with: Hussam Abu- Libdeh , Mahesh Balakrishnan , Ken Birman , Gregory Chockler , Qi Huang, Jure Leskovec , Deepak Nataraj and Yoav Tock. Group communication.

ivo
Download Presentation

Affinity in Distributed Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Affinity in Distributed Systems Ph.D. thesisdefense. August 21, 2009. ÝmirVigfússon Joint work with: Hussam Abu-Libdeh, Mahesh Balakrishnan, Ken Birman, Gregory Chockler, Qi Huang, Jure Leskovec, Deepak Nataraj and Yoav Tock.

  2. Group communication • Most network trafficisunicast communication (one-to-one). • But a lot of content isidentical: • Audio streams, videobroadcasts, system updates, etc. • To minimizeredundancy, wouldbenice to multicast communication (one-to-many).

  3. Multicast by Unicast

  4. IP Multicast

  5. Gossip

  6. Group communication

  7. Talk Outline • Dr. Multicast (MCMD) • Group scalability in IP Multicast. • GossipObjects (GO) platform • Group scalability in gossip. • Affinity • GO+MCMD optimizationsbased on group overlaps • Explore the properties of overlaps in data sets • Conclusion

  8. IP Multicast in Data Centers Smaller scale – well defined hierarchy Single administrativedomain Firewalled– can ignore malicious behavior

  9. IP Multicast in Data Centers • Useful, but rarelyused. • Variousproblems: • Security • Stability • Scalability

  10. IP Multicast in Data Centers

  11. IP Multicast in Data Centers • Useful, but rarelyused. • Variousproblems: • Security • Stability • Scalability • Bottom line:Administrators have no control over IPMC. • Thustheychoose to disableit.

  12. Wishlist • Policy: Enable control of IPMC. • Transparency:Should be backward compatible with hardware and software. • Scalability:Needs to scale in number of groups. • Robustness: Solution should not bring in new problems.

  13. Acceptable Use Policy • Assume a higher-level network management tool compiles policy into primitives. • Explicitly allow a process (user) to use IPMC groups. • allow-join(process ID, logical group ID) • allow-send(process ID, logical group ID) • Multicast by point-to-point unicast always permitted. • Additional restraints. • max-groups(process ID, limit) • force-unicast(process ID, logical group ID)

  14. Dr. Multicast (MCMD) • Translates logical IPMC groups into either physical IPMC groups or multicast by unicast. • Optimizes resource use.

  15. Network Overhead • Gossip Layer uses constant background bandwidth on average 2.1 kb/s

  16. Application Overhead • Insignificant overhead when mapping logical IPMC group to physical IPMC group.

  17. Optimization questions Multicast BLACK Users Groups Users Groups

  18. Optimization Questions • Assign IPMC and unicast addresses s.t.  • Min. receiver filtering • Min. network traffic • Min. # IPMC addresses • … yet have all messages delivered to interested parties

  19. Optimization Questions • Assign IPMC and unicast addresses s.t.  • % receiver filtering (hard) • Min. network traffic • # IPMC addresses (hard) (1) • Prefers sender load over receiver load. • Control knobs part of administrative policy.

  20. MCMD Heuristic Groups in `user-interest’ space (1,1,1,1,1,0,1,0,1,0,1,1) (0,1,1,1,1,1,1,0,0,1,1,1) Grad Students Free Food

  21. MCMD Heuristic Groups in `user-interest’ space 224.1.2.4 224.1.2.5 224.1.2.3

  22. MCMD Heuristic Groups in `user-interest’ space Sending cost: MAX Filtering cost:

  23. MCMD Heuristic Groups in `user-interest’ space Unicast Sending cost: MAX Filtering cost:

  24. MCMD Heuristic Unicast Groups in `user-interest’ space 224.1.2.4 Unicast 224.1.2.5 224.1.2.3

  25. Dr. Multicast • Policy: Permits data center operators to selectively enable and control IPMC. • Transparency: Standard IPMC interface to user, standard IGMP interface to network. • Scalability: Uses IPMC when possible, otherwise point-to-point unicast. • Robustness: Distributed, fault-tolerant service.

  26. Talk Outline • Dr. Multicast (MCMD) • Group scalability in IP Multicast. • GossipObjects (GO) platform • Group scalability in gossip. • Affinity • GO+MCMD optimizationsbased on group overlaps • Explore the properties of overlaps in data sets • Conclusion

  27. Gossip • Def:Exchange information with a randomnode once per round. • Has appealingproperties: • Bounded network traffic. • Scalable in group size. • Robustagainstfailures. • Simple to code. • When # of groups scales up, lose

  28. GO Platform

  29. Randomgossip • Recipientselection: • Picknoded uniformlyatrandom. • Content selection: • Pick a rumorruniformlyatrandom.

  30. Observations • Gossiprumorsusuallysmall: • Incremental updates. • Few bytes hash of actual information. • Packet size below MTU irrelevant. • Stackrumors in a packet. • But whichones? • Rumorscanbedeliveredindirectly. • Uninterestednodemightforward to an interested one.

  31. Randomgossip w. stacking • Recipientselection: • Picknoded uniformlyatrandom. • Content selection: • Fillpacketwithrumorspickeduniformlyatrandom.

  32. GO Heuristic • Recipientselection: • Picknoded biasedtowardshigher group traffic. • Content selection: • Compute the utility of includingrumorr • Probability of rinfecting an uninfected host whenitreaches the target group. • Pickrumors to fillpacketwithprobabilityproportional to utility.

  33. GO Heuristic • Recipientselection: • Picknoded biasedtowardshigher group traffic. • Content selection: • Compute the utility of includingrumorr • Probability of rinfecting an uninfected host whenitreaches the target group. • Pickrumors to fillpacketwithprobabilityproportional to utility. Target group of r Include r ?

  34. Evaluation • IBM Websphere trace (1364 groups)

  35. Evaluation • IBM Websphere trace (1364 groups)

  36. Evaluation • IBM Websphere trace (1364 groups)

  37. Talk Outline • Dr. Multicast (MCMD) • Group scalability in IP Multicast. • GossipObjects (GO) platform • Group scalability in gossip. • Affinity • GO+MCMD optimizationsbased on group overlaps. • Explore the properties of overlaps in data sets. • Conclusion

  38. Affinity • BothMCMD and GO have optimizationsthatdepend on pairwisegroup overlaps (affinity). • Whatdegree of affinityshouldweexpect to arise in the real-world?

  39. Data sets/models • What’s in a ``group’’ ? • Social: • Yahoo! Groups • Amazon Recommendations • Wikipedia Edits • LiveJournalCommunities • MutualInterest Model • Systems: • IBM Websphere • Hierarchy Model Users Groups

  40. Social data sets • User and group degree distributions appearto followpower-laws. • Power-lawdegree distributions oftenmodeled by preferentialattachment. • MutualInterestmodel: • Preferentialattachment for bipartite graphs. Groups Users

  41. Systems Data Set • IBM Websphere has remarkable structure! • Typical for real-world systems? • Only one data point.

  42. Systems Data Set • Distributedsystems tend to behierarchicallystructured. • Hierarchymodel • Motivated by Live Objects. Thm:Expect a pair of users to overlap in groups .

  43. Data sets/models • Social: • Yahoo! Groups • Amazon Recommendations • Wikipedia Edits • LiveJournalCommunities • MutualInterest Model • Systems: • IBM Websphere • Hierarchy Model Users Groups

  44. Group similarity • Def: Similarity of groups j,j’ is • Wikipedia • LiveJournal

  45. Group similarity • Def: Similarity of groups j,j’ is • Mutual Interest Model

  46. Group similarity • Def: Similarity of groups j,j’ is • IBM Websphere • Hierarchy model

  47. Baseline overlap • Is the similarityweseea real effect? • Consider a random graph with the samedegree distributions as a baseline. • Spokes model:

  48. Baseline overlap • Plot differencebetween data and Spokes • Atmost 50 samples per group size pair. Looking pretty random

  49. Conclusions • Group communication important, but group scalability is lacking. • Dr. Multicast harnesses IPMC in data centers. • Impact:HotNets paper + NSDI Best Poster award. • Solution being adopted by CISCO and IBM.

  50. Conclusions • GO provides group scalability for gossip. • Impact: LADIS paper + Invited to the P2P Conference. • Platform will run under the Live Objects framework. • Characterizing and exploiting group affinity in systems is exciting current and future work.

More Related