Small-Scale Peer-to-Peer Publish/Subscribe

1. Small-Scale Peer-to-Peer Publish/Subscribe Vinod Muthusamy, Hans-Arno Jacobsen University of Toronto July 17, 2005 P2PKM 2005San Diego, CA

2. Motivation • Many distributed hash table (DHT)-based peer-to-peer applications • Storage (OceanStore, Past) • Database (PIER) • Naming service (Chord-DNS) • Publish/subscribe (Scribe, P2P-ToPSS, Meghdoot) • Typically designed for and evaluated on a large-scale • Small-scale performance is important • Even large systems start small • Benefits even on small scale • Small-scale case study of a DHT-based pub/sub protocol Small-Scale P2P Publish/Subscribe

3. Agenda • Background • Publish/subscribe model • Distributed hash tables • Small-scale benefits • Publish/subscribe on DHT • Evaluation • Conclusions Small-Scale P2P Publish/Subscribe

4. Notification Notification Publish/subscribe model TSX Stock markets NASDAQ • Simple interface • Decoupled interaction • Network efficiency NYSE Publisher Publisher AMGN=58 Publications IBM=84 ORCL=12 JNJ=58 HON=24 INTC=19 MSFT=27 BrokerNetwork Subscriptions Subscriber Subscriber Subscriptions: IBM > 85 ORCL < 10 JNJ > 60 Small-Scale P2P Publish/Subscribe

5. Distributed hash tables • Example: Pastry • Distributed version of a hash table data structure • Store (key, value) pairs • Each node receives at most K/N keys • Each node has about O(logN) routing state • Lookups resolved with O(logN) hops Small-Scale P2P Publish/Subscribe

6. Small-scale benefits Small-Scale P2P Publish/Subscribe

7. Service oriented architecture (SOA) • An illustration of small-scale benefits • SOA: a distributed architecture • Loosely coupled reusable components • Enterprise service bus (ESB) comm. fabric • Can be realized as a pub/sub broker network • Pub/sub support in IBM Websphere, Sonic ESBs • Example • Three-tier Web server • Communication between components through pubs and subs • Decoupled interaction • Declarative bindings • Simple to add a Monitor component • Incremental service deployment • Late binding Application Server Enterprise Service Bus User Products Monitor Database Database Small-Scale P2P Publish/Subscribe

8. Pub/sub case study • P2P-ToPSS • A pub/sub system built over DHT • Large-scale performance previously studied [under submission] • Evaluate small-scale benefits (of the same algorithm) now • Subscription load balance Small-Scale P2P Publish/Subscribe

9. Subscription Publication price < 50 weight > 50 price = 10 weight = 70 Distributed multidimensional matching (DMM) 1) Pub/sub domain: 3) Network domain: - 0 1 2) Spatial domain: 100 00 01 10 11 01 11 0 1 weight 00 10 DHT network used to map from tree nodes to peers 0 0 price 100 Small-Scale P2P Publish/Subscribe

10. peer-id AF3483BQ1 hash Region weight price name/code z-code p_w_1101 1101 Distributing the tree • Let region r have z-code z, and tree node n(r) • Store n(r) at peer p(r), where p(r) is where DHT would store key z • Each peer performs mapping locally • Hash function ensures even distribution Network Subscription price = … weight = … p/w Small-Scale P2P Publish/Subscribe

11. Load balancing • Peers can be overloaded due to • Excessive subscriptions storage • Excessive publication traffic • Overloaded peer delegates subscriptions to children in the DMM tree • A local decision Network 0 00 01 p/w Small-Scale P2P Publish/Subscribe

12. Evaluation • SimPastry simulator (Microsoft Research) • Pastry DHT • Simulate network latencies, message costs • Nodes connected in a LAN • 1ms latency between every pair of nodes • Subscriptions • Attribute name + range • E.g.: “20 < weight < 40” • Metric • Storage load of subscriptions Small-Scale P2P Publish/Subscribe

13. DHT performance • 1000 random keys • DHT successfully balances load Small-Scale P2P Publish/Subscribe

14. Pub/sub performance weight price • 1000 fine-grained subscriptions • Load balancing distributes storage load Small-Scale P2P Publish/Subscribe

15. Pub/sub performance weight price • 1000 coarse-grained subscriptions • Load balancing distributes but also increases storage load Small-Scale P2P Publish/Subscribe

16. Observation • Load balancing technique assumes available resources somewhere in the network • Leads to runaway subscription delegation • Common assumption in DHT applications • CAN DHT’s active caching and replication • Meghdoot’s hotspot circumvention • In DHT, no knowledge of aggregate available resources • Fundamental assumption of DHT applications not valid in small-scale networks Small-Scale P2P Publish/Subscribe

17. Conclusions • Pub/sub and DHTs • Proven benefits in large-scale network • Untapped benefits in small-scale networks • Common assumption on DHT applications • Sufficient available resources in the network • Small-scale networks break this assumption • Future work • Implement algorithms to estimate global resource availability • Curb runaway subscription delegation with this information • Evaluate other DHT applications in small-scale networks Small-Scale P2P Publish/Subscribe

18. Q&A Small-Scale Peer-to-Peer Publish/Subscribe www.msrg.toronto.edu Small-Scale P2P Publish/Subscribe