1 / 30

Adaptive Content-based Routing In General Overlay Topologies

Adaptive Content-based Routing In General Overlay Topologies. Guoli Li, Vinod Muthusamy Hans-Arno Jacobsen Middleware Systems Research Group University of Toronto. http://padres.msrg.toronto.edu. Publisher. Subscriber. Subscriber. Distributed Publish/Subscribe. Advertisement.

dara-holcomb
Download Presentation

Adaptive Content-based Routing In General Overlay Topologies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Adaptive Content-based Routing In General Overlay Topologies Guoli Li, Vinod Muthusamy Hans-Arno Jacobsen Middleware Systems Research Group University of Toronto http://padres.msrg.toronto.edu

  2. Publisher Subscriber Subscriber Distributed Publish/Subscribe Advertisement • An acyclic overlay is sensitive to: • Congestion • Broker failures • Benefits of a general overlay: • Routing around congestion and failures • Handling imbalanced workloads Publication Subscription • Applications • Business process execution e.g., BPEL • Business activity monitoring • Service discovery and integration • … Middleware 2008 @ Leuven, Belgium

  3. Challenges With General Overlays • Subscriptions are routed in loops • Brokers receive duplicate subscriptions • Subscription copies exacerbate the problem • Same problem for publications S X Adv 1 S 1 2 3 4 S 5 6 Adv 2 Middleware 2008 @ Leuven, Belgium

  4. Agenda • Content-based routing protocol for general overlays • Atomic and composite subscriptions • Optimal publication routing • Evaluation • Dynamic publication routing • Adaptive composite subscription routing Middleware 2008 @ Leuven, Belgium

  5. TID-based Approach S X • Each advertisement is assigned to a unique tree identifier (TID) • Each subscription has a TID predicate with a variable 1 2 Adv 1 3 4 5 6 Adv 2 Middleware 2008 @ Leuven, Belgium

  6. Subscription Routing S X S: [class=stock][symbol=*] [TID=$Z] At Broker 1: Adv1: [class=stock][symbol=IBM] [TID=Adv1] Adv2: [class=stock][symbol=HP] [TID=Adv2] S matching Adv1: [class=stock][symbol=*][TID=Adv1] S matching Adv2: [class=stock][symbol=*] [TID=Adv2] SA1 SA2 1 2 Adv 1 3 4 5 6 Adv 2 Middleware 2008 @ Leuven, Belgium

  7. Publication Routing • Each publication is assigned the TID of its matching advertisement • e.g., p [class, stock][symbol,HP][TID, adv_msg_id] • Publications are routed: • Fixed TID routing: a publication is routed to subscribers along its advertisement tree. • Dynamic publication routing: a publication may be routed to subscribers across advertisement trees. Middleware 2008 @ Leuven, Belgium

  8. Fixed TID Routing Adv 1 P • Property • No broker receives duplicate publication messages Adv 2 1 2 P 3 4 5 6 Sub X Middleware 2008 @ Leuven, Belgium

  9. Dynamic Publication Routing Adv 1 • Publication’s TID is changeable • Routing heuristic • Util = Routput / Rsending • Property • Changing a publication’s TID while in transit will not change the set of notified subscribers. Adv 2 1 2 P 3 4 5 6 Sub X Middleware 2008 @ Leuven, Belgium

  10. Advantages • Retains the publish/subscribe client interface • Speeds up subscription and publication matching • Avoids duplicate subscriptions and publications • Routes publications dynamically across multiple alternatives • Enables routing around failures, congestion and load imbalances Middleware 2008 @ Leuven, Belgium

  11. Composite Subscription • A composite subscription consists of atomic subscriptions linked by logical operators (e.g., AND, OR). • Composite subscription routing • Topology-based routing • Adaptive routing AND e.g., CS= {[class=stock][symbol=YHOO][price>12]} AND {[class=stock][symbol=MSFT][price<20]} S1 S2 Middleware 2008 @ Leuven, Belgium

  12. Topology-based CS Routing Adv 3 Adv 2 1 2 7 Broker 4 and 8 are the joint point brokers S2A2 CS’ S3A3 3 4 8 S1A1 5 6 9 Adv 1 CS CS={{S1 AND S2} ANDS3} CS’ ={S1 AND S2} Middleware 2008 @ Leuven, Belgium

  13. Adaptive CS Routing • CS’s joint points are determined according to potential publication traffic, bandwidth, latency, etc. Adv 2 Adv 2 2 2 1 3 1 3 Adv 1 Adv 1 CS={S1 AND S2} CS={S1 AND S2} Middleware 2008 @ Leuven, Belgium

  14. Matching Engine + Routing Table output queue B1 input queue subscription dest output queue B2 Cost Model • Routing cost of CS • RC(CS)) = + + • Subscription cardinality • |P(S)| : The number of matching publications per unit of time. • |P(S)| = • |P(CS)| = |P(Sl)| + |P(Sr)| if op = or Broker symbol=IBM B1 symbol=HP B2 Middleware 2008 @ Leuven, Belgium

  15. Adaptive CS Routing Adv 3 Adv 2 1 2 7 CS’ S3A3 CS’ 3 4 8 S2A2 Adv 1 5 6 9 S1A1 CS CS={{S1 AND S2} ANDS3} CS’ ={S1 AND S2} Middleware 2008 @ Leuven, Belgium

  16. Evaluation • Setup • Overlays of 32 brokers with different connection degrees • Cluster (each node:1.86GHz, 4G) and PlanetLab • Workloads: • Yahoo!Finance stock quote traces • http://research.msrg.utoronto.ca/Padres/DataSets • Metrics • End to end notification delay • Network traffic Middleware 2008 @ Leuven, Belgium

  17. Dense vs. Sparser Topologies 4% 20% Note: The benefit is not proportional to the connection degree. Middleware 2008 @ Leuven, Belgium

  18. Higher Publication Rate stabilized Middleware 2008 @ Leuven, Belgium

  19. Publication Burst Burst Middleware 2008 @ Leuven, Belgium

  20. With Broker Failures 2nd failure 1st failure Middleware 2008 @ Leuven, Belgium

  21. CS Routing Traffic Middleware 2008 @ Leuven, Belgium

  22. Conclusions • Enables routing around failures, congestion and load imbalances • Allows publications routing across alternative paths • Improves the notification delay by 20% • Enables flexible CS routing • Reduces 80% publication traffic • Improves the notification delay by 55% • Simplifies solutions for failure recovery and load balancing Middleware 2008 @ Leuven, Belgium

  23. Questions? P A D R E S • http://padres.msrg.utoronto.ca Middleware 2008 @ Leuven, Belgium

  24. More Publishers Middleware 2008 @ Leuven, Belgium

  25. Effect of Subscriber Distance Middleware 2008 @ Leuven, Belgium

  26. On PlanetLab Middleware 2008 @ Leuven, Belgium

  27. CS Delay Middleware 2008 @ Leuven, Belgium

  28. Faster Matching with TIDs Subscriptions are augmented with TIDs only once at the first broker. Other brokers can route the subscription based on the TID alone. Similar argument applies to publication routing. Middleware 2008 @ Leuven, Belgium

  29. Advertisement Routing • Each advertisement forms a spanning advertisement tree • Duplicated advertisements are discarded by brokers • Each advertisement is assigned a unique tree identifier (TID) • e.g., a [class,eq,stock]……[TID,eq,adv_msg_id] • Subscription Routing Table (SRT) • A set of [advertisement, last hop] Middleware 2008 @ Leuven, Belgium

  30. Subscription Routing • Each subscription has a TID predicate with a variable. • e.g., s [class,eq,stock]……[TID,eq,$X] • The variable is bound to the TID of a matching advertisement • Publication Routing Table (PRT) • A set of [subscription, {TID, last hop of subscription }] Middleware 2008 @ Leuven, Belgium

More Related