1 / 26

Sven Bittner and Annika Hinze , 22 June 2006

Talk at the Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007) The Arbitrary Boolean Publish/Subscribe Model: Making the Case. Sven Bittner and Annika Hinze , 22 June 2006. Motivation. Assumption: Applications require Boolean semantics. Question:

scout
Download Presentation

Sven Bittner and Annika Hinze , 22 June 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Talk at the Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007)The Arbitrary Boolean Publish/Subscribe Model: Making the Case Sven Bittner and Annika Hinze, 22 June 2006

  2. Motivation Assumption: Applications require Boolean semantics. Question: Why do pub/sub systems typically only support conjunctive subscriptions and advertisements? Answer: General ones can be converted to DNF.

  3. Motivation: Conversion Consequences of conversion: + Individual subscription and advertisement less complex –Exponential number of subscriptions and advertisements Is there an equivalent of conversion? • Queries, e.g., in DBMSs, are converted to DNF • Really equivalent?

  4. Interaction Semantics View Subscriptions Event messages Advertisements System input System output Captures semantics from user perspective Notifications Publish/subscribe system Queries Data update/ insert/delete Schema/access privileges Query results Database manage- ment system

  5. Both transient, both in canonical form Both stored “as is” Data Storage View Event messages Notifications Stored data Transient data Subscription/ advertisement base Captures data storage from system perspective Advertisements Subscriptions Queries Query results Data update/ insert/delete Schema/ access privileges Subscription/ advertisement base

  6. Motivation: Implications 1. Event messages already in canonical form, attribute-value pairs (as queries after conversion in DBMS) 2. Subscriptions should be stored as is (as data in DBMS) 3. Conversion increases problem size  Pub/sub systems should not convert subscriptions.

  7. Structure of Talk • Motivation • BoP System • Filtering • Event Routing • Advertisements • Experiments • Summary & Future Work Annika Hinze - The Arbitrary Boolean Publish/Subscribe Model: Making the Case

  8. BoP System (Boolean Pub/Sub) • Event messages as attribute-value pairs • Support of general Boolean subscriptions • Operators “and”, “or”, and “not” • Any number of predicates per attribute • Support of general Boolean advertisements • Definition as subscriptions Motivation BoP System Experiments Summary

  9. AND AND AND title like “ Harry Potter ” OR OR OR endingWithin endingWithin < 1 day < 1 day AND AND AND AND AND AND price < 10.0 price < 10.0 price < 10.0 condition = NEW condition = NEW condition = NEW price < 15.0 price < 15.0 price < 15.0 condition = USED condition = USED condition = USED Filtering in BoP: Subscriptions • Subscription tree (similar to syntax tree) • Inner nodes: operators • Leaf nodes: predicate identifiers • Negation removal • Shifting down into leaf nodes • Encoding • Straightforward storage of operators and predicate identifiers Motivation BoP System Experiments Summary

  10. Filtering in BoP: Algorithm • Extension of counting approach, 3 steps [BH05] • Predicate matching • One-dimensional predicate indexes • Candidatesubscription matching • Counting of fulfilled predicates • Comparison to minimal number of required predicates • Realsubscription matching • Evaluation of tree structure • Optimizations (cf. paper) Motivation BoP System Experiments Summary

  11. Routing in BoP • Usage of subscription forwarding or advertisement forwarding • Event routing optimization: Subscription pruning Motivation BoP System Experiments Summary

  12. Subscription Pruning: Idea • Broadening of subscriptions by removing branches of subscription tree [BH06a] Pruning Pruning Pruning Motivation BoP System Experiments Summary

  13. Subscription Pruning: Selection • Ordering of pruningoperations [BH06b] • Accuracy • Estimated changed in selectivity of subscription • Efficiency • Intertwined with filter algorithm (minimal predicate number) • Size • Reduction in size of tree • Accuracy and popularity • Selectivity and usage of predicates • Goal: keep common parts of subscriptions Motivation BoP System Experiments Summary

  14. Advertisements in BoP • Overlap calculation • Based on disjoint predicates • Three algorithm steps • Disjoint predicate matching • Candidateoverlapping matching • Realoverlapping matching • Subscription routing optimization • Advertisement pruning [BH06c] • Order based on estimatedincrease in overlap (minimize additional subscription forwarding) Motivation BoP System Experiments Summary

  15. Structure of Talk • Motivation • BoP System • Filtering • Event Routing • Advertisements • Experiments • Summary & Future Work Annika Hinze - The Arbitrary Boolean Publish/Subscribe Model: Making the Case

  16. Experiments • Setup • Onlineauction scenario • Distributions of messages based on analysis of eBay • Characteristic classes of subscriptions and advertisement • Classes determine tree structure • Random population of operands in predicates • Results: real system analysis for five brokers (pruning of independent of network) MotivationBoP System Experiments Summary

  17. Both linear with advantageous cache usage for small subscription numbers (increasing counters) Experiments: Local Filtering Predicate distributions: u – uniform, z – Zipf, n – normal, rz – reversed Zipf, rn – reversed normal MotivationBoP System Experiments Summary

  18. Experiments: Distributed Filtering • Analysis of • Boolean filtering algorithm • Subscription pruning vs. • Counting filtering algorithm • Subscription covering • Covering proportion = • Proportion of removed non-local routing entries  the higher, the more covering • Created by varying attribute domain sizes MotivationBoP System Experiments Summary

  19. Boolean approach more time efficient for non-extreme covering proportions (left on abscissa) Experiments: Distributed Filtering MotivationBoP System Experiments Summary

  20. Boolean approach more space efficient for non-extreme covering proportions (left on abscissa) Experiments: Distributed Filtering MotivationBoP System Experiments Summary

  21. Structure of Talk • Motivation • BoP System • Filtering • Event Routing • Advertisements • Experiments • Summary & Future Work Annika Hinze - The Arbitrary Boolean Publish/Subscribe Model: Making the Case

  22. Summary • Motivation • Data storage view: subscriptions conform to data, and should not be converted • BoP • Support general Booleansubscriptions and advertisements • Solutions for filtering, event routing optimization, overlap calculation, subscription routing optimization MotivationBoP SystemExperiments Summary

  23. Conclusion • Experiments show that • Boolean subscriptions worthwhile with respect to • Central filtering (more time and space efficient) • Distributed filtering for non-extreme covering proportions (more time and space efficient) • Similar for advertisements (cf. paper) MotivationBoP SystemExperiments Summary

  24. Future Work • Integration of low-level filtering algorithm optimizations • Analysis of further application areas • Integration of routing and filtering algorithm  Optimization on more fine-grained basis MotivationBoP SystemExperiments Summary

  25. Thank you for your attention! Contact: Annika Hinze a.hinze@cs.waikato.ac.nz

  26. References [BH05] S. Bittner and A. Hinze. On the Benefits of Non-Canonical Filtering in Publish/Subscribe Systems. In Proceedings of the 4th International Workshop on Distributed Event-Based Systems (DEBS '05), pp. 451–457, Columbus, USA, 2005. [BH06a] S. Bittner and A. Hinze. Pruning Subscriptions in Distributed Publish/Subscribe Systems. In Proceedings of the Twenty-Ninth Australasian Computer Science Conference (ACSC 2006), pp. 197–206, Hobart, Australia, 2006. [BH06b] S. Bittner and A. Hinze. Dimension-Based Subscription Pruning for Publish/Subscribe Systems. In Proceedings of the 5th International Workshop on Distributed Event-Based Systems (DEBS '06), Lisbon, Portugal, 2006. [BH06c] S. Bittner and A. Hinze. Optimizing Pub/Sub Systems by Advertisement Pruning. In Proceedings of the 8th International Symposium on Distributed Objects and Applications (DOA 2006), pp. 1503–1521, Montpellier, France, 2006.

More Related