ZStream: A Cost-based Query Processor for Adaptively Detecting Composite Events

1. ZStream: A Cost-based Query Processor for Adaptively Detecting Composite Events Presented by Rabia Nuray-Turan and Xiaozhi Yu

2. What is ZStream • It is a composite event processing system(CEP). • It uses tree based query plans for both the logical and physical representation of query patterns. • It is able to unify the evaluation of sequence, conjunction, disjunction, negation, and Kleene closure as variants of the join operator. • It is able to capture the actual runtime behavior of a plan, and chooses the optimal plan.

3. Language Specification • Example: Stock market monitoring query Pattern T1;T2;T3 WHERE T1.name = T3.name; AND T2.name = ‘Google’ AND T1.price > (1+x%)*T2.price AND T3.price < (1-y%)* T2.price WITHIN 10 secs RETURN T1, T2, T3 Event Operators : Sequence------A;B Disjunction----A|B Negation------!A Conjunction ----A&B

4. Tree-Based Plans

5. Operator Evaluation • Sequence Operator • Has two operands: first and second event class in the sequential pattern • Right buffer is the main iteration to make sure the output is sorted in end time order • Any event in right/left buffer with start time less than EAT is discarded • Only valid events in right buffer are compared against the events in left buffer • Negation Operator • Must be combined with other operators • Pushed down in the tree to avoid unnecessary intermediate results • Finds time intervals when non-negation events can produce only valid results • Searches for negation events that can negate non-negation event instance

6. Operator Evaluation • Conjunction • Similar to sequence operator, but do not distinguish between the right and left buffers • Works like a sort-merge join • It maintains two cursors to both input buffers • Finds the earlier not-yet-matched event in one buffer and merges with the earlier events in the other buffer satisfying the constraints • Disjunction • Input is directly added to the output buffer if it satisfies both time and value constraints • Merging is done according to their end time • Kleene Closure • Trinary operator, with start and end closure • Kleene closure looks for matches in the midle buffer • If the count is not specified, maximum number of middle buffer events are found • If not, a window of specified size is applied

7. Optimal Query Plan • Rule Based Transformations • Smaller number of operators • If the number of operators same, plan with the lower cost operators will be selected • Hashing for equality Predicates • For sequence A;B where A.f = B.f, built hashtable on A • For conjunction built hash tables for A and B • Reduces the search cost for equality predicates • Optimal Reordering • Dynamic Programming based • Finds the optimal subtrees to find the optimal physical plan • Plan Adaptation • Recompute the physical query plan on the fly based on the collected statistics. • Change the plan without any damage to the expected output.

8. Performance Evaluation Query 7: Negate Pattern “IBM;!SUN;ORACLE” Pattern IBM;!SUN;ORACLE WITHIN 200 Units