Accommodating Bursts in Distributed Stream Processing Systems. Yannis Drougas, ESRI firstname.lastname@example.org Vana Kalogeraki, AUEB email@example.com. Stream Processing Applications. Large class of emerging applications in which data streams must be processed online Example applications include:
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Processed result streamsDistributed Stream Processing Systems
On-line processing functions / continuous query operators implemented on each node:
Application layer: Execution of stream processing applications.
Overlay network consisted of processing nodes. Built over a DHT (currently, Pastry).
The physical (IP) network.
Initial resource requirements
Additional resource requirements due to single burst
p’ = p +δ
p2 dominates p, however,
when there is a burst we may
have to change plans to p3
Index Point (Pareto Point) Database calculation time, this is why index points need to be pre-calculated.
BARRE avoids missing data units during the burst and minimizes lost data units. Also, resulting plan is “safer”, so it prevents future drops
BARRE results in fewer data units dropped. It can sustain up to about 80% (vs. about 20%) application rate increase without dropping any data unit.
BARRE achieves a high percentage of data unit delivery.