1 / 28

Aurora

A new model and architecture for data stream management. Aurora. Sample Problem. Inspiration & Domain. Stream Processing Engines. HADP vs DAHP Events & Triggers Continuous Queries Real-time processing Transient data Lossy information. Application Domains. Online Auctions

daire
Download Presentation

Aurora

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A new model and architecture for data stream management Aurora

  2. Sample Problem Inspiration & Domain

  3. Stream Processing Engines • HADP vs DAHP • Events & Triggers • Continuous Queries • Real-time processing • Transient data • Lossy information

  4. Application Domains • Online Auctions • Network Traffic Management • Habitat Monitoring • Military Logistics • Immersive Environments • Road Traffic Monitoring • System Monitoring • Smart Energy Grid

  5. Aurora Overview

  6. The Topic • Aurora • The prototype • DBMS / SPE / DSMS • UI • The query language • The project • The authors

  7. The Authors • M.I.T. , Department of EECS and Laboratory of Computer Science • Michael Stonebraker • Brandeis University, Department of Computer Science • Daniel J. Abadi • Mitch Cherniack • Brown University , Department of Computer Science • Don Carney • Uğur Çetintemel • Christian Convey • Sangdon Lee • Nesime Tatbul • Stan Zdonik

  8. Talk Overview • Stream Processing Engines • SQuAl • Runtime • Related work

  9. Aurora SQuAl (Stream Query Algebra)

  10. SQuAl Overview • Connection Points • Models • Continuous Query • View • Ad-hoc Query • Operators • Order-agnostic • Order-sensitive

  11. SQuAl Operators • Order-agnostic • Filter • Map • Union • Order-sensitive • BSort • Aggregate • Join • Resample • Quirks!

  12. Filter (Unordered)

  13. Map (Unordered)

  14. Union (Unordered)

  15. BSort (Ordered)

  16. Aggregate (Ordered)

  17. Join (Ordered)

  18. Resample (Ordered) • Based on RRDTool’s philosophy? • Paper: • Simple interpolation • Use The Force, Read The Source: • Average • Count • Sum • Max • Min • LastVal

  19. SQuAl: Example

  20. Aurora Runtime

  21. Query Optimization • Dynamic Continuous Query Optimization • Inserting projections • Combining boxes • Reordering boxes • Ad-hoc query optimization

  22. Real-time Scheduling • Timestamped Tuples • Train scheduling • Interbox nonlinearities • Intrabox nonlinearities • Superboxes • Introspection • Static • Run-time

  23. Handling overload • QoS specifications • Response times • Tuple drops • Values produced • Load Shedding • Not Implemented at the time

  24. Aurora Related work

  25. Related work • STREAM • Stanford University, 2000-2006 • Telegraph • UC Berkley, 2000-2007? • SASE • UC Berkley / Mass Amherst, 2006-2008? • Cayuga • Cornell University, 2005-2007? • PIPES • University of Marburg, 2003-2007? • NiagaraCQ • University of Wiscon-Madison, 1999-2002

  26. Aurora’s Evolution

  27. Complex Event Processing Today • Oracle • Oracle CEP • Microsoft • MS SQL Server StreamInsight • Open Source • OpenPDC • Aleri • Coral8 • TruViso • StreamBase • Aurora’s Grandchild • IBM • SPADE • Active Middleware Technology

More Related