1 / 6

Motivation

Motivation. Data Streams. Traditional DBMS – data stored in finite, persistent data sets New Applications – data input as continuous, ordered data streams Network monitoring and traffic engineering Telecom call records Network security Financial applications Sensor networks

kael
Download Presentation

Motivation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Motivation

  2. Data Streams • Traditional DBMS – data stored in finite, persistentdata sets • New Applications – data input as continuous, ordereddata streams • Network monitoring and traffic engineering • Telecom call records • Network security • Financial applications • Sensor networks • Manufacturing processes • Web logs and clickstreams • Massive data sets

  3. Data Stream Management System User/Application Register Query Results Data Stream Management System (DSMS) Stream Query Processor Scratch Space (Memory and/or Disk)

  4. Meta-Questions • Killer-apps • Application stream rates exceed DBMS capacity? • Can DSMS handle high rates anyway? • Motivation • Need for general-purpose DSMS? • Not ad-hoc, application-specific systems? • Non-Trivial • DSMS = merely DBMS with enhanced support for triggers, temporal constructs, data rate mgmt?

  5. Sample Applications • Network security(e.g., iPolicy, NetForensics/Cisco, Niksun) • Network packet streams, user session information • Queries: URL filtering, detecting intrusions & DOS attacks & viruses • Financial applications(e.g., Traderbot) • Streams of trading data, stock tickers, news feeds • Queries: arbitrage opportunities, analytics, patterns

  6. Persistent relations One-time queries Random access (pull) “Unbounded” disk store Only current state matters Passive repository Relatively low update rate No real-time services Assume precise data Access plan determined by query processor, physical DB design Transient streams Continuous queries Sequential access (push) Bounded main memory History/arrival-order is critical Active stores Possibly multi-GB arrival rate Real-time requirements Data stale/imprecise Unpredictable/variable data arrival and characteristics DBMS versus DSMS

More Related