StreamInsight 01 – Introducing SQL Server 2008 R2 StreamInsight - PowerPoint PPT Presentation

StreamInsight01 – Introducing SQL Server 2008 R2 StreamInsight

SQL10R2UPD05-DECK-01

[Presenter Name][Presenter Title] [Company Name]

• Introducing StreamInsight
• Querying Events in StreamInsight
• Designing StreamInsight Event Models and Adapters
• Installing, Deploying, and Maintaining the StreamInsight Runtime Engine

• Complex Event Processing
• Introducing StreamInsight and the Overall Architecture
• Demo: End-to-End Solution

• Processing and querying of event data streams
• Data queried while “in flight”
• May involve multiple concurrent event sources
• Works with high data rates
• Aims for near-zero latency

Event

request

output stream

input stream

response

• Identify from seemingly unrelated events:
• Patterns
• Relationships
• Gaps (expected events that did not occur)
• Abstractions
• Trigger immediate response actions

Relational Database Applications

CEP Target Scenarios

Operational Analytics Applications (e.g., Logistics)

Data Warehousing Applications

Web Analytics Applications

Manufacturing Applications

Financial Trading Applications

Monitoring Applications

Aggregate Data Rate (Events/sec)

• Algorithmic trading
• Smart order routing
• Real-time profit and loss
• Rapid analysis of transactional cost
• Fraud detection
• Risk management

• Often 100,000 events per second
• Automate
• Page layout
• Navigation
• Presentation
• Targeted advertising

• Real-time network monitoring
• Quality of service monitoring
• Location-based services
• Fraud detection
• Intrusion detection

• Battlefield control
• Monitoring of resource locations
• Intrusion detection
• Network traffic analysis
• Emails
• Network traffic
• Watch lists
• Financial movements

• Asset monitoring
• Aggregation of machine-based sensor data
• Generation of alerts in error conditions
• Identifying the “golden batch”

• Gaming machine event analysis
• Card table analysis
• Fraud detection
• Profit and loss in real-time
• Targeted advertising
• Player behavior
• Loyalty system implementation

• Real-time monitoring
• Managing player interest
• Website traffic analysis
• Detecting and eliminating undesired behaviors
• Understanding behavioral patterns

• Patient management
• Outbreak management
• Trend detection
• Insurance risk analysis

• Vehicle management
• Supply chain forecasting and tracking
• Maritime logistics
• GPS tracking

• Monitoring
• Consumption
• Variations
• Detecting outages
• Smart grid management
• Aggregating data across the grid

Industry Forum: http://complexevents.com

• Complex Event Processing
• Introducing StreamInsight and the Overall Architecture
• Demo: End-to-End Solution

• Platform for development and deployment of CEP applications
• High-throughput stream processing architecture
• .NET-based development environment

• Monitor data from multiple sources and detect:
• Meaningful patterns
• Trends
• Exceptions
• Opportunities
• Analyze data without storing it first
• Provide low-latency processing
• Trigger response actions
• Mine events for new business KPIs

• Use .NET languages such as C#
• Query using LINQ
• Take advantage of developer familiarity with .NET
• Reduce development times (and costs)
• Extend StreamInsight with .NET code

• Highly parallel execution platform
• In-memory caches
• Incremental result computation
• All processing triggered by incoming events
• Avoids polling overhead
• Can handle out-of-order events
• Can incorporate static reference or historical data

• Multiple deployment scenarios
• Fully integrated via embedded DLL
• Standalone server (multiple apps and users)
• Built-in monitoring and management
• Management interface
• Diagnostic views
• Manageability framework allows for remote monitoring
• Standalone event flow debugger

Data Sources, Operations, Assets, Feeds, Sensors, Devices

Input Data Streams

OutputData Streams

Input Data Streams

CEP Engine

Monitor

& Record

Mine

& Design

Manage

&

Benefit

f(x)

f'(x)

g(y)

h(x,y)

Results

Deploy

History

Operational Data Store & Archive

CEP Engine

f(x)

g(y)

f'(x)

h(x,y)

StreamInsight Architecture

End-to-End Solution

SQL10R2UPD05-DEMO-01

Demo

• Major highway
• 8 lanes
• 2 directions
• 6 toll points
• Vehicle types
• Car
• Bus
• Truck
• Ambulance
• Taxi

• Vehicles measured multiple times
• Electronic tag captured
• Speed measured
• Vehicle type determined

• EventID – guid
• TollPointId – which toll point: 0 to 5
• DirectionId – which direction: 0 (North) or 1 (South)
• Lane – which lane: 0 to 7
• VehicleTypeId – enumeration: 0-car, 1-bus, 2-truck, 3-taxi, 4-ambulance
• TagId – vehicle’s individual tag
• EnterGate – datetime when vehicle entered gate
• MillisecondsToPassSpeedCheckPoint – how long vehicle took to travel 10m
• ExitGate – datetime when vehicle exited gate

• StreamInsight Website
• http://www.microsoft.com/sqlserver/2008/en/us/R2-complex-event.aspx
• StreamInsight Books Online
• http://msdn.microsoft.com/en-us/library/ee362541(SQL.105).aspx
• StreamInsight Forums
• http://social.msdn.microsoft.com/Forums/en-US/streaminsight/threads
• StreamInsight Whitepaper
• http://download.microsoft.com/download/F/D/5/FD5E855C-D895-45A8-9F3E-110AFADBE51A/Microsoft%20CEP% 20Overview.docx

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.