StreamInsight 01 – Introducing SQL Server 2008 R2 StreamInsight

1. StreamInsight01 – Introducing SQL Server 2008 R2 StreamInsight SQL10R2UPD05-DECK-01 [Presenter Name][Presenter Title] [Company Name]

2. Module Overview • Introducing StreamInsight • Querying Events in StreamInsight • Designing StreamInsight Event Models and Adapters • Installing, Deploying, and Maintaining the StreamInsight Runtime Engine

3. Agenda • Complex Event Processing • Introducing StreamInsight and the Overall Architecture • Demo: End-to-End Solution

4. Complex Event Processing • Processing and querying of event data streams • Data queried while “in flight” • May involve multiple concurrent event sources • Works with high data rates • Aims for near-zero latency

5. Isn’t This Just a Database Application? Event request output stream input stream response

6. Goals of Complex Event Processing • Identify from seemingly unrelated events: • Patterns • Relationships • Gaps (expected events that did not occur) • Abstractions • Trigger immediate response actions

7. “Sweet Spot” for Complex Event Processing Relational Database Applications CEP Target Scenarios Operational Analytics Applications (e.g., Logistics) Data Warehousing Applications Web Analytics Applications Manufacturing Applications Financial Trading Applications Monitoring Applications Aggregate Data Rate (Events/sec)

8. Usage Example: Capital Markets • Algorithmic trading • Smart order routing • Real-time profit and loss • Rapid analysis of transactional cost • Fraud detection • Risk management

9. Usage Example: Click-Stream Analysis • Often 100,000 events per second • Automate • Page layout • Navigation • Presentation • Targeted advertising

10. Usage Example: Communications • Real-time network monitoring • Quality of service monitoring • Location-based services • Fraud detection • Intrusion detection

11. Usage Example: Command Intelligence • Battlefield control • Monitoring of resource locations • Intrusion detection • Network traffic analysis • Emails • Network traffic • Watch lists • Financial movements

12. Usage Example: Manufacturing • Asset monitoring • Aggregation of machine-based sensor data • Generation of alerts in error conditions • Identifying the “golden batch”

13. Usage Example: Casino Monitoring • Gaming machine event analysis • Card table analysis • Fraud detection • Profit and loss in real-time • Targeted advertising • Player behavior • Loyalty system implementation

14. Usage Example: MPG and Virtual Worlds • Real-time monitoring • Managing player interest • Website traffic analysis • Detecting and eliminating undesired behaviors • Understanding behavioral patterns

15. Usage Example: Public Health • Patient management • Outbreak management • Trend detection • Insurance risk analysis

16. Usage Example: Logistics • Vehicle management • Supply chain forecasting and tracking • Maritime logistics • GPS tracking

17. Usage Example: Energy Management • Monitoring • Consumption • Variations • Detecting outages • Smart grid management • Aggregating data across the grid

18. Competitive Landscape Industry Forum: http://complexevents.com

19. Agenda • Complex Event Processing • Introducing StreamInsight and the Overall Architecture • Demo: End-to-End Solution

20. Microsoft StreamInsight • Platform for development and deployment of CEP applications • High-throughput stream processing architecture • .NET-based development environment

21. StreamInsight Purposes • Monitor data from multiple sources and detect: • Meaningful patterns • Trends • Exceptions • Opportunities • Analyze data without storing it first • Provide low-latency processing • Trigger response actions • Mine events for new business KPIs

22. Benefit: .NET Development Environment • Use .NET languages such as C# • Query using LINQ • Take advantage of developer familiarity with .NET • Reduce development times (and costs) • Extend StreamInsight with .NET code

23. Benefit: Performance and Data Throughput • Highly parallel execution platform • In-memory caches • Incremental result computation • All processing triggered by incoming events • Avoids polling overhead • Can handle out-of-order events • Can incorporate static reference or historical data

24. Benefit: Deployment and Management • Multiple deployment scenarios • Fully integrated via embedded DLL • Standalone server (multiple apps and users) • Built-in monitoring and management • Management interface • Diagnostic views • Manageability framework allows for remote monitoring • Standalone event flow debugger

25. StreamInsight Event Data Flow Data Sources, Operations, Assets, Feeds, Sensors, Devices Input Data Streams OutputData Streams Input Data Streams CEP Engine Monitor & Record Mine & Design Manage & Benefit f(x) f'(x) g(y) h(x,y) Results Deploy History Operational Data Store & Archive CEP Engine f(x) g(y) f'(x) h(x,y)

26. StreamInsight Architecture

27. End-to-End Solution SQL10R2UPD05-DEMO-01 Demo

28. Demo Scenario: Highway Monitor • Major highway • 8 lanes • 2 directions • 6 toll points • Vehicle types • Car • Bus • Truck • Ambulance • Taxi

29. Toll-Point Timings • Vehicles measured multiple times • Electronic tag captured • Speed measured • Vehicle type determined

30. Toll-Point Measurements • EventID – guid • TollPointId – which toll point: 0 to 5 • DirectionId – which direction: 0 (North) or 1 (South) • Lane – which lane: 0 to 7 • VehicleTypeId – enumeration: 0-car, 1-bus, 2-truck, 3-taxi, 4-ambulance • TagId – vehicle’s individual tag • EnterGate – datetime when vehicle entered gate • MillisecondsToPassSpeedCheckPoint – how long vehicle took to travel 10m • ExitGate – datetime when vehicle exited gate

31. Resources • StreamInsight Website • http://www.microsoft.com/sqlserver/2008/en/us/R2-complex-event.aspx • StreamInsight Books Online • http://msdn.microsoft.com/en-us/library/ee362541(SQL.105).aspx • StreamInsight Forums • http://social.msdn.microsoft.com/Forums/en-US/streaminsight/threads • StreamInsight Whitepaper • http://download.microsoft.com/download/F/D/5/FD5E855C-D895-45A8-9F3E-110AFADBE51A/Microsoft%20CEP% 20Overview.docx

32. © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.