1 / 28

Deliver an elastic data warehouse as a service

Deliver an elastic data warehouse as a service. Agenda. Introducing Azure SQL Data Warehouse Under the hood – architecture overview Demo 1 – Create a data warehouse in seconds Demo 2 – Elastic scale and pause Demo 3 – Enable real-time telemetry data storage Use case scenarios

dpowell
Download Presentation

Deliver an elastic data warehouse as a service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Deliver an elastic data warehouse as a service

  2. Agenda • Introducing Azure SQL Data Warehouse • Under the hood – architecture overview • Demo 1 – Create a data warehouse in seconds • Demo 2 – Elastic scale and pause • Demo 3 – Enable real-time telemetry data storage • Use case scenarios • Demo 4 – Integration with PowerBI

  3. Demand for data warehousing is rising Information managers are seeking innovative DBMS’s which are able to handle large data volumes in new ways or to optimize existing products and processes. Gartner (April 2015)

  4. Azure SQL Data Warehouse Saas A relational data warehouse-as-a-service, fully managed by Microsoft. Industries first elastic cloud data warehouse with proven SQL Server capabilities. Support your smallest to your largest data storage needs. Azure Office 365 Office 365 Public Cloud Market Leading Price & Performance Elastic scale & performance Powered by the Cloud Azure Azure Get started in minutes Integrated with Azure ML, PowerBI & ADF Scales to petabytes of data Massively Parallel Processing Instant-on compute scales in seconds Query Relational / Non-Relational Simple billing compute & storage Pay for what you need, when you need it with dynamic pause

  5. Under the hood of Azure SQL Data Warehouse

  6. Azure SQL Data Warehouse Architecture Storage and Compute are de-coupled, enabling a true elastic service and separate charging for both compute and storage Application or User connection DMS (Data Movement Service) executes across all database nodes Data Loading (SSIS, REST, OLE, ADO, ODBC, WebHDFS, AZCopy, PS) DMS Massively Parallel Processing (MPP) Engine 100 DWU < > 2000 DWU Compute Scale compute up or down when required (SLA <= 60 seconds). Pause, Resume, Stop, Start. DMS DMS DMS DMS SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Azure Infrastructure and Storage Storage Add\Load data to WASB(S) without incurring compute costs Blob storage [WASB(S)] HDInsight

  7. Azure SQL Data Warehouse – Control Node SQL DB SQL DB Massively Parallel Processing (MPP) Engine SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Control Node • Endpoint for connections • Regular SQL endpoint (TCP 1433) • Persists no user data (metadata only) • Coordinates compute activity using MPP Blob storage [WASB(S)] HDInsight

  8. Azure SQL Data Warehouse - Compute Nodes SQL DB SQL DB Massively Parallel Processing (MPP) Engine SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Compute Node(s) Azure SQL Database An increase of DWU will increase the number of compute nodes Blob storage [WASB(S)] HDInsight

  9. Azure SQL Data Warehouse – Blob storage SQL DB Massively Parallel Processing (MPP) Engine • RA-GRS storage • +PB’s of storage • Ingest data without incurring compute costs SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Blob storage [WASB(S)] HDInsight

  10. Demo 1Creating a new Azure SQL Data Warehouse

  11. Enabling the data-driven landscape Any data Anywhere Any size Massively Parallel Processing Built for modern data

  12. Demo 2Elastic scale and pause

  13. Integrates with existing processes Dashboards and Visualizations Information Management Data Store Machine Learning and Analytics Power BI Business apps People Azure Machine Learning Azure Data Factory Personal Digital Assistant Cortana Azure Data Catalog Azure SQL Data Warehouse Azure HDInsight (Hadoop) ???? Custom apps Azure Event Hub Business Scenarios Azure Stream Analytics Azure Stream Analytics Recommendations, customer churn, forecasting, etc. IoT - Sensors and devices Automated Systems Azure Blob Storage DATA INTELLIGENCE ACTION

  14. Demo 3IoT - Vehicle telemetry

  15. Unlocking scalable solutions eCommerce Azure SQL Data Warehouse Logistics Retail

  16. Fraud Detection • Historical or batch-based analysis • Fast pattern trending and behaviour analysis • MPP analysis of high output telemetry (gaming, finance, manufacturing) • Monitor Distribution and Supply Chains • Perform mission critical intrusion analysis Trend Analysis Azure SQL Data Warehouse Transaction Detail Historical Transaction Data Distribution Management Intrusion Analysis Reference data

  17. Sales Forecasting • Historical sales data analysis (P&L) • Product trending over massive data volumes • Customer segmentation based on new marketing or sales initiatives • Gap analysis & competitor impact P&L Analysis Azure SQL Data Warehouse Product trending Historical Sales Data Market & Gap Analysis Competitor impact

  18. CHURN • Reduce customer turnover • Increase profitability • Maintain loyalty • Increase perception • Fast scenario testing Reduce customer turnover What’s hot and what’s not Loyalty & Perception Azure SQL Data Warehouse Data Sources Data Sources Scenario Simulations Data Sources

  19. Predictive Maintenance • Identification of near-life assets • Aggregation of historical telemetry • Enhanced customer service and insight • Product improvement using Machine Learning • Execute workload simulations using trusted data New Asset Data Known Device Data Location External Data …..

  20. Supports data ingestion from literally anywhere… Hadoop Azure Data Factory Data/ Workload Azure SQL DataWarehouse Other Azure components PaaS IaaS Analysis Services (MOLAP) Azure Machine Learning Azure Blob Storage

  21. Demo 4Integrated with PowerBI

  22. Please followhttp://aka.ms/SQLDWto learn more

  23. Thank you Matt Usher Snr. Program Manager SQL Data Warehouse Engineering

  24. Azure SQL Data WarehouseEnabling the era of modern data warehousing

  25. Preferred text layout (no bullets) • Main topic 1: size 40pt • Size 20pt for the subtopics • Size 20pt for the subtopics • Main topic 2: size 40pt • Size 20pt for the subtopics • Size 20pt for the subtopics • Main topic 3: size 40pt • Size 20pt for the subtopics • Size 20pt for the subtopics

  26. Built on Azure.

More Related