280 likes | 294 Views
Learn about Azure SQL Data Warehouse architecture, demos for creating, scaling, and integrating with PowerBI, use cases, and data storage needs. Get started with Azure's elastic cloud data warehouse in minutes.
E N D
Agenda • Introducing Azure SQL Data Warehouse • Under the hood – architecture overview • Demo 1 – Create a data warehouse in seconds • Demo 2 – Elastic scale and pause • Demo 3 – Enable real-time telemetry data storage • Use case scenarios • Demo 4 – Integration with PowerBI
Demand for data warehousing is rising Information managers are seeking innovative DBMS’s which are able to handle large data volumes in new ways or to optimize existing products and processes. Gartner (April 2015)
Azure SQL Data Warehouse Saas A relational data warehouse-as-a-service, fully managed by Microsoft. Industries first elastic cloud data warehouse with proven SQL Server capabilities. Support your smallest to your largest data storage needs. Azure Office 365 Office 365 Public Cloud Market Leading Price & Performance Elastic scale & performance Powered by the Cloud Azure Azure Get started in minutes Integrated with Azure ML, PowerBI & ADF Scales to petabytes of data Massively Parallel Processing Instant-on compute scales in seconds Query Relational / Non-Relational Simple billing compute & storage Pay for what you need, when you need it with dynamic pause
Azure SQL Data Warehouse Architecture Storage and Compute are de-coupled, enabling a true elastic service and separate charging for both compute and storage Application or User connection DMS (Data Movement Service) executes across all database nodes Data Loading (SSIS, REST, OLE, ADO, ODBC, WebHDFS, AZCopy, PS) DMS Massively Parallel Processing (MPP) Engine 100 DWU < > 2000 DWU Compute Scale compute up or down when required (SLA <= 60 seconds). Pause, Resume, Stop, Start. DMS DMS DMS DMS SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Azure Infrastructure and Storage Storage Add\Load data to WASB(S) without incurring compute costs Blob storage [WASB(S)] HDInsight
Azure SQL Data Warehouse – Control Node SQL DB SQL DB Massively Parallel Processing (MPP) Engine SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Control Node • Endpoint for connections • Regular SQL endpoint (TCP 1433) • Persists no user data (metadata only) • Coordinates compute activity using MPP Blob storage [WASB(S)] HDInsight
Azure SQL Data Warehouse - Compute Nodes SQL DB SQL DB Massively Parallel Processing (MPP) Engine SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Compute Node(s) Azure SQL Database An increase of DWU will increase the number of compute nodes Blob storage [WASB(S)] HDInsight
Azure SQL Data Warehouse – Blob storage SQL DB Massively Parallel Processing (MPP) Engine • RA-GRS storage • +PB’s of storage • Ingest data without incurring compute costs SQL DB SQL DB SQL DB SQL DB Compute Node Compute Node Compute Node Compute Node Control Node Blob storage [WASB(S)] HDInsight
Enabling the data-driven landscape Any data Anywhere Any size Massively Parallel Processing Built for modern data
Integrates with existing processes Dashboards and Visualizations Information Management Data Store Machine Learning and Analytics Power BI Business apps People Azure Machine Learning Azure Data Factory Personal Digital Assistant Cortana Azure Data Catalog Azure SQL Data Warehouse Azure HDInsight (Hadoop) ???? Custom apps Azure Event Hub Business Scenarios Azure Stream Analytics Azure Stream Analytics Recommendations, customer churn, forecasting, etc. IoT - Sensors and devices Automated Systems Azure Blob Storage DATA INTELLIGENCE ACTION
Unlocking scalable solutions eCommerce Azure SQL Data Warehouse Logistics Retail
Fraud Detection • Historical or batch-based analysis • Fast pattern trending and behaviour analysis • MPP analysis of high output telemetry (gaming, finance, manufacturing) • Monitor Distribution and Supply Chains • Perform mission critical intrusion analysis Trend Analysis Azure SQL Data Warehouse Transaction Detail Historical Transaction Data Distribution Management Intrusion Analysis Reference data
Sales Forecasting • Historical sales data analysis (P&L) • Product trending over massive data volumes • Customer segmentation based on new marketing or sales initiatives • Gap analysis & competitor impact P&L Analysis Azure SQL Data Warehouse Product trending Historical Sales Data Market & Gap Analysis Competitor impact
CHURN • Reduce customer turnover • Increase profitability • Maintain loyalty • Increase perception • Fast scenario testing Reduce customer turnover What’s hot and what’s not Loyalty & Perception Azure SQL Data Warehouse Data Sources Data Sources Scenario Simulations Data Sources
Predictive Maintenance • Identification of near-life assets • Aggregation of historical telemetry • Enhanced customer service and insight • Product improvement using Machine Learning • Execute workload simulations using trusted data New Asset Data Known Device Data Location External Data …..
Supports data ingestion from literally anywhere… Hadoop Azure Data Factory Data/ Workload Azure SQL DataWarehouse Other Azure components PaaS IaaS Analysis Services (MOLAP) Azure Machine Learning Azure Blob Storage
Thank you Matt Usher Snr. Program Manager SQL Data Warehouse Engineering
Azure SQL Data WarehouseEnabling the era of modern data warehousing
Preferred text layout (no bullets) • Main topic 1: size 40pt • Size 20pt for the subtopics • Size 20pt for the subtopics • Main topic 2: size 40pt • Size 20pt for the subtopics • Size 20pt for the subtopics • Main topic 3: size 40pt • Size 20pt for the subtopics • Size 20pt for the subtopics