510 likes | 764 Views
70-534 Microsoft Azure Solutions certification exam prep training includes test prep questions, exam questions and free study guide downloads.http://www.pass4sureexam.co/70-534.html
E N D
Big Data Briefing How to implement Big Data on Windows Azure http://www.pass4sureexam.co/70-534.html
Big Data; running cheaply and efficiently on Windows Azure to discover business value from data you already hold about your business.
Introduction Who Andy Cross, Director of Microsoft Partner Elastacloud, Windows Azure MVP, Big Data specialist and international speaker. Agenda Windows Azure Intro HDInsightIntro Architecture BI and Visualisations Case Studies Time until next coffee 55:00 30 minutes 30 minutes 30 minutes 15 minutes 15 minutes Find out more at http://www.Microsoft.com http://www.pass4sureexam.co/70-534.html
Introduction to Windows Azure http://www.pass4sureexam.co/70-534.html
Today’s Transformation – Cloud Pooled Resources Self-Service Elastic Usage Based • Economics ▪ Agility ▪ Focus http://www.pass4sureexam.co/70-534.html
IT Continuum Evolution toward highly-virtual and beyond to cloud PaaS SaaS http://www.pass4sureexam.co/70-534.html Physical Virtual / Private IaaS
Microsoft On The Internet 400m active accounts 1.0 One of 3rd world’s largest private networks More than 3 billion queries per month Microsoft account 1 billion+ authentications per day 1.0 750 million unique visitors at any given time Global Foundation Services http://www.pass4sureexam.co/70-534.html
The Microsoft Cloud ~Globally Distributed Data Centers Quincy, WA Chicago, IL San Antonio, TX Dublin, Ireland Generation 4 DCs
Cloud Computing Patterns Inactivity Period On and Off – Start/End Semester Compute • On & off workloads (e.g. batch job) • Over provisioned capacity is wasted • Time to market can be cumbersome t Growing Fast – Research Project • Successful services needs to grow/scale • Keeping up w/ growth is big IT challenge • Cannot provision hardware fast enough Compute t Unpredictable Bursting – Web demand • Unexpected/unplanned peak in demand • Sudden spike impacts performance • Can’t over provision for extreme cases Compute Predictable Burst – Registration t • Services with micro seasonality trends • Peaks due to periodic increased demand • IT complexity and wasted capacity Compute t
Applicationbuildingblocks Big data Database Media Storage Traffic Messaging Identity Caching CDN Networking
HDInsight Summary GA, recent preview of Hadoop 2 Managed Hadoop on Windows Azure Familiar tools such as Hive, Pig, Oozie Additional BoB Microsoft ecosystem tooling with .net SDK Powershell and .net for provision Execution with .net and powershell for Hive Paired with Hortonworks HDP for on-premises Hadoop; compatible with all major Hadoop implementations Combined with Excel and traditional Microsoft BI stack for compelling solutions http://www.pass4sureexam.co/70-534.html
Position in Cloud PaaS
Centralised Resources State – a Windows Azure SQL DB stores metadata to allow identical clusters to be spawned Data – all your data resides on Windows Azure Blob Storage; resilient, secure and cost effective Logs – machine state and cluster uptime is stored in Windows Azure Table Storage
Provision hundreds of machines against your data; gain understanding rapidly
Speed to understanding 500 servers. 20 minutes. 40,000,000,000 data points
Drive Investment Buy the Look
Exploring Sentiment, Social and Metadata Graphs Person Product Product Alike Owns Owns Person Alike? Social Media
Microsoft Big Data Solution FAMILIAR END USER TOOLS Excel with PowerPivot Power View Predictive Analytics Embedded BI BI PLATFORM SSAS SSRS Microsoft EDW Connectors Hadoop On Windows Azure Hadoop On Windows Server UNSTRUCTURED & STRUCTURED DATA Sensors Devices Bots Crawlers ERP CRM LOB APPs
Microsoft Offerings INSIGHTS Hive ODBC Driver & Hive Add-in for Excel Integration with Microsoft PowerPivot ENTERPRISE READY Hadoop based distribution for Windows Server & Azure Strategic Partnership with Hortonworks JavaScript framework for Hadoop RTM of Hadoop connectors for SQL Server and PDW BROADER ACCESS
Hadoop on WindowsInsights to all users by activating new types of data Differentiation INSIGHTS Integrate with Microsoft Business Intelligence Choice of deployment on Windows Server + Windows Azure Integrate with Windows Components (AD, Systems Center) ENTERPRISE READY Easy installation and configuration of Hadoop on Windows Simplified programming with . Net & Javascript integration Integrate with SQL Server Data Warehousing BROADER ACCESS • Contributions proposed back to community distribution
Microsoft Big Data Roadmap Microsoft is extending its leadership in business intelligence and data warehousing to provide insights to all users by activating new types of data of any size To accelerate the delivery of Microsoft’s Hadoop based solution for Windows Server and service for Windows Azure, Microsoft is announcing a partnership with Hortonworks Microsoft is announcing an end-to-end roadmap for Big Data that embraces Apache HadoopTM by distributing enterprise class Hadoop based solutions on both Windows Server and Windows Azure Microsoft is committed to broadening accessibility and usage of Hadoop to end users, developers and IT professionals in organizations of all sizes
Microsoft’s value adds Why Elastacloud recommend Microsoft’s platform
Skill reuse Your existing development team can immediately realise value The frameworks facilitate deterministic testing for highly reliable queries Express elegant solutions in C# Familiar Unit Testing patterns Concise programmatic terseness Complex logic is best expressed in programmatic form
Commoditised query Value of query Value De-provision Provision Cost Time Action Cost Execute
Hybrid Compatibility Microsoft are the only vendor with enterprise on premises and cloud big data offerings Hadoop On Premises HDInsight in Azure Name=Andy Pnid=123456 123456 4712
Just as Big Data is more than Hadoop; Windows Azure is more than HDInsight HDP Spark Storm
Elastacloud provisioned a bespoke 500 core virtual cluster on Windows Azure, allowing us to cut down a compute intensive task from 20 days over 8 cores to under 12 hours over 500 cores. Jedidiah Francis, Data Scientist, ASOS
Architectural Considerations Solving Big Data challenges cohesively
The first challenge you face with Big Data is somewhat the hardest
Elastacloud’s tooling aims to solve these challenges Ingress, Compute, Egress automated; allowing focus on providing value
Big Data solutions are rarely the result of a single piece of code running; instead jobs transform data into intermediate domains before completing an overall piece of work Workflows
The overall architectural challenge of Big Data is that just as Data can vary, so must architectures. Batch Interactive Real Time Hadoop can operate with O(n) over Petabytes of data Drill, Stinger and Tez bring must quicker querying Storm allows enormous scalable throughput
Batch Process Batch Static Data Precompute Serving Operational Data Store User Query Speed Data Stream Real Time Increment
Batch Static Data HDInsight Precompute Serving User Query SQL/NoSQLDatabase Speed Data Stream Increment Storm
Blob Persist Elastacloud Service Bus Spout Windows Azure Service Bus
AWS didn’t work Mediatonic are a leading UK games company They had deployed an analytics platform to AWS They required quite a bit of manpower to maintain their system operationally Their biggest cost was an Amazon Redshift data warehouse which was a bottleneck in their design Their system was built so that if a server went down whilst collecting gaming messages they could lose 15 minutes of data They found the “Big Data” part so complex that only highly-paid specialists could improve the system
How Windows Azure Helped A joint collaboration between Microsoft, Elastacloud and Mediatonic migrated their Game Fuel platform to Windows Azure With the Windows Azure Service Bus we were able to ensure that their messaging was continuous and not discrete so no messages were ever lost With HDInsight we were able to teach their staff how to build “Big Data” applications We were able to deliver a system with unique components of Windows Azure, removing the expensive data warehousing bottleneck but delivering the same functionality
“I can’t believe that I could get up and running with HDInsight doing real work in 30 minutes. It took me days to do the same thing with Elastic Map Reduce on Amazon” Adam Fletcher – Gamefuel Project Lead
Initial State Buy the Look Traditional Analytics
Limited Traditional Analytics allows for the tracing of single actions on a website; who added a product to a bag, where did they come from, where did they go?
Advancement Cross Sell Buy the Look
Improvements possible Beyond immediacy is the ongoing workflow of a system. Who added items to a basket after using the Buy the Look functionality. However, this lacks comprehension of customer behaviour.
Cutting edge Who bought this?
Cutting edge – understand your customer February January Purchase propensity Time
Proving through Big Data technologies that engaged customers will return through engagement following life events such as pay-days to continue the engagement. Building business on this basis. http://www.pass4sureexam.co/70-534.html