820 likes | 1.07k Views
Big Data Hands-On Labs:. Or d ownload : Big Data Lite Virtual Machine. Oracle Big Data Appliance for Customers and Partners. Jean-Pierre Dijcks Oracle Big Data Product Management Paul Kent SAS VP Big Data. Oracle Big Data Appliance for Customers and Partners. 1.
E N D
Big Data Hands-On Labs: Or download: Big Data Lite Virtual Machine
Oracle Big Data Appliance for Customers and Partners Jean-Pierre Dijcks Oracle Big Data Product Management Paul Kent SAS VP Big Data
Oracle Big Data Appliance for Customers and Partners 1 Big Data Appliance Recap Why You Should Consider Big Data ApplianceDriving Business Value with SAS on Big Data Appliance Q&A 2 3 4
Oracle Big Data Management System Oracle Big Data SQL Oracle Database Oracle IndustryModels Oracle Advanced Analytics Oracle Spatial & Graph Cloudera Hadoop Oracle NoSQL Database Oracle R Advanced Analytics for Hadoop Oracle R Distribution Oracle Database Oracle Advanced Security Oracle Advanced Analytics Oracle Spatial & Graph Oracle Big DataConnectors Oracle DataIntegrator Big Data Appliance OracleExadata SOURCES
Recap: Big Data Appliance Overview Big Data Appliance X4-2 Sun Oracle X4-2L Servers with per server: 2 * 8 Core Intel Xeon E5 Processors 64 GB Memory 48TB Disk space Integrated Software: Oracle Linux, Oracle Java VM Oracle Big Data SQL* Cloudera Distribution of Apache Hadoop – EDH Edition Cloudera Manager Oracle R Distribution Oracle NoSQL Database * Oracle Big Data SQL is separately licensed
Recap: Standard and Modular • Starter Rack is a fully cabled and configured for growth with 6 servers • In-Rack Expansion delivers 6 server modular expansion block • Full Rack delivers optimal blend of capacity and expansion options • Grow by adding rack – up to 18 racks without additional switches
Recap: Harness Rapid Evolution • BDA 4.0 – Sept 2014 • Big Data SQL • Node Migration • BDA 2.x – April 2013 • Starter Rack • In-Rack Expansion • EM Integration • BDA 3.x – April 2014 • CDH 5.0 (MR2 & YARN) • AAA Security • Encryption • BDA 1.0 – Jan 2012 • Initial BDA • Mammoth Install
Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data
Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data • Oracle Big Data SQL • Oracle SQL on ALL your data • All Native Oracle SQL Operators • Smart Scan for Optimized Performance • Oracle Security • Govern all Data through a Single Set of Security Policies
Oracle Big Data SQL – A New Architecture • Powerful, high-performance SQL on Hadoop • Full Oracle SQL capabilities on Hadoop • SQL query processing local to Hadoop nodes • Simple data integration of Hadoop and Oracle Database • Single SQL point-of-entry to access all data • Scalable joins between Hadoop and RDBMS data • Optimized hardware • Balanced Configurations • No bottlenecks Oracle Confidential – Internal/Restricted/Highly Restricted
Big Data SQL SELECT w.sess_id, c.name FROM web_logs w, customers c WHERE w.source_country = ‘Brazil’ AND w.cust_id = c.customer_id; Relevant SQL runs on BDA nodes Big Data SQL 10’s of Gigabytes of Data WEB_LOGS CUSTOMERS Only columns and rows needed to answer query are returned Hadoop Cluster Oracle Database
Big Data SQL SELECT w.sess_id, c.name FROM web_logs w, customers c WHERE w.source_country = ‘Brazil’ AND w.cust_id = c.customer_id; • SQL Push Down in Big Data SQL • Hadoop Scans on Unstructured Data • WHERE Clause Evaluation • Column Projection • Bloom Filters for Better Join Performance • JSON Parsing, Data Mining Model Evaluation Relevant SQL runs on BDA nodes Big Data SQL 10’s of Gigabytes of Data WEB_LOGS CUSTOMERS Only columns and rows needed to answer query are returned Hadoop Cluster Oracle Database
Oracle Communications Data Model Reference Architecture Oracle Comms Apps (BSS/OSS) ETL/ELT Adapters Customer Experience Big Data Platform(Hadoop/NoSQL) Oracle CommsNtwk Products (Tekelec & Acme) Real-Time Adapters Operations Other Oracle Apps (CRM, ERP, etc.) Relational Data Warehouse (OCDM) ThirdParty Monetization Third Party Sources Data Management Analytic Apps Adapters DataSources Feedback Loop To Other Apps
Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data
Core Design Principles for Big Data Appliance Operational Simplicity Simplify Access to ALL Data • No Bottlenecks • Full Stack Install and Upgrades • Simplified Management • Cluster Growth • Critical Node Migration • Always Highly Available • Always Secure • Very Competitive Price Point
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day 1 • 12 node BDA for Production • Hadoop HA and Security Set-up • Ready to Load Data Full install with a single command: ./mammoth –i rck_1 RCK_1
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day 1 RCK_1 Example Service: Hadoop Name Nodes N N
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day 90 Add 12 New Nodes across two Racks Cluster expansion with a single command: mammoth –e newhost1,…,newhostn RCK_2 RCK_1 N N
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Cluster Expansion with a single command: mammoth –e newhost1,…,newhostn RCK_2 RCK_1 This expansion automatically optimizes HA setup across multiple racks N Because of uniform nodes and IB networking,no data is moved N
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues Day n Critical Node Failure => Primary Name Node RCK_2 RCK_1 N N
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues RCK_2 RCK_1 N N Automatic Failover to other NameNode Automatic Service Request to Oracle for HW Failure
Successful Big Data Systems Grow From Cluster Install with HA to Large Clusters to Dealing with Operational Issues RCK_2 RCK_1 N N Restore HA with a Single command bdacliadmin_cluster migrate N1 Reinstate the Repaired Node with a Single Command: bdacliadmin_clusterreprovision N1
Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board Core Design Principles for Big Data Appliance Operational Simplicity 30% Quicker to Deploy “Oracle Big Data Appliance is an excellent choice for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.” 21% Cheaper to Buy
Big Data Initiative @ Oracle Global Support Services Real-time access to better data means better insights, which means better decisions and better business results Integrate data associated with customer telemetry, configurations, service history, diagnostics, knowledge & support information Anticipate Detect Predict Automate Delight
Core Design Principles Enable Success Operational Simplicity Simplify Access to ALL Data
There is one more thing… • Business Value = Applications
Big Data Appliance powers instant Business Value Customer Experience Management CommunicationsData Model Cyber SecuritySolutions
Introducing • Paul Kent - SAS
Big Data and Big Analytics – So Much more Gunpowder! Paul Kent VP BigData, SAS Research and Development
[CON8279] Oracle Big Data Appliance: Deep Dive and Roadmap for Customers and Partners Oracle Big Data Appliance is the premier Hadoop appliance in the market. This session describes the roadmap for customers in the areas of high-performance SQL on Hadoop and securing big data, plus overall performance improvements for Hadoop. A special focus in the session is the roadmap and benefits Oracle Big Data Appliance brings to Oracle partners. To illustrate the benefits of running on a standardized and optimized Hadoop platform, SAS presents the findings of its tests of SAS In-Memory Analytics on Oracle Big Data Appliance.
SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda
Reflection on a stronger partnership than ever • Both leaders in Big Data – • Jointly solving the most difficult and demanding Big Data Problems • Providing simplicity and agility to create flexible configurations • Extensive engineering collaboration • Can we answer: • How Does it Work? • How Does it Perform? 2014
the tamoxifen dilemma SOURCE: http://commons.wikimedia.org/wiki/File:Tamoxifen-3D-vdW.png
SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda
Elephant :: 3 Good Ideas !! Never forgets Is a good (hard) worker Is a Social Animal (teamwork)
MPP (Massively Parallel) hardware running database-like software “data” is stored in parts, across multiple worker nodes “work” operates in parallel ,on the different parts of the table Hadoop – Simplified View Controller Worker Nodes
Idea #2 – MapReduce – Send the work to the Data • We Want the Youngest Person in the Room • Each Row in the audience is a data node • I’ll be the coordinator • From outside to center, accumulate MIN • Sweep from back to front. • Youngest Advances
SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda
Recap: Standard and Modular • Starter Rack is a fully cabled and configured for growth with 6 servers • In-Rack Expansion delivers 6 server modular expansion block • Full Rack delivers optimal blend of capacity and expansion options • Grow by adding rack – up to 18 racks without additional switches
Oracle Big Data SQL – A New Architecture • Powerful, high-performance SQL on Hadoop • Full Oracle SQL capabilities on Hadoop • SQL query processing local to Hadoop nodes • Simple data integration of Hadoop and Oracle Database • Single SQL point-of-entry to access all data • Scalable joins between Hadoop and RDBMS data • Optimized hardware • Balanced Configurations • No bottlenecks Oracle Confidential – Internal/Restricted/Highly Restricted
Diversity. It’s a good thing! Impala Nyala
SAS & Oracle Partnership Family Stories Hadoop Oracle Engineered Systems Family SAS Software Family Deployment Patterns Agenda
4 Important Things #1 Join the Family
SAS ACCESS to Hadoop HADOOP SAS SERVER Hive QL #2 Be Familiar