1 / 18

THE BIG DATA ECOSYSTEM AND YOU !

THE BIG DATA ECOSYSTEM AND YOU !. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM:. Social Media and Consumer Sentiment. External Users. Backoffice (ERP). Internal Users. Statistics. Transactional Applications. Unstructured Data. ETL. Business Intelligence. Transactional Data ( OLTP).

zephania-stone
Download Presentation

THE BIG DATA ECOSYSTEM AND YOU !

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. THE BIG DATA ECOSYSTEM AND YOU!

  2. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Social Media and Consumer Sentiment External Users Backoffice (ERP) Internal Users Statistics Transactional Applications Unstructured Data ETL Business Intelligence Transactional Data (OLTP) Data Warehouse

  3. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Social Media and Consumer Sentiment • Transactional Data – Source Systems and/or Point of Sale • Vary Greatly from Company to company • EX: Insurance Policy and Claims Systems, Brokerage Trade Systems, Academic Enrollment Systems, Enery Smart Grids, Hospital Patient Management Systems, etc, etc….. External Users Backoffice (ERP) Internal Users Statistics Transactional Applications Unstructured Data ETL Business Intelligence Transactional Data (OLTP) • ERP Data – Operational Management, Planning and Feedback • EX: Supply Chain, Finance & Accounting, Inventory Management, HR, CRM, etc, etc... Data Warehouse

  4. Big Data Decision Point! – Data Mart or EDW P ETL, Replication, Data Quality Key Drivers: • What is the Size of your Source Data? • Should Big Data sit inside or Outside of your EDW? • Normalize or Denormalize? • Integrated Reporting or Isolated Reporting? Enterprise Data Warehouse Bulk & Trickle Loads SQL, ODBC, JDBC ETL, Replication, Data Quality App./Reporting front-end BI or Reporting Interface

  5. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Social Media and Consumer Sentiment • Moving data from one system to another and getting it right is often the most frustrating and time consuming piece of the BI Ecosystem puzzle • Quality - Time - $$$ (is good enough… good enough?) ETL – Extraction, Transformation, Loading Or ELT – Extraction, Loading, Transformation External Users Backoffice (ERP) Internal Users Statistics Transactional Applications Unstructured Data ETL Business Intelligence Transactional Data (OLTP) Data Warehouse

  6. Big Data Decision Point! – ETL, ELT or Data Virtualization? Key Drivers: • What is the Quality of your Source Data? • How complex is the model? • Can the database handle ELT batch rework in time? • Do you have the right tools (Informatica, CompositeSW)? • Can you live with the maintenance overhead, performace?

  7. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Social Media and Consumer Sentiment Data Warehouse(s?) Central Storage Tank for most the company’s most critical data True success requires Executive Evangelism The ecosystem lives or dies based on the performance of your data warehouse! External Users Backoffice (ERP) Internal Users • Often very difficult to ROI: “Are your users satisfied with your company’s BI delivery?” - Gartner Statistics Transactional Applications Unstructured Data ETL Business Intelligence Transactional Data (OLTP) Data Warehouse

  8. Old School Tech (Row Store) or New School Tech (Columnar, NoSQL)? AAPL AAPL BBY BBY AAPL NYASE NYAASE NYSE NYASE NGGYSE NYGGGSE NYSE NYSENYSE143.74NYSE NYSENYSE5/05/09 Key Drivers: • Can your database handle your data volumes? • Does the cost of performance tuning justify status quo? • Real-time or batch? • Commodity MPP or Big Iron? • Are you monetizing your data or heating your data center?

  9. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Social Media and Consumer Sentiment Business Intelligence and Advanced Analytics Is your corporate culture one that views Business Intelligence output as a key driver to your company’s success? Can analysts use BI output to make positive, effective, fact-based decisions? External Users Backoffice (ERP) Internal Users Statistics Transactional Applications Unstructured Data ETL Business Intelligence Transactional Data (OLTP) Data Warehouse

  10. In-Database Analytics or Application Based Analytics? Key Drivers: • Performance – Cubes or Relational? • Are you living in a canned report or ad-hoc world? • Does your business have highly advanced, specialized analytics needs? • Do your data mining users require the aid of an analytics GUI?

  11. BIG DATA ECOSYSTEM OVERVIEW DIAGRAM: Unstructured Data: The final piece of the big-data puzzle? Is there data floating around the you need to understand but can’t figure out how to analyze? Do you know what’s happening on your company’s network? How consumers feel about your product? If your call centers are actually helping your business? Social Media and Consumer Sentiment Web logs / Clickstream Social Media Call Center Consumer Sentiment Lobs, Blobs, and Clobs External Users Backoffice (ERP) Internal Users Statistics Transactional Applications Unstructured Data ETL Business Intelligence Transactional Data (OLTP) Data Warehouse

  12. Hadoop / Vertica: ETL Hadoop or not to Hadoop? Key Drivers: • Is it critical you understand your company’s messiest data? • Do you have the skills and know-how? • HDFS vsMapReduce • Could you take action on the information if you had it? • Exposing MapReduce results through BI or EDW • Are there alternatives to Hadoop? Computer Cluster Computer Cluster Data data datadatadatada data datadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata data datadatadatadatadata DFS Block 1 DFS Block 1 DFS Block 1 DFS Block 1 Map Map DFS Block 1 DFS Block 1 DFS Block 2 DFS Block 2 Map Map Reduce Reduce Hadoop / Vertica: Advanced Analytics DFS Block 2 DFS Block 2 DFS Block 2 DFS Block 2 Map Map DFS Block 3 DFS Block 3 DFS Block 3 DFS Block 3

  13. Big Data Trends - Data Governance: • Data About Your Data • Do you really understand your data? Do your users? • How many versions of the truth are there? • Does your organization have the commitment to build adhere to standards?

  14. Big Data Trends - Cloud: • You *MAY* put your data in the Cloud, but DON’T put your Head there! • Public cloud vs. Private Cloud vs. SaaS – What’s best for you? • Is it secure? Is it disaster-proof? • Does is perform? • Is it cost effective?

  15. Big Data Trends – Mobile: • If you’re not delivering Mobile BI to your user and customers, you’ve fallen behind. • Tools are making Mobile Delivery easier every day. • Your customers expect it. Your internal users *should* expect it. • Do you have a Mobile Data Security Strategy?

  16. Advice for Survival in the Big-Data world: • BE PROACTIVE IN TRANSFORMING YOUR BUSINESS! • STAY ON TOP OF NEW TECHNOLOGIES! • PARTNER WITH THE BUSINESS! • BE AWARE OF THE BIG DATA TECHNOLOGY YOU INTERACT WITH EVERY DAY, AND APPLY IT TO YOUR BUSINESS!

  17. thank you for your time…and good luck in your BIG DATA efforts!

  18. q&a

More Related