1 / 41

Big Data and Business Intelligence Virgil Dodson

Explore the concept of big data and its impact on business intelligence. Learn about the Eclipse BIRT project and survey results on big data usage and technologies. Discover the benefits and challenges of implementing big data initiatives.

patterson
Download Presentation

Big Data and Business Intelligence Virgil Dodson

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Big Data and Business IntelligenceVirgil Dodson

  2. Introduction to Big Data Eclipse Survey Results Independent Survey Results Introduction to BIRT Big Data Connections Live Demo Questions Today’s Agenda and Goals

  3. Big Data Definition Big data is a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. web logs RFID sensors social networks Internet text search indexes call detail records astronomy atmospheric info genomics biogeochemical biological military surveillance medical records photographs video large-scale e-commerce - Wikipedia

  4. IDC 2013 Big Data Predictions • The “Digital Universe” will expand to over 4 zettabytes… Over 50% growth from 2012 • The Big Data focus will shift “up the stack”, toward analytics and discovery, and analytic applications • Spending will reach $10 billion in 2013, over $20 billion by 2016 Source: IDC, IDC Predictions 2013 presentation

  5. Eclipse BIRT Survey – Oct/Nov 2012 • Big Data or Little Data - How Do You Display Yours?The Eclipse Foundation would like to better understand how developers are using Eclipse with big data and reporting projects. • We ran this survey to get the pulse of what technologies where in demand related to Eclipse/BIRT technologies. • Eclipse Promoted the Survey. • 60% of 518 responders claimed to be big data users

  6. Eclipse BIRT Survey - Technology Choices Note: Responders could choose more than one option

  7. Eclipse BIRT Survey - Other Mentions

  8. Eclipse BIRT Survey - Data Visualization

  9. Report/Visualization Tools Note: Responders could choose more than one option

  10. Independent Big Data Survey – Sept/Oct 2012 Goals: • How many large firms (>$1B) are conducting Big Data projects • What are such companies doing with their Big Data projects • What are the expected benefits for those Big Data initiatives • What are the inhibitors • King Research received 516 surveys • 316 completed and 200 partially completed surveys • Completed surveys were the primary source of analysis • 32% of those who completed survey (98 respondents) work at companies with revenue of $1B or more

  11. Independent Big Data Survey – Key Findings • 26% of large companies have Big Data projects. 40% have not evaluated Big Data or have evaluated and decided not to proceed. The balance (34%) are either evaluating or planning such initiatives. • “Not enough staff with expertise” and “Expected cost of Big Data initiatives” are the major inhibitors • Major benefits expected from Big Data initiatives are: • Make better decisions, faster • Gain competitive advantage • Improve efficiency • Improve customer targeting • Major benefits realized from Big Data initiatives are: • Gain competitive advantage • Improve customer targeting • Make better decisions, faster • Improve efficiency

  12. Independent Big Data Survey – Big Data Usage Does your organization have a Big Data implementation today? • More large companies have implemented Big Data projects (26%) than the universe of companies represented in this survey (19%) • Conversely, far fewer respondents at large companies responded “No” to this question (40% versus the universe of respondents 49%) $1B+ Revenue Universe of Respondents

  13. Independent Big Data Survey – Big Data Technologies What Big Data technologies do you plan to use? (eval/planning) • We asked about their planned use of 15 technologies, and the top 5, in descending order of frequency of mention are displayed above • Other technologies planned for use at $1B+ organizations include: Apache Cassandra, 12%; Hortonworks Hadoop, 12%;Amazon DynamoDB, 9%; Apache CouchDB, 9%; VoltDB, 9%; HyperTable, 6%; 10gen MongoDB, 3%; Datastax Cassandra, 3% $1B+ Revenue Universe of Respondents

  14. Independent Big Data Survey – Application Types What are likely to be your Big Data applications? (responses from those who are evaluating or planning Big Data implementations) • Our survey listed 23 frequently reported Big Data applications and when asked which of these they have evaluated or planned to use, they indicated an average 4.5 apps each. • Shown above are the 14 apps that were most frequently indicated

  15. Independent Big Data Survey – Number of End Users How many people in your organization will consume information from or use your Big Data applications? (evaluating/planning) • Clearly companies with revenues of $1B or greater plan to share their Big Data information with large audiences across their companies

  16. Actuate Launches the BIRT Project Actuate proposed and started BIRTBusiness Intelligenceand Reporting Tools Project … a top-level Eclipse project Actuate Joins Eclipse Foundationas Strategic Developerand Board Member Adds BI and Reportingas Open Source Project Professional open sourcePrimary development resources funded by Actuate Contributions from many sourcesIBM, Innovent Solutions and community AUGUST2004

  17. Business Intelligence and Reporting Tools A New Generation of Data Visualization Technology • Makes all data-driven content development easy • Modern, web-page design metaphor • Open and standards-based • Flexible with rich programmatic control • Full support for libraries and reuse • Foundation for a range of solutions Simplicity that makes simple layouts easy Power to createvery complexlayouts BIRT

  18. Ground-up initiative: Innovative approach to layout and design Developed in the open with community feedback at all stages BIRT Release History

  19. BIRT Example Key Capabilities Very Simple to Very Complex Layouts Listings, cross-tab, dashboard, pixel-perfect, charts … Grouping, advanced aggregations, sub-totals, calculations Multi-section and sub-reports Conditional sections and logic Full programmatic control/scripting Embedded images… Comprehensive Data Access SQL databases, Web Services, Flat Files, XML, scripted data sources … Multiple data sources in one design… Output Formats HTML, PDF, Excel, Word, PowerPoint… Internationalization of labels and text Bi-Directional language display • Re-use and Developer Productivity • Library support for publishing and sharing components • Leverages common standards (SQL, HTML, JavaScript, Java, XML) • Cascading Style Sheets • Built-in debugger… • Interactivity and Linking • Data driven hyperlinks • Drill-through charts and graphics… • Multiple Usage and Productivity Aids • Graphical layout and design • Query & metadata editors • Formatting Builder • Grouping Builder • Customizable cheat sheets and templates…

  20. Getting to Know BIRT DEMO

  21. BIRT Design Gallery Charts and Tables Listing with Groups and Sub-Totals

  22. BIRT Design Gallery Crosstab and Charts Crosstabs

  23. BIRT Design Gallery Forms Calendar / Schedule

  24. BIRT Design Gallery Multi-Language and Bi-Directional Dashboards

  25. BIRT Chart Gallery

  26. BIRT Chart Gallery

  27. BIRT Chart Gallery

  28. High-Level BIRT Architecture BIRT Designer EclipseDesigner Eclipse DTP, WTP,… Chart Designer Design Engine XMLDesign BIRT Engine Document Generation Services Charting Engine HTML PDF Excel Word PowerPoint PostScript … Data Data Services Presentation Services Data

  29. High Level BIRT Architecture Produces XML Report, Templates, and Library Designs DE API Design Engine Runs Reports and produces output – PDF, HTML, Doc, XLS, PS, PPT Etc RE API Report Engine Consume Chart EMF model and produces Chart Output. Supports 14 Main types and many sub types. Ouputs to PNG, JPG, BMP, SVG, PDF, SWT, and SWING CE API Chart Engine All Engines can be ran with or without OSGi Core BIRT Open Source Products Report Designer Chart Builder Example Viewer Can be ran outside of BIRT

  30. BIRT AJAX Based Viewer

  31. BIRT Data Access • BIRT Offers many ways to get data • Standard Data Sources • Flat File (CSV, TSV, SSV, PSV) • Hive Data Source • Cassandra Scripted Data Source • JDBC Textual or Graphical • Web Service - XPath syntax • XML - XPath syntax • XLS/XLSX • Scripted Data Source Written in Java or JavaScript • Open Data Access (ODA) DTP Project • Extensible JDBC Driver Framework

  32. Live Demo – New MongoDB ODA DEMO

  33. Connecting to Hadoop

  34. Hive JDBC – HQL Sub Query Example

  35. Hive JDBC – get_json_object UDF

  36. Hive JDBC – RegExP Example

  37. Hive JDBC – HQL Hints example

  38. Hive JDBC – Transform Example

  39. BIRT Exchange Community Site • Centralized hub for BIRT developers • Access demos, tutorials, tips and techniques, documentation… • Enables developers to be more productive and build applications faster • Marketplace for applications • Explore • Search/sort • Rate, comment • Forums • Download • Documentation • Software • Examples • Contribute • BIRT designs, code • Technical tips • Contests

  40. Plug in to BIRT Spring 2013 Contest Contest runs from March 28, 2013 to April 30, 2013 Plug-In Categories Open Data Access (ODA) Drivers Output Emitters Report Item Extensions Chart Extensions New iPad for Top 3Plug-Ins! • Visit BIRT Exchange for full contest details

  41. Questions? Big Data and Business IntelligenceVirgil Dodsonvdodson@actuate.com

More Related