420 likes | 577 Views
Demystifying Systems for Interactive and Real-time Analytics. The BigFrame Team. Duke University, Hong Kong Polytechnic University, and HP Labs. Analytics System Landscape. Streaming. Dataflow. MapReduce. Graph. Multi-tenant. MPP DB. Array DB. Columnar. Mixed. Text Analytics.
E N D
Demystifying Systems for Interactive and Real-time Analytics The BigFrame Team Duke University, Hong Kong Polytechnic University, and HP Labs
Analytics System Landscape Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics
Analytics System Landscape Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics SQL Server Parallel Data Warehouse Teradata Gamma DB2 PE Aster Greenplum Netezza
Analytics System Landscape Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics HP Vertica Redshift Vectorwise ParAccel
Analytics System Landscape Hadoop Hive HadoopDB Mahout Pig Streaming Tenzing Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics
Analytics System Landscape Dremel Dryad Spark SCOPE Drill Impala Stinger Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics
Analytics System Landscape Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics Bigtable HBase HANA Megastore Cassandra Spanner Druid Splunk
Analytics System Landscape Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics Storm Streambase ElasticSearch Pregel GraphLab GraphX Cloudera Search Cassovary SciDB MadLINQ HAMA Solr
Analytics System Landscape Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Mesos Text Analytics Serengeti YARN Cloud platforms
What does this mean for Big Data Practitioners? Streaming Dataflow MapReduce Graph Multi-tenant MPP DB Array DB Columnar Mixed Text Analytics
Gives them a lot of power! From: http://animeonly.org/Digital-Wallpapers/Digital-renders/Spiderman-95061p.html
Challenges for Practitioners • Features (e.g., graph data) • Performance (e.g., claims like System A is 50x faster than B) • Resource efficiency • Growth and scalability • Multi-tenancy Which system to use for the app that I am developing? App Developers, Data Scientists
Challenges for Practitioners Different parts of my app have different requirements Which system to use for the app that I am developing? App Developers, Data Scientists Compose “best of breed” systems OR Use “one size fits all” system? Managing many systems is hard! System Admins
Challenges for Practitioners Different parts of my app have different requirements Which system to use for the app that I am developing? App Developers, Data Scientists Total Cost of Ownership (TCO)? Managing many systems is hard! CIO System Admins
One Approach Categorize systems Develop a benchmark per system category
Useful, But … Terasort HiBench Linear Road TPC-H / TPC-DS Graph 500 PageRank RDF Benchmarks Star Schema Benchmark DFSIO SWIM Streaming Counting triangles GridMix MulTe Dataflow MapReduce Graph Multi-tenant MPP DB Array DB SS-DB Columnar Mixed MapReduce Vs. Parallel DB / Hive Benchmark (in HiBench)/ Berkeley Big Data Benchmark Information Extraction Benchmark Text Analytics Yahoo Cloud Serving Benchmark (YCSB) YCSB Variants CH-benchCHmark
Problem #1 May Miss the Big Picture Cannot capture the complexities and end-to-end behavior of big data applications and deployments: (i) Bottlenecks(ii) Data conversion, transfer, & loading overheads(iii) Storage costs & other parts of the data life-cycle(iv) Resource management challenges(v) Total Cost of Ownership (TCO)
Problem #2 Benchmark Give a man a fish and you will feed him for a day. Give him fishing gear and you will feed him for life. -- Anonymous Benchmark Generator
How a user uses BigFrame bigif (benchmark input format) BigFrame Interface Benchmark Generator bspec (benchmark specification) results Hive Benchmark Driver for System Under Test run the benchmark Map HBase Reduce System Under Test
bspec: Benchmark Specification System Under Test 3. Query streams Hive 4. Evaluation metrics Map HBase Reduce 2. Data refresh pattern 1. Data for initial load Time
What does the user (want to) specify? bigif (benchmark input format) BigFrame Interface
The 3Vs Velocity Streaming Variety Dataflow MapReduce Graph Multi-tenant Volume MPP DB Array DB Columnar Mixed Text Analytics
bigif: BigFrame’sInputFormat Data Volume Query concurrency & classes Data Variety Query Volume Small, medium, large Relational, text, array, graph Data Velocity Micro, Macro At rest, slow, fast Exploratory, Continuous Query Variety Query Velocity
Benchmark Generation bigif (benchmark input format) bspec (benchmark specification) Benchmark Generator 1. Initial data to load 2. Data refresh pattern 3. Query streams 4. Evaluation metrics bigif describes points in a discrete space of {Data,Query} X {Variety,Volume,Velocity} Benchmark generation can be addressed as a search problem within a rich application domain
Application Domain Modeled Currently E-commerce sales, promotions, recommendations Social media sentiment & influence Benchmark generation can be addressed as a search problem within a rich application domain
Application Domain Modeled Currently Tweets Item Web_sales Customer Relationships Promotion
Application Domain Modeled Currently Web_sales Promotion Item
Application Domain Modeled Currently
Benchmark Generation bigif (benchmark input format) bspec (benchmark specification) Benchmark Generator bigif describes points in a discrete space of {Data,Query} X {Variety,Volume,Velocity} 1. Initial data to load 2. Data refresh pattern 3. Query streams 4. Evaluation metrics BigFrame can generate Data, Queries, and Arrival Patterns with the user-specified {Variety,Volume,Velocity} requirements from the application domain
Use Case I: Exploratory BI Data Variety = {Relational} • Large volumes of relational data • Mostly aggregation and few joins • Can Spark’s performance match that of an MPP DB? Query Variety = Micro BigFrame will generate a benchmark specification containing relational data and (SQL-ish) queries
Use Case II: Complex BI • Large volumes of relational data Data Variety = {Relational, Text} • Even larger volumes of text data • Combined analytics Query Variety = Macro (application-focused instead of micro-benchmarking) BigFrame will generate a benchmark specification that includes sentiment analysis tasks over tweets
Use Case III: Dashboards Data Velocity = Fast • Large volume and velocity of relational and text data Query Velocity = Continuous (as opposed to Exploratory) • Continuously-updated Dashboards BigFrame will generate a benchmark specification that includes data refresh as well as continuous queries whose results change upon data refresh
Use Case IV: Does One Size Fit All? • Growing set of applications have to process relational, text, & graph data • Compose “best of breed” systems or use a “one size fits all” system? Data Variety = {Relational, Text, Graph} Query Variety = Macro BigFrame will generate a benchmark specification that includes composite workflows with relational, text, and graph analytics
Use Case V: Multi-tenancy and SLAs • Big data deployments are increasingly multi-tenant and need to meet SLAs Specified through Query Volume dimension BigFrame can generate a benchmark specification containing a specified number of concurrent query streams with class labels for queries (e.g., Batch, Interactive, or Streaming)
Working with the Community • First release of BigFrame planned for August 2013 • With feedback from benchmark developers (BigBench) • Open-source with extensibility APIs • Benchmark Drivers for more systems • Utilities (accessed through the Benchmark Driver to drill down into system behavior during benchmarking) • Instantiate the BigFrame pipeline for more app domains
Take Away • “Benchmarks shape a field (for better or worse) …” • -- David Patterson, Univ. of California, Berkeley • Benchmarks meet different needs for different people • End customers, application developers, system designers, system administrators, researchers, CIOs • BigFramehelps users generate benchmarks that best meet their needs