1 / 6

Scalable Structured Data Storage for Web 2.0

Framework for web scalability using Hbase, integrating with Ruby on Rails for cleaner interface and declarative constructs. Compares with relational databases. Risk and plan assessment included.

kbyrne
Download Presentation

Scalable Structured Data Storage for Web 2.0

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Michael Armbrust David Zhu Barret Rhoden Scalable Structured Data Storage for Web 2.0

  2. Objectives To provide a framework for Web applications to scale to YouTube or MySpace sizes Use an alternative data store more suitable for typical Web workloads (Hbase)‏ Willing to trade consistency for scalability and availability Integrate with Ruby on Rails to give developers a clean interface to the data store Use declarative constructs from Rails to express the application's needs from the data store

  3. State of the Art Current status of data storage for Web applications: Large relational databases running on expensive hardware Manual horizontal and vertical partitioning of data Requires redesign at each scaling milestone Other work and differences: C-Store/Vertica: Maintains full SQL semantics Dynamo: Optimized for Amazon's writes PNUTS: Hosted service, work in progress

  4. Our Idea Use a large-scale distributed database suitable for Web applications Relaxed consistency, No ad-hoc queries Can run on 1000+ of shared-nothing commodity servers Interface with ActiveRecord-like layer in Ruby on Rails Provides simple relationships and consistency guarantees between models has_many belongs_to searchable_by (for full-text search)‏ Pre-compute joins for quick reads

  5. Risks Hbase Performance Hbase is under development and may have implementation problems Rails Scaling Once we successfully remove the data store bottleneck from Rails, we may discover unknown bottlenecks at the Web Application processing layer

  6. Plan Workload Simple App with Simple workload Complicated App Joins Sessions Access Locality Full Fledged App Possibly use Sun's Rails benchmark in addition to our workload ActiveRecord Talk to Hbase Single Lookup No Joins Three Basic Joins Validations and Prefetching • Data Store • Scalability of Hbase • Determine comparisons with other stores • Define data layout • Indexing options • Hard off-line queries Key: - Week 8 - Week 10 (Mid-Course)‏ - Week 14 (End)‏

More Related