1 / 29

ElasTraS: Database for the Cloud

ElasTraS is an elastic, self-managing database system for the cloud, addressing challenges of large-scale data management through schema level partitioning, live data migration, and advanced transaction management. Explore the architecture, scalability, and fault-tolerance capabilities of ElasTraS for mission-critical applications.

plante
Download Presentation

ElasTraS: Database for the Cloud

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. ElasTraS: Database for the Cloud CSE880 Professor Pramanik Shayne Rashedan & Chongguang Bi 1

  2. Agenda • Background • Motivation and Challenges • Examples • Architecture • Schema Level Partitioning • Live Database Migration • Evaluation 2

  3. Pay per use model 3

  4. What is cloud computing Software as Service, Apps are located in Cloud, software experiences are delivered through the internet Platform as Service, Internet based platforms for developers who want to create services but don’t want to build their own cloud Infrastructure as Service, Cloud users rent storage, computation and maintenance from cloud providers (pay per use utilities) Data as Service, Infrastructurefor web-scale data mining and knowledge discovery 4

  5. Large numbers of application databases (or tenants) with diverse schemas, unpredictable load and transactions. Scaling out using clusters of commodity servers and sharing resources among tenants (multitenancy) Pay-Per-Use: paying a small subscription fee on per usage basis every time the software is being used (e.g. hourly) Elastic Scaling: When the load increases, add more servers to the database tier to distribute the load, and vice versa. Fault-tolerant, self-managing and highly available to support mission critical applications Challenges 5

  6. ElasTranS: Elastic, Scalable, Self-Managing 6

  7. Key-value stores Bigtable, PNUTS and Dynamo, serving hundreds of thousands of concurrent requests Schema Level Partitioning Microsoft SQL Azure ,Relational Cloud and a prototype from Yahoo!, standard RDBMSs (such as SQL Server or MySQL) Decoupled shared storage Project Deuteronomy and Hyder, persistent data layer is separate from the logic that executes operations on that data,on the other hand in Amazon EC2, each node is data node and the transaction processor Examples: 7

  8. Live data migration VM pages, iterative phase-by-phase copying pages, migrates live processes and network connections using OS and networking primitives Zephyr, for shared nothing database architectures such as Cloud SQL Server or Relational Cloud. The persistent data is stored in disks locally attached to every node. Hence, the persistent data must also be migrated Advanced transactions at Scale, Megastore, Percolator, ecStore and G-Store Sinfonia, two phase commit (2PC) protocol to guarantee atomicity of operations in a distributed set of nodes Examples: 8

  9. Transaction Manager Row Groups Architecture of ElasTraS Server Owning Transaction Manager Partitions of Data 9 Append-only Multiversion Data

  10. Routing OTM Routing Architecture of ElasTraS Routing Data Storage Data Storage OTM OTM Routing Routing Data Storage OTM Data Storage User / Client TM Master Routing 10

  11. Static Partitioning 11

  12. Online Multi-player Game 12

  13. Row Groups 13

  14. Row Groups 14

  15. Row Groups = Partitions 15

  16. Why TPC-C Tree Schema? When amounts of data and transactions are large, assign some jobs to more nodes…. Assign all data and transactions related to Michigan State University Football Team to another node? How?? 16

  17. Why TPC-C Tree Schema? Partition Keys 17

  18. Distribute Transactions? Snapshot & Multiversion Data Problem 1 Read A Read Lock? Write Lock? Manager Data Read B Problem 2 Read Manager Data A Write B 18

  19. Snapshot & Multiversion Data Problem 1 Just Read Snapshot! Read A OTM Data Read B Problem 2 Just Read Snapshot! Read OTM Data A While “A”s are reading, B can update database. Write B 19

  20. Albatross Migration Timeline Source Ownership Destination Ownership Steady State 1.Begin Migration 2.Iterative Copying 3.Atomic Handover Steady State Clients TM Master 20

  21. Minimum node unavailability achieved by iteratively copying database cache and state of active transactions Elastic Load Balancing, rule-based model, if OTMs load is above the threshold, new OTMs are added, if load is below a min-threshold, partitions are consolidated into a smaller number of nodes. Partition assignment algorithm optimization problem with Greedy Heuristic based on load statistics of partitions Atomic ownership handover, ensure efficient and ACID execution of transactions, transactions must execute during migration, synchronizing commit logs of two nodes Aborted transactions, minimize loss of data, restart aborted transactions in destination Failure Handling, how to recover migration state after failure Challenges 21

  22. User Data Elastic Load Balancing User Data User OTM OTM OTM OTM TM Master Data Data 22

  23. Benchmarks: Yahoo! Cloud Serving Benchmark Key-value stores 3 tables, 10 VARCHAR(100) column per table TPC-C Benchmark 9 tables describing wholesale supplier 5 Transactions: NewOrder, Payment, OrderStatus, Delivery, StockLevel Evaluation 23

  24. Comparing with Albatross: S&M: Stop and Migrate stop serving, flush, transfer ownership F&M: Flush and Migrate flush, stop serving, transfer ownership Evaluation 24

  25. User Data User Migration Elastic Load Balancing User User Data Data Data OTM OTM TM Master OTM OTM Data Data 25

  26. Evaluation Albatross: Low Unavailability Low Latency 26

  27. Evaluation Albatross: Almost no additional cost for read operations Migration costs much time 27

  28. Evaluation Albatross: Low Latency Long Migration Time 28

  29. Not Enough Scale Evaluation Scale is Large Enough Load is Low Migrate to Low Scale 29

More Related