Database Scalabilty , Elasticity, and Autonomic Control in the Cloud

1. Database Scalabilty, Elasticity, and Autonomic Control in the Cloud Divy Agrawal Department of Computer Science University of California at Santa Barbara Collaborators: Amr El Abbadi, Sudipto Das, Aaron Elmore DASFAA'2011 Keynote Address

2. Outline • Infrastructure Disruption • Enterprise owned  Commodity shared infrastructures • Disruptive transformations: Software and Service Infrastructure • Clouded Data Management • State of the Art lacks “cloud” features • Alternative data management models • Application development landscape • Architecting Data Systems for the Cloud • Design Principles • Data Fusion and Fission • Elasticity • Autonomic Control DASFAA'2011 Keynote Address

3. WEB is replacing the Desktop DASFAA'2011 Keynote Address

4. Paradigm Shift in Computing DASFAA'2011 Keynote Address

5. Cloud Computing: Why Now? • Experience with very large datacenters • Unprecedented economies of scale • Transfer of risk • Technology factors • Pervasive broadband Internet • Maturity in Virtualization Technology • Business factors • Minimal capital expenditure • Pay-as-you-go billing model DASFAA'2011 Keynote Address

6. Economics of Data Centers • Risk of over-provisioning: underutilization • Money & Time Questions: • How much? • How Long? Capacity Resources Demand Time Static data center DASFAA'2011 Keynote Address

7. Economics of Internet Users • Heavy penalty for under-provisioning Resources Resources Resources Capacity Capacity Capacity Lost revenue Demand Demand Demand 2 2 2 3 3 3 1 1 1 Time (days) Time (days) Time (days) Lost users DASFAA'2011 Keynote Address

8. Economics of Cloud Computing • Pay by use instead of provisioning for peak Capacity Resources Resources Capacity Demand Demand Time Time Static data center Data center in the cloud DASFAA'2011 Keynote Address

9. The Big Picture • Unlike the earlier attempts: • Distributed Computing, Distributed Databases, Grid Computing • Cloud Computing is REAL: • Organic growth: Google, Yahoo, Microsoft, and Amazon • IT Infrastructure Automation • Economies-of-scale • Fault-tolerance: automatically deal with failures • Time-to-market: no upfront investment DASFAA'2011 Keynote Address

10. Cloud Reality • Facebook Generation of Application Developers • Animoto.com: • Started with 50 servers on Amazon EC2 • Growth of 25,000 users/hour • Needed to scale to 3,500 servers in 2 days (RightScale@SantaBarbara) • Many similar stories: • RightScale • Joyent • … DASFAA'2011 Keynote Address

11. Outline • Infrastructure Disruption • Enterprise owned  Commodity shared infrastructures • Disruptive transformations: Software and Service Infrastructure • Clouded Data Management • State of the Art lacks “cloud” features • Alternative data management models • Application development landscape • Architecting Data Systems for the Cloud • Design Principles • Data Fusion and Fission • Elasticity • Autonomic Control DASFAA'2011 Keynote Address

12. Scaling in the Cloud Client Site Client Site Client Site HAProxy (Load Balancer) Elastic IP Apache + App Server Apache + App Server Apache + App Server Apache + App Server Apache + App Server Database becomes the Scalability Bottleneck Cannot leverage elasticity MySQL Master DB MySQL Slave DB Replication DASFAA'2011 Keynote Address

13. Scaling in the Cloud Client Site Client Site Client Site HAProxy (Load Balancer) Elastic IP Apache + App Server Apache + App Server Apache + App Server Apache + App Server Apache + App Server Scalable and Elastic But limited consistency and operational flexibility Key Value Stores DASFAA'2011 Keynote Address

14. Internet Chatter DASFAA'2011 Keynote Address

15. Application Simplicity using DBMS public void confirm_friend_request(Alice, Bob) {  begin_transaction(); 
   update_friend_list(Alice, Bob); // Tokyo
   update_friend_list(Bob, Alice); // Hong Kong
 end_transaction(); }  DASFAA'2011 Keynote Address

16. Application Challenges usingKey-Value Stores public void confirm_friend_request(Alice, Bob){  try { update_friend_list(Alice, Bob); // Tokyo } catch(exception e) { report_error(e);  return;}  try { update_friend_list(Bob, Alice); // Hong Kong} catch(exception e) { report_error(e); revert_friend_list(Alice, Bob); 
  return; } } DASFAA'2011 Keynote Address

17. Eventual Consistency Model public void confirm_friend_request_B(Alice, Bob){  try { update_friend_list(Alice, Bob); // Tokyo} catch(exception e) {  add_to_retry_queue(<updatefriendlist, Alice, Bob>); }  try { update_friend_list(Bob, Alice); // Hong Kong}
 catch(exception e) { add_to_retry_queue(<updatefriendlist, Bob, Alice);}
} DASFAA'2011 Keynote Address

18. Eventual Consistency Challenge /* get_friends() method has to reconcile results returned by get_friends() because there may be data inconsistency due to a conflict because a change that was applied from the message queue is contradictory to a subsequent change by the user.  In this case, status is a bitflag where all conflicts are merged and it is up to app developer to figure out what to do. */   public list get_friends(user1){ 
      list actual_friends = new list();
      list friends = get_friends();        foreach (friend in friends){             if(friend.status == friendstatus.confirmed){ //no conflict
           actual_friends.add(friend);        }else if((friend.status &= friendstatus.confirmed) 
                   and !(friend.status &= friendstatus.deleted)){          // assume friend is confirmed as long as it wasn’t also deleted
          friend.status = friendstatus.confirmed;              
          actual_friends.add(friend); 
          update_friends_list(user1, friend, status.confirmed);        }else{ //assume deleted if there is a conflict with a delete
          update_friends_list( user1, friend, status.deleted)
        }      }//foreach   return actual_friends;
} I love eventual consistency but there are some applications that are much easier to implement with strong consistency. Many like eventual consistency because it allows us to scale-out nearly without bound but it does come with a cost in programming model complexity. It gets too complicated with Reduced Consistency Guarantees February 24, 2010 DASFAA'2011 Keynote Address

19. Outline • Infrastructure Disruption • Enterprise owned  Commodity shared infrastructures • Disruptive transformations: Software and Service Infrastructure • Clouded Data Management • State of the Art lacks “cloud” features • Alternative data management models • Application development landscape • Architecting Data Systems for the Cloud • Design Principles • Data Fusion and Fission • Elasticity • Autonomic Control DASFAA'2011 Keynote Address

20. Design Principles: Scalable Systems • Separate System and Application State • Limit Application interactions to a single node • Decouple Ownership from Data Storage • Limited distributed synchronization is feasible DASFAA'2011 Keynote Address

21. Scalability of Data in the Cloud • Data Fusion • Enrich Key Value stores [Gstore: ACM SoCC’2010, MegaStore: CIDR’2011] • Data Fission • Cloud enabled relational databases [ElasTras: HotCloud’2009, Relational Cloud: CIDR’2011,SQL Azure: ICDE’2011] DASFAA'2011 Keynote Address

22. Data Fusion DASFAA'2011 Keynote Address

23. Atomic Multi-key Access • Key value stores: • Atomicity guarantees on single keys • Suitable for majority of current web applications • Many other applications warrant multi-key accesses: • Online multi-player games • Collaborative applications • Enrich functionality of the Key value stores [Google AppEngine & MegaStore] DASFAA'2011 Keynote Address

24. GStore: Key Group AbstractionACM SoCC’2010 • Define a granule of on-demand transactional access • Applications select any set of keys • Data store provides transactional access to the group • Non-overlapping groups DASFAA'2011 Keynote Address

25. Horizontal Partitions of the Keys Key Group Keys located on different nodes A single node gains ownership of all keys in a KeyGroup Group Formation Phase DASFAA'2011 Keynote Address

26. Key Grouping Protocol • Conceptually akin to “locking” • Allows collocation of ownership • Transfer key ownership from “followers” to “leader” • Guarantee “safe transfer” in the presence of system dynamics: • Dynamic migration of data and its control • Failures DASFAA'2011 Keynote Address

27. Implementing GStore Application Clients Transactional Multi-Key Access Grouping Middleware Layer resident on top of a Key-Value Store Grouping Layer Transaction Manager Grouping Layer Transaction Manager Grouping Layer Transaction Manager Key-Value Store Logic Key-Value Store Logic Key-Value Store Logic Distributed Storage G-Store DASFAA'2011 Keynote Address

28. Latency for Group Operations DASFAA'2011 Keynote Address

29. Google MegaStoreCIDR’2011 • Transactional Layer built on top of BigTable • “Entity Groups” form the logical granule for consistent access • Entity group: a hierarchical organization of keys • “Cheap” transactions within entity groups • Expensive or loosely consistent transactions across entity groups • Use 2PC or Persistent Queues • Transactions over static entity groups DASFAA'2011 Keynote Address

30. Data Fission DASFAA'2011 Keynote Address

31. Elastic Transaction ManagementElasTras: HotCloud’2009, UCSB TR’2010 • Designed to make RDBMS cloud-friendly • Database viewed as a collection of partitions • Suitable for: • Large single tenant database instance • Database partitioned at the schema level • Multi-tenancy with a large number of small DBs • Each partition is a self contained database DASFAA'2011 Keynote Address

32. Application Clients Application Logic ElasTraS Client DB Read/Write Workload Metadata Manager TM Master Lease Management Health and Load Management Master Proxy MM Proxy OTM OTM Txn Manager Log Manager OTM P1 Pn P2 DB Partitions Durable Writes Distributed Fault-tolerant Storage DASFAA'2011 Keynote Address

33. Elastic Transaction Management • Elastic to deal with workload changes • Load balance partitions • Recover from node failures • Dynamic partition management • Transactional access to database partitions DASFAA'2011 Keynote Address

34. ElasTras: Throughput Evaluation DASFAA'2011 Keynote Address

35. Microoft Product: SQL AzureICDE’2011 Machine 5 Machine 4 Machine 6 SQL Instance SQL Instance SQL Instance SQL DB SQL DB SQL DB UserDB1 UserDB1 UserDB1 UserDB2 UserDB2 UserDB2 UserDB3 UserDB3 UserDB3 UserDB4 UserDB4 UserDB4 SDS Provisioning (databases, accounts, roles, …, Metering, and Billing Scalability and Availability: Fabric, Failover, Replication, and Load balancing DASFAA'2011 Keynote Address • Shared infrastructure at SQL database and below • Request routing, security and isolation • Scalable HA technology provides the glue • Automatic replication and failover • Provisioning, metering and billing infrastructure Scalability and Availability: Fabric, Failover, Replication, and Load balancing Slides adapted from authors’ presentation

36. MIT Project: Relational CloudCIDR’2011 • SQL Server replaced by Open-source database engines Workload driven tenant placement and consolidation [SIGMOD 2011] Workload driven database partitioning [VLDB 2010] DASFAA'2011 Keynote Address

37. Elasticity in the Cloud: Live Data Migration DASFAA'2011 Keynote Address

38. Elasticity • A database system built over a pay-per-use infrastructure • Infrastructure as a Service for instance • Scale up and down system size on demand • Utilize peaks and troughs in load • Minimize operating cost while ensuring good performance DASFAA'2011 Keynote Address

39. Elasticity in the Database Layer DBMS DASFAA'2011 Keynote Address

40. Elasticity in the Database Layer Capacity expansion to deal with high load – Guarantee good performance DBMS DASFAA'2011 Keynote Address

41. Elasticity in the Database Layer Consolidation during periods of low load – Cost Minimization DBMS DASFAA'2011 Keynote Address

42. Live Database MigrationA Critical operation for effective elasticity • Elasticity induced dynamics in a Live system • Minimal service interruption for migrating data fragments • Minimize operations failing • Minimize unavailability window, if any • Negligible performance impact • No overhead during normal operation • Guaranteed safety and correctness DASFAA'2011 Keynote Address

43. Shared storage architectureAlbatross: VLDB 2011 • Proactive state migration • No need to migrate persistent data • Migrate database cache and transaction state proactively • Iteratively copy the state from source to destination • Ensure low impact on transaction latency and no aborted transactions • Migrate transactions on-the-fly • Transactions start at source and complete at destination DASFAA'2011 Keynote Address

44. Albatross Cached DB State Transaction State Cached DB State Transaction State DBMS Node Tenant/DB Partition Tenant/DB Partition Destination Source Persistent Image DASFAA'2011 Keynote Address

45. Albatross: Evaluation Summary • Two transaction processing benchmarks • YCSB and TPC-C • Unavailability windowof 300-800ms • Naïve solutions: 2-4 secondunavailability • Nofailed requests • Naïve solutions: hundreds of failed requests • 15-30%increase in transaction latency after migration • Negligible performance impact during migration • Naïve solutions: 200-400% increase in latency • Data transferred: 1.3-1.6 times database cache • Naïve solutions: approximately the size of the cache Sudipto Das {sudipto@cs.ucsb.edu}

46. Shared nothing architecture Zephyr: SIGMOD 2011 • Reactive state migration • Migrate minimal database state to the destination • Source and destination concurrently execute transactions • Synchronized DUAL mode • Source completes active transactions • Transfer ownership to the destination • Persistent image migrated asynchronously on demand DASFAA'2011 Keynote Address

47. Zephyr Freeze Index Structures Destination Source DASFAA'2011 Keynote Address

48. Zephyr Evaluation Summary • Yahoo! Cloud Serving Benchmark • Nounavailability window • Naïve solution: 5-8 seconds • 50-100failed requests • Naïve solution: ~1000 • ~20%increase in transaction latency over entire workload • Naïve solution: ~15%increase in latency • Higher latency due to on-demand remote fetch • Data transferred: 1.02-1.04 time database size • Naïve solution: size of the database DASFAA'2011 Keynote Address

49. Autonomic Control: DBMS Administration in the Cloud DASFAA'2011 Keynote Address

50. Current State: Database Administration Significant Operational Challenges: • Provisioning for Peak Demand • Resource under-utilization • Capacity planning: too many variables • Storage management: a massive challenge • Software and system upgrades: extremely time-consuming • Complex mine-field of software and hardware licensing  Unproductive use of people-resources from a company’s perspective DASFAA'2011 Keynote Address