1 / 28

Database and the Cloud IGT Cloud Computing WG 20/4/2009

Database and the Cloud IGT Cloud Computing WG 20/4/2009. moshe.kaplan@rocketier.com http://top-performance.blogspot.com. RockeTier Methodology. RockeTier specializes in analyzing and boosting existing software performance and developing efficient high-load systems with a unique methodology:

vin
Download Presentation

Database and the Cloud IGT Cloud Computing WG 20/4/2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Database and the CloudIGT Cloud Computing WG 20/4/2009 moshe.kaplan@rocketier.com http://top-performance.blogspot.com

  2. RockeTier Methodology RockeTier specializes in analyzing and boosting existing software performance and developing efficient high-load systems with a unique methodology: • Detect • Rate • Immediate Effective Relief • Roadmap • Scale up and Scale out

  3. Assumptions • Cloud Computing  Virtualization • Virtualization  Low end servers • Very large databases  High end servers • Therefore: • Very large databases ≠ Cloud Computing

  4. Assumptions…

  5. Major Options

  6. Presentation Objectives • Who is using MySQL? • MySQL Limitations • How to get over this? • Move to another DB and scale up… • Vertical Sharding • Horizontal Sharding • Sharding test case

  7. Who is Using MySQL?

  8. MySQL Limitations • Table sizes: 50-100M records per table • Reads: 50 queries/second

  9. Why Do I Care? • From 0 to 100 (US mass adaptation) • Phone: 100 yrs • Radio: 40 yrs • TV: 30 yrs • Mobile: 20 yrs • Internet: 10 yrs • Facebook: 2 yrs

  10. 100K New Users/Week

  11. The Network Effect

  12. What Should I Do? • Oracle • SQL Server • $$$

  13. Sharding

  14. Vertical Sharding

  15. Horizontal Sharding Mod 10 = 0 Mod 10 = 1 • Static Hashing • Complex growth • Simple Mod 10 = 2 Mod 10 = 3 Mod 10 = 4 Mod 10 = 5 Mod 10 = 6 Mod 10 = 7 Mod 10 = 8 Mod 10 = 9

  16. Horizontal Sharding • Key locations are defined in a directory • Simple growth • Directory is SPOF

  17. Horizontal Sharding • Static hashing with directory mapping • Simple growth • Small Directory still SPOF Mod 1000 = 4

  18. Horizontal Sharding • Each key signed by DB# generated on creation • Simple growth • New key generation is SPOF

  19. Sharding Management • No mature tools in the market • Hibernate Shards – not recommended • Hibernate… • Beta • Required Mechanisms • Distribution of changes in DB schema

  20. Reporting

  21. Best Practices • $connection = new_db_connection("customer://1234"); • $statement = $connection->prepare( $sql_statement, $params ); • $result = $statement->execute();

  22. Lessons • Vertical Sharding: • User Actions, Users, Comments, Items • Horizontal Sharding • Denormalization • MySQL Replication

  23. Lessons • Vertical Sharding: • User Actions, Users, Comments, Items • Horizontal Sharding • Denormalization • MySQL Replication

  24. Lessons • 100M views per day • The path to Sharding: • Single server • Single master with multiple read slaves • Partitioned • Sharding

  25. Lessons • Master-Master replication • Each Shard is 50% loaded • 40K queries/second

  26. Ad Network Reference Architecture

  27. The Bottom Line: Grow ∞

  28. Startup your Engines Thank you moshe.kaplan@rocketier.com http://top-performance.blogspot.com

More Related