1 / 21

Tamar Bercovici Box Engineering @TamarBercovici @ BoxEng

One to Many: A Database Scaling Story. Tamar Bercovici Box Engineering @TamarBercovici @ BoxEng. 2013. The goal of every web start-up…. When usage takes off, the scalability of your frameworks is put to the test…. “Scalability” is the ability to maintain performance under growing load.

amy
Download Presentation

Tamar Bercovici Box Engineering @TamarBercovici @ BoxEng

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. One to Many:A Database Scaling Story Tamar Bercovici Box Engineering @TamarBercovici @BoxEng 2013

  2. The goal of every web start-up… Grace Hopper Conference 2013

  3. When usage takes off, the scalability of your frameworks is put to the test… Grace Hopper Conference 2013

  4. “Scalability” is the ability to maintain performance under growing load Grace Hopper Conference 2013

  5. Two Scaling Approaches Server Server Server Server Server Server Server Vertical Horizontal Grace Hopper Conference 2013

  6. CHAPTER 1: Good Problems

  7. The simplest way for enterprises to share and access data Every file on Box => One row in the files table More users => More files => More DB load Grace Hopper Conference 2013

  8. Two years ago... MySQL 1M DB 1.7B millionsof queries USERS lines of PHP 1 day tensofmillionsof FOLDERS hundredsofmillionsof FILES Grace Hopper Conference 2013

  9. We started to have good problems… We’re so awesome!! We need to scale our database!!! Oh **** !! We need to scale our database… Do we partition data? Mirror it?... Both? What about NoSQL DBs?! Cluster DBs?!?! We decided to use sharding Grace Hopper Conference 2013

  10. So what is sharding anyway? Database T1 T2 Shard1 Shard2 T1 T2 T2 T1 Shard4 Shard5 Shard6 Shard7 T2 T2 T1 T1 T1 T1 T2 T2 True horizontal scaling Grace Hopper Conference 2013

  11. CHAPTER 2: In Search of the Perfect Design V1

  12. Scaling is an optimization problem. Sam Schillace Senior VP of Engineering @ Box Grace Hopper Conference 2013

  13. Three core components of a sharding architecture What Where How SOLVE PAIN POINT Shard File and Folder tables FLEXIBILITY TO CO-LOCATE Lookup / Mapping Database MINIMIZE DB QUERIES Shard by user Grace Hopper Conference 2013

  14. Querying the Shards • SELECT * FROM file WHERE user_idIN (123, 456); Mapping DB • Look up shards forusers 123, 456 • Shard ids 1,2 Shard1 • File1, Folder3, user123 • File2, Folder5, user123 Shard2 • File8, Folder4, user456 Grace Hopper Conference 2013

  15. CHAPTER 3: Making the Transition Without breaking the site…

  16. Scaling = replacing all the components of a car while driving it at 100mph. Mike Kreiger Co-founder @ Instagram Grace Hopper Conference 2013

  17. The key is to make incremental changes OLD NEW application shard0 DB Grace Hopper Conference 2013

  18. CHAPTER 4: TADA!!

  19. Where we are today more than 3X 4servers 100M FOLDERS queries over 40 shards 1 day 60% of queries offloaded from main database billions of FILES mapping 95% reads Grace Hopper Conference 2013

  20. Lessons Learned • Horizontal scaling rocks • Scaling is about optimizing for your use-case • Incremental changes reduce risk • Working on scaling challenges is very fun!! ridiculously, amazingly, awesomely FUN!!! Grace Hopper Conference 2013

  21. The End Rate and Review the session using the GHC Mobile App To download visit www.gracehopper.org 2013 Email tamar@box.com Twitter @TamarBercovici @BoxEng Tech blog tech.blog.box.com

More Related