1 / 25

Distributed Load Balancing for Key-Value Storage Systems

Distributed Load Balancing for Key-Value Storage Systems. Imranul Hoque Michael Spreitzer Malgorzata Steinder. Key-Value Storage Systems. Usage: Session state, tags, comments, etc. Requirements: Scalability Fast response time High availability & fault tolerance

india
Download Presentation

Distributed Load Balancing for Key-Value Storage Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Load Balancing for Key-Value Storage Systems ImranulHoque Michael Spreitzer MalgorzataSteinder

  2. Key-Value Storage Systems • Usage: • Session state, tags, comments, etc. • Requirements: • Scalability • Fast response time • High availability & fault tolerance • Relaxed consistency guarantee • Example: Cassandra, Dynamo, PNUTS, etc.

  3. Load Balancing in K-V Storage • Hash partitioned vs. range partitioned • Range partitioned data ensures efficient range scan/search • Hash partitioned data helps even distribution THU SUN MON TUE SAT WED FRI SUN FRI SAT MON Table Tablets THU TUE WED Server 3 Server 4 Server 1 Server 2

  4. Issues with Load Balancing • Uneven space distribution due to range partitioning • Solution: partition the tablets and move them around • Few number of very popular records SUN FRI SAT MON THU TUE WED Server 3 Server 4 Server 1 Server 2

  5. Contribution • Algorithms for solving the load balancing problem • Load = space, bandwidth • Evenly distribute the spare capacity • Distributed algorithm, not a centralized one • Reduce the number of moves • Previous solutions: • One dimensional/key-space redistribution/bulk loading

  6. Outline • Motivation • System modeling and assumptions • Algorithms • One-to-one • One-to-n • Move suppression • Design decisions • Experimental results Emulation of proposed distributed algorithms • Future works

  7. System Modeling and Assumptions B1, S1 B1, S1 B4, S4 Server A Tablet B5, S5 BA, SA B2, S2 Tablet Server B BB, SB B3, S3 Table Tablet Server C BC, SC <= 0.01 in both dimensions 2. # of tablets >> # of nodes

  8. System State Target Zone: helps achieve convergence S Target Point B Goal: Move tablets around so that every server is within the target zone

  9. Load Balancing Algorithms • Phase 1: • Global averaging scheme • Variance of the approximation of the average decreases exponentially fast • Phase 2: • One-to-one gossip • One-to-n gossip • Move suppression t Phase 2 Phase 2 Phase 1 Phase 1

  10. One-to-One Gossip • Point selection strategy • Midpoint strategy • Greedy strategy • Tablet transfer strategy • Move to the selected point with minimum cost (space transferred)

  11. Tablet Transfer Strategy Server 2 Target for Server 1 Server 1 S B

  12. Tablet Transfer Strategy (2) • Start with an empty bag • Goal: take vectors from the servers so that they add up to the target vector • If slope(bag + left + right) < slope(target): • Add right to bag, move right • Otherwise, add left to bag move left Server 1 Right Left

  13. Initial Configurations Uniform Two Extreme Mid Quadrant

  14. Point Selection Strategy • Midpoint Strategy + Guaranteed convergence + No need to run phase 1 • Lots of extra movement • Visualization Demo • Uniform • Two extreme • Mid quadrant Server 2 S Server 1 B

  15. Point Selection Strategy (2) • Greedy Strategy • Take the point closer to the target • Move it to the target, if • improves the position of the other point • does not worsen by more than δ • Reduces movement Server 2 Server 1 Takes long time to converge in some cases

  16. DHT-based Location Directory

  17. DHT + Midpoint • Greedy + fallback to DHT: • Convergence problem exists for some configurations • Visualization Demo • Solution: • Greedy + fallback to DHT with Midpoint • Demo: uniform, two extreme, mid quadrant • Alternate approach: • Greedy + fallback to Midpoint • Trade-off: movement cost vs. DHT overhead

  18. Experimental Evaluation • Uniform configuration • Greedy + DHT (Midpoint) • Midpoint • Greedy + Midpoint (No DHT) • Effect of varying target zone • Effect of failed gossip count • Metrics • Amount of space moved • # of gossip rounds • Multiple tablet move

  19. Uniform Configuration: Results

  20. Effect of Varying Target Zone Larger target zone = fast convergence, less accuracy Target zone width should depend on the target point value

  21. Effect of Failed Gossip Count (Greedy) Large failed gossip count = More time in greedy mode, more unproductive gossip at the end

  22. One-to-N Gossip • Contact a few random nodes • Locked/unlocked mode • Pick the most profitable one • Distance from the target is minimized • Advantage • Better choices • Initial results • Locked mode: may lead to deadlock • Unlocked mode: most of the cases other nodes start transfer

  23. Move Suppression • Two global stages • Stage 1: • One-to-One gossip, but moves are hypothetical • Stage 2: • Change to chosen placement • Advantage • Tablet not moved multiple times • Challenges • When to switch to Stage 2 from Stage 1

  24. Future Works • Handling initial placement • Frequency of running the placement algorithm • Considering the network hierarchy • Handling failures • Extending to heterogeneous resources Questions?

More Related