Tight Bounds for Parallel Randomized Load Balancing Christoph Lenzen and Roger Wattenhofer

1. Tight Bounds for Parallel Randomized Load Balancing Christoph Lenzen and Roger Wattenhofer

2. What is Load Balancing? optimizing storage utilization work sharing hashing low-congestion routing

3. An Example of Parallel Load Balancing fully connected network small & equal bandwidths (one message/round) n nodes need to send/receive up to n messages minimize number of rounds

4. An Example of Parallel Load Balancing fully connected network small & equal bandwidths (one message/round) n nodes need to send/receive up to n messages minimize number of rounds

5. An Example of Parallel Load Balancing fully connected network small & equal bandwidths (one message/round) n nodes need to send/receive up to n messages minimize number of rounds ? ? but... ?

6. Abstraction: Parallel Balls-into-Bins • distribute nballs into nbins • replace knowledge by randomization! • n instances (one for each receiver) = =

7. Naive Approach: Fire-and-Forget throw balls uniformly independently at random (u.i.r.) max. load with high probability (w.h.p.)

8. The Power of Two Choices (e.g. Azar et al., SIAM J. of Comp.‘99) inspect two bins and decide take least loaded max. load w.h.p. d choices: w.h.p. possible ...but not parallel! Vöcking, JACM‘92

9. The Parallel Power of Two Choices strongest upper bound: max. load in r rounds tight for constant r ...and certain algorithms: non-adaptive (fix bins to contact in advance) symmetric (all choices uniform) Stemann, SPAA‘96 Adler et al., STOC‘95

10. An Adaptive Algorithm contact one bin every bin accepts one ball ≈ n/e balls remain (w.h.p.) contact 2<e bins < n/e2 balls remain (w.h.p.) contact k<e2bins, and so on

11. The Power of the Tower termination in rounds cap # contacted bins at total messages w.h.p. in exp. (for each ball and bin) can enforce max. load2 tolerates message loss & faulty bins

12. Optimality? for symmetric algorithms: many balls in symmetric trees for rounds balls cannot contact many bins w/o incurring messages if balls in such a tree terminate root gets expected load many such trees => max. load w.h.p. root

13. No Faster Solution Possible, unless... bin loads of are accepted, bins have "identities" known to all balls messages are used place s balls at once

14. Exploiting Asymmetry/Bin ID‘s subsets of bins contact random “leader” bin “leader” bins distribute balls in their subset can place balls right away continue with previous algorithm max. load3 in rounds ? ? ? #2 #3 #1

15. How to Use Messages balls “coordinate” constant fraction of bins each ball contacts bins balls find coordinated bins coordinatorsassign balls to “their” bins proceed with symmetric algorithm

16. Summary optimal symmetric solution max. load 2, mess., rounds constant-time if: global enumeration of bins messages max. load

17. ...hey, What Happened to the Original Problem?! can be solved in rounds can be used to sort keys inrounds Patt-Shamir and Teplitsky, PODC‘11

18. Thank You! Questions or Comments?