1 / 25

Randomized Smoothing Networks

Explore the benefits of Randomized Smoothing Networks for load balancing tasks in distributed systems, presented at IPDPS 2004. Learn about their random balancers, easy initialization, and smoothness properties. Discover how they outperform deterministic networks.

cdenis
Download Presentation

Randomized Smoothing Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Randomized Smoothing Networks Maurice Herlihy, Brown University Srikanta Tirthapura, Iowa State University

  2. Distributed Load Balancing Load Balancing Network Example: Routing Tasks to Processors on a Grid Randomized Smoothing Networks IPDPS 2004

  3. Smoothing Networks 2-smoothing Network In a k-smoothing Network, the numbers of Tokens on different output wires differ by at most k Randomized Smoothing Networks IPDPS 2004

  4. How to Build Smoothing Data Structures? • Centralized Solution • Random routing • Distributed and local • The smoothness can diverge without boundas more tokens enter the network • Balancing Networks Randomized Smoothing Networks IPDPS 2004

  5. Overview of Talk • Describe Smoothing Networks built out of balancers • Our Result: Randomized Smoothing Networks better than Deterministic Ones • Analysis of the Randomized Butterfly (or Block) network Randomized Smoothing Networks IPDPS 2004

  6. Basic Component: Balancer Randomized Smoothing Networks IPDPS 2004

  7. Balancer Randomized Smoothing Networks IPDPS 2004

  8. Balancer Randomized Smoothing Networks IPDPS 2004

  9. Smoothing Networks:Scalable Distributed Data Structures Bitonic[2] Bitonic[4] Randomized Smoothing Networks IPDPS 2004

  10. Prior Work: Deterministic Smoothing Networks • The Network Balancers are Initialized in a very specific way • (Usually) All balancers are pointing up depth = 3, width =4 Randomized Smoothing Networks IPDPS 2004

  11. Our Work: Randomized Smoothing Networks • Random balancers –initialized as follows • With prob = ½, up • With prob = ½, down • Resulting Initial Network State also random • How good are the smoothing properties of randomized networks? randomly initialized balancers Randomized Smoothing Networks IPDPS 2004

  12. Benefits of Randomized Networks • Easy Initialization • Each balancer sets itself to a random state • Completely Local Actions • Easy to recover from faults • Global reconfiguration unnecessary • Better Smoothing Properties Randomized Smoothing Networks IPDPS 2004

  13. Width = 2 Width = 4 Width = 8 Butterfly Network: Inductive Construction Randomized Smoothing Networks IPDPS 2004

  14. Our Main Result • Many Randomized Networks produce much smoother outputs than Regular Deterministically Initialized Networks • For a Butterfly network of width w: • The output of a randomized network has smoothness with high probability • For any deterministic initialization, worst case output smoothness is no better than O(log w) Randomized Smoothing Networks IPDPS 2004

  15. Implications • nearly constant for practical purposes • If w =10^9, this is 14 • Smoothness independent of the number of tokens entering the network • Network is very simple and easy to construct • No further constants hidden in O(…) Randomized Smoothing Networks IPDPS 2004

  16. Previous Work • Aspnes Herlihy and Shavit, 1991 • Bitonic and Periodic Counting networks (which are also 1-smoothing) • Isomorphic to sorting networks • Width w, depth • Klugerman and Plaxton, 1992 • Better depth constructions of counting networks • Width w, depth • Constants are very high, not practical Randomized Smoothing Networks IPDPS 2004

  17. Previous Work (2) • Aiello, Venkatesan and Yung, 1993 • Counting and Smoothing Networks using random and deterministic balancers • An alternate definition of a randomized balancer • Our results apply to their definition of random balancers also • In contrast, we use only randomized balancers • Herlihy and Tirthapura, 2003 • Worst case smoothness of deterministic Butterfly, Periodic and Bitonic networks • Matching upper and lower bounds • Self-stabilizing constructions Randomized Smoothing Networks IPDPS 2004

  18. Analysis: Random balancer a, b = numbers of tokens entering the balancer c, d = numbers of tokens exiting the balancer a c r = + ½ with probability ½ = - ½ with probability ½ c = (a+b)/2 + D(a+b) r d = (a+b)/2 – D(a+b) r b d D = odd characteristic function Randomized Smoothing Networks IPDPS 2004

  19. y1 depends only on the balancer decisions r1…r7 and on x1..x8 y1 x1 y2 x2 y3 x3 y1 = f(x1,x2,x3,..x8, r1,r2,………,r7) y4 x4 x5 y5 x6 y6 E[y1] = (x1+x2+…+x8)/8 x7 y7 y8 x8 Width = 8 Analysis (2): Butterfly[8] r1 r5 r7 r2 r3 r6 r4 Randomized Smoothing Networks IPDPS 2004

  20. Random variable y1 is hard to work with directly Define Alternate RandomVariable z1: z1 = (r1+r2+r3+r4)/4+ (r5+r6)/2 + r7 + (x1+…+x8)/8 y1 x1 r1 r5 r7 y2 x2 y3 r2 x3 y4 x4 x5 y5 r3 r6 x6 y6 x7 y7 r4 y8 x8 Analysis (3) Width = 8 Randomized Smoothing Networks IPDPS 2004

  21. Analysis (4) • Random Variable z1 is easier to handle, since it is a linear function of r1, r2, … r7 • Use Hoeffding’s inequality to bound the tail of z1, and hence y1 Randomized Smoothing Networks IPDPS 2004

  22. Theorem The output of Butterfly[w] with random balancers is smooth with probability at least (1-4/w) , irrespective of the input sequence Contrast: For any deterministic initialization, there exist inputs which lead to (log w) smooth outputs Randomized Smoothing Networks IPDPS 2004

  23. Conclusions • Simple network (butterfly) with small depth and very good smoothness • Easy, local reconfiguration after faults Randomized Smoothing Networks IPDPS 2004

  24. Open Questions • Do there exist simple, log-depth networks with constant output smoothness (with high probability)? • Can the upper bound on the butterfly network be improved? • Matching Lower bounds? • Smoothing networks for tokens of unequal weights? Randomized Smoothing Networks IPDPS 2004

  25. Randomized Smoothing Networks IPDPS 2004

More Related