1 / 25

Applying Biology to Distributed Computing

Applying Biology to Distributed Computing. Teodoro Cipresso tcipress@hotmail.com Course: CS249 FALL 2006 Professor: Dr. Teng Moh. Based on the research papers: [ 1 ] O. Babaoglu et al., Design Patterns From Biology for Distributed Computing

jetta
Download Presentation

Applying Biology to Distributed Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applying Biology to Distributed Computing Teodoro Cipresso tcipress@hotmail.com Course: CS249 FALL 2006 Professor: Dr. Teng Moh Based on the research papers: [1] O. Babaoglu et al., Design Patterns From Biology for Distributed Computing [2] G. Canright et al., Chemotaxis-Inspired Load Balancing [3] Van Renesse, R. The importance of aggregation Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  2. Bio-Inspired Design Patterns • Patterns emerge through experience • Observe natures successful solutions to problems • Ecosystems are rich in well tested solutions • Cooperation and Competition • Fault-tolerance • Large scale and dynamic distributed systems • Have similar problems found in ecosystems: • Sustain unexpected events • Adapt behavior with limited information • Handle a massive population (network) Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  3. Plain Diffusion In Biology • Spontaneous spreading of matter • Particles move from area of high concentration to low • Areas of low concentration can observe a gradient • Eventually equal distribution is achieved High “Fick's law of diffusion” Low Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  4. Nodes are Idealized Portions of Space • Nodes are connected using the fluid transport • An overlay network may be constructed P P P P P P P Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  5. Material (information) Diffuses Using Passive Transport Liquid Passive Transport Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  6. Spanning Overlay Network Nodes Form a Connected Graph • Epidemic-like behavior touches all nodes • Nodes interact at random to converge to a state • An example of such state, is a global average Particles are continually moving from areas of high concentration to lower. P0 P10 P1 P2 P3 P11 P12 P13 P4 P5 P6 P14 P15 P16 P7 P8 P9 P17 P18 P19 Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  7. Mapping Diffusion to Distributed Computing • Plain Diffusion Pattern Identified in [1] • Applications to Distributed Computing: • Use Distributed Aggregation to solve for the: • Network size (COUNT) • Total free storage • Maximum load • Location and Intensity of hotspots (gradients) • Control, monitor and optimize the network Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  8. 1 .5 .5 .375 .375 .282 0 0 0 .5 .25 .25 .25 .375 .25 .188 .25 .282 0 0 0 0 .188 .19 .282 .259 .259 ≈.25 … .25 .236 .25 .259 .25 .248 Σ≈.25 ≈.25 ≈.25 .236 .236 .248 ≈.25 Applying Plain Diffusion Pattern : Network Size • Aggregate computation without central control • Want to estimate size θof an overlay network • Infuse a value  = 1 into the network at node Pi • Nodes converge to a global average Σ by combining estimates =1 Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  9. Applying Plain Diffusion Pattern : Network Size • Find network size estimate θ from  and Σ =1 [infused value] / [global average] ≈ [network size]  / Σ = θ 1 / .25 = 4 processors 1 0 0 0 ≈.25 Σ≈.25 ≈.25 ≈.25 ≈.25 Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  10. Plain Diffusion Uniform Algorithm* Initially neighborEstimate = 0, localEstimate = 0; 01: Upon receiving no message: // periodically initiate 02: wait(τ time units) // create cycles (T=cycle) 03: Pj = getNeighbor() // select a random neighbor 04: send <localEstimate> to Pj// push local estimate to neighbor 05: neighborEstimate = receive(Pj) // pull neighbor estimate (wait) 06: localEstimate = // converge toward average (localEstimate + neighborEstimate) / 2 07: Upon receiving <estimate> from neighbor Pi : // neighbor initiated 08: neighborEstimate = <estimate> // store neighbors estimate 09: send <localEstimate> to Pi// send local estimate to neighbor 10: localEstimate = // converge toward average (localEstimate + neighborEstimate) / 2 *This algorithim is a less abstract rewrite of figure 1 on page 37 [1] Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  11. Σ=1 Σ=0 Σ=.5 Σ=.5 Plain Diffusion Visualization <connect0> <estimate1> A A A A P0 P1 P0 P1 P P P P S S S S <connected1> <disconnect0> A A A A P0 Σ=1 P1 Σ=0 P0 Σ=.5 P1 Σ=.5 P P P P S S S S <disconnected1> <estimate0> A A A A P0 Σ=1 P1 Σ=0 P0 Σ=.5 P1 Σ=.5 P P P P S S S S A Active Thread P Passive Thread C Synchronization Thread Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  12. Diffusion JR Implementation Challenges • Processors execute asynchronously • Asynchronous message passing for synchronization • Synchronous message passing for estimate exchange • Active and Passive Threads both update local estimate • Inherent Problem of Deadlock • Used randomization to alleviate deadlock (yield) • Processors do random bounded wait if neighbor connected Theme for the performance analysis on next 3 slides… Connectedness of network influences convergence rate Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  13. 0 1 2 3 4 5 6 Mean of 3 trials at each cycle marker Convergence Performance I • Spanning tree overlay network with 7 processors Spanning tree has worst case performance, why? Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  14. 0 2 1 3 4 6 5 Mean of 3 trials at each cycle marker Convergence Performance II • Ring overlay network with 7 processors Ring converges faster than spanning tree, why? Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  15. 0 2 1 3 4 6 5 Convergence Performance III • Mesh (fully connected) network with 7 processors Mean of 3 trials at each cycle marker Mesh has best case convergence, why? Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  16. JR Demo JR Implementation of Network Size Count CONN = Connect to processor RCON = Result of connect attempt DISC = Disconnect from processor RDIS = Result of disconnect attempt CALL = Send synchronous message SEND = Send asynchronous message RECV = Got message from neighbor PUSH = Send estimate to neigbor PULL = Get estimate from neighbor UPDT = Update local estimate Plain Diffusion Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  17. + stimulus - Chemotaxis In Biology • Taxis is an innate behavioral response • Respond to stimulus coming from some direction (signal) • Positive and Negative Taxis Chemotaxis Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  18. Chemotaxis In Biology • Chemotaxis is taxis in response to chemical stimuli • Positive taxis, or move towards food (glucose) • Negative taxis, or move away from, a poison (formalin) • Increasing concentration of chemical (gradient is signal) - + Chemotaxis Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  19. P0 max load is 12 =48 P0 48 12 0 0 0 0 0 12 12 0 12 Mapping Chemotaxis to Distributed Computing • Chemotaxis composite pattern (uses Plain Diffusion) • Plain Diffusion can be used in primitive load balancing • Nodes send fractions of excess load to neighbors • Problem arises in creation of heavily loaded network areas • Movement of load not based on knowledge of hotspots • Chemotaxis inspired load balancing can help Chemotaxis Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  20. Lightly Loaded Area Heavily Loaded Area Applying Chemotaxis Pattern : Load Balancing • Propogate a signal (stimulus) from all nodes of the network • Load can move towards/away from signal (+/- chemotactic) • Gradients are created which indicate load on network areas • Signal is lightweight and moves quicker than load (fast diffusion) Chemotaxis Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  21. Applying Chemotaxis Pattern : Load Balancing • Load entering network will be forwarded to solve imbalance • Moves away from hotspots, toward lightly loaded areas Chemotaxis Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  22. Future Work Solve open problems in Distributed Computing through identification of additional patterns in Biology. Implement a simulation of Chemotaxis-Inspired Load Balancing (Signal-aided Diffusion) by extending Plain Diffusion JR implementation. Summary Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  23. Thank You! End Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  24. References I [1] O. Babaoglu et al., Design Patterns From Biology for Distributed Computing, ACM Transactions on Autonomous and Adaptive Systems (TAAS), Volume 1, Issue 1 (September 2006) at http://doi.acm.org/10.1145/1152934.1152937 [2] G. Canright et al., Chemotaxis-Inspired Load Balancing, TELENOR Research and Development, Fornebu, Norway at http://www.cs.unibo.it/bison/publications/chemo.ECCS05.pdf [3] Van Renesse, R. The importance of aggregation. Cornell University, Ithaca, NY. At http://www.cs.cornell.edu/home/rvr/papers/ImportanceAggregation.pdf [4] S.I. Rubinow., Introduction to Mathematical Biology, Graduate School of Medical Sciences, Cornell University, 1975 by Dover Publications, Inc. [5] A. W. Keen et al., The JR Programming Language concurrent programming in an Extended Java, 2004 by Kluwer Academic Publishers. [6] S. S. Epp, Discrete Mathematics with Applications, second edition, 1995 by PWS Publishing Company. [7] Diffusion, Wikipedia the free encyclopedia (retrieved October 2006), at http://en.wikipedia.org/w/index.php?title=Diffusion&printable=yes [8] Taxis, Wikipedia the free encyclopedia (retrieved October 2006), at http://en.wikipedia.org/w/index.php?title=Taxis&printable=yes Backup Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

  25. References II [9] Chemotaxis, Wikipedia the free encyclopedia (retrieved October 2006), at http://en.wikipedia.org/w/index.php?title=Chemotaxis&printable=yes [10] Humoral immunity, Wikipedia the free encyclopedia (retrieved October 2006), at http://en.wikipedia.org/w/index.php?title=Humoral_immunity&printable=yes [11] Overlay network, Wikipedia the free encyclopedia (retrieved October 2006), at http://en.wikipedia.org/w/index.php?title=Overlay_network&printable=yes [12] Virus, Encyclopedia Britannica, Encyclopedia Britannica Online (retrieved October 2006), at http://www.britannica.com/eb/article-32750 [13] Glossary of graph theory, Wikipedia the free encyclopedia (retrieved October 2006), at http://en.wikipedia.org/w/index.php?title=Glossary_of_graph_theory&printable=yes Backup Teodoro Cipresso, tcipress@hotmail.com, CS249, Fall 2006

More Related