Polystyrene: Survivable Shape for Self-Organising Data

1. SocioPlug Polystyrene: Survivable Shape for Self-Organising Data François Taïani Joint work with: Hoel Kervadec (INSA Rennes)Simon Bouget (ENS Rennes)Anne Marie Kermarrec (ASAP)

2. Focus • Epidemic Topology Construction algorithms • Decentralized, fast, scalable • Fundamental building block to higher-level services(DHT, Multicast, Pub-Sub, Recommendations) Taken from [JMB09] F. Taiani

3. Problem: Catastrophic Failure • The topology heals • But the overall shape is lost ? How to recreate whole shapefrom surviving nodes?

4. Outline • Background: Decentralized Topology Construction • Polystyrene: Architecture and Protocol • Evaluation • Outlook F. Taiani

5. Decentralized Topology Const. • Each node : some data • Find k “closest” nodes in system • Decentralized approach, asynchronous rounds C E topology layer gossip-based topology construction (e.g. T-Man) B D A C E random sampling (RPS) B D A random link topology link node node position

6. Decentralized Topology Const. • Main idea: greedyneighbourhood optimization F C F F C C A A B B E E D D E D neighborhoodoptimization exchange ofneighbors lists 1 2

7. Polystyrene’s Architecture Polystyrene Neighbours Node position Topology Construction(T-Man, Vicinity, Gossple) Peer Sampling Service(RPS, Cyclon, SCAMP)

8. Polystyrene Protocol recovery 2 ghosts guests 1 FD 1’ backup (outgoing) backup (incoming) migration 4 3 Topology Construction projection Neighbours 3’ Node position

9. The Migration Process p q p.guestst p.post b a c e d q.post f q.guestst F. Taiani

10. The Migration Process • Bi-clustering of guest points • Heuristics : diameter p q b a c e d f F. Taiani

11. The Migration Process • Bi-clustering of guest points • Heuristics : diameter p q points closer to b points closer to d b a c e d f F. Taiani

12. The Migration Process • Bi-clustering of guest points • Heuristics : diameter + minimum move p q points closer to b points closer to d b p.post+1 a q.post+1 c p.guestst+1 e q.guestst+1 d f F. Taiani

13. Evaluation • Shape : 2D 40x80 logical torus • Round 20 : 50% correlated node crashes After failure (r=20) (r=22) (r=28) Polystyrene recreates shapewith surviving nodes F. Taiani

14. Eval: Quality of Neigborhoods Polystyrene maintains good neighborhoods

15. Eval: Quality of Shape And the torus gets restored!

16. Eval: Scalability • Time (rounds) until homeogeneity less than Logarithmic convergence! F. Taiani

17. Outlook • An example of advanced topology construction • Replicated, highly robust, self-organising • Potential extension to load-balancing • Good for plug heterogeneity • Concrete application on top of polystyrene • DHT, recommendation, queries, search • Larger picture • Self-organising data primitives for plug infrastructures

18. References • [KMG03] Kermarrec A.-M., Massoulie L., Ganesh, A.J., Reliable Probabilistic Communication in Large-Scale Information Dissemination Systems, IEEE Transactions on Parallel and Distributed Systems, March 2003, (14:3) • [JGK04] Jelasity, M., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. (2004). The peer sampling service: experimental evaluation of unstructured gossip-based implementations. Middleware ’04, pages 79–98, New York, NY, USA. Springer- Verlag New York, Inc. • [VS05] Voulgaris, S. & Steen, M. V. Epidemic-style Management of Semantic Overlays for Content-Based Searching. Proc. of the 11th Int. Euro-Par Conf. on Parallel Processing (Euro-Par'05), Springer, 2005, 1143-1152 • [Jelasity, Alberto Montresor, and OzalpBabaoglu. 2009. T-Man: Gossip-based fast overlay topology construction.Comput. Netw. 53, 13 (August 2009), 2321-2339. • [BFG+10] Bertier, M.; Frey, D.; Guerraoui, R.; Kermarrec, A.-M. & Leroy, V.The GOSSPLE anonymous social network. Proc. of the ACM/IFIP/USENIX 11th Int. Conf. on Middleware, 2010, 191-211 • [TLB14] Taiani, F., Lin, S. and Blair, G. S. (2014) GossipKit: A Unified Component Framework for Gossip. IEEE TSE, Preprint, doi: 10.1109/TSE.2013.50