Peer-to-Peer Membership Management for Gossip-Based Protocols

1. Peer-to-Peer Membership Management for Gossip-Based Protocols Ayalvadi J. Ganesh, Anne-Marie Kermarrec, and Laurent Massoulie Presented by Thadpong (Ted) Pongthawornkamol Some slides are borrowed from the author’s presentation

2. Outline • Motivation • SCAMP • Basic operations • Rebalancing techniques • Evaluations • Conclusion

3. Motivation • Reliable group communication is an important building block for any distributed systems • Current multicast algorithms can be divided into two categories • Network-level reliable multicast protocols • Built directly in network-level devices (i.e. hosts, routers) • Example : SRM, RMTP • Still not currently widely deployed • Application-level multicast protocols • Built on top of overlay networks • Can be deterministic or probabilistic (gossip-based) • Deterministic approaches suffer from scalability problem under failures

4. Motivation (Cont.) • Gossip-based dissemination protocols • Each member forwards the message to randomly chosen group members • Probabilistic guarantee ( guarantee message delivery with high probability) • Scalable • Resilient to node/link failures • However, traditional gossip-based multicast protocols rely on nonscalable membership protocol • Each node has the complete view of the system • High overhead in storage and synchronization

5. SCAMP • A scalable probabilistic membership protocol for gossip-based multicast • Scalable, fully decentralized • Each node maintains partial, yet sufficient system view • Self-reconfigurable • View size in each member can change when system size changes • Any isolated node can rejoin the system automatically with isolation recovery mechanism • Complementary with gossip-based multicast

6. SCAMP Operations • Basic membership management • Subscription • Unsubscription • Recovery Isolation • Graph rebalancing mechanisms • Indirection • Lease Mechanism

7. Membership List • Each node k maintains two lists of group members • PartialView : a list containing its gossip targets • InView : a list of nodes which k is one of their gossip targets

8. Subscription • A new node sends a subscription request to an arbitrary member called contact member • New node’s view contains only its contact node • When the contact node receives the request from a subscribing node • Forward new node’s ID to all members of the contact’s local view • Forward c additional requests to randomly chosen nodes in its view (c is a failure-tolerated parameter) • When a node k receives a forwarded subscription from the contact node • With probability p = 1/(1+size of PartialViewk), add the subscriber into its PartialView • Otherwise, forward the subscription to a randomly chosen node from k’s PartialView • When a node i decides to add node jinto i’s PartialView • Node i sends a message to node j , telling node j to add node i into j’s InView

9. Subscription (c=0) Let E[Mn] = expected #arcs (links) of system with n nodes E[Mn]= E[Mn-1]+E[Mn-1]/(n-1)+c+1 ≈ (c+1)nlogn 7 6 Local view 1 4 5 6 4 0 6 0 6 7 2 6 6 1 2 6 0 8 3 4 7 0 1 5 6 6 6 5 8 7

10. Unsubscription • When a node n wants to unsubscribe itself from the system • Let PartialViewn = { i(1), i(2), … , i(l) } • Let InViewn = { j(1), j(2), … ,j(l’) } • Node n informs nodes j(1), j(2), … , j(l’-c-1) to replace its id with i(1),i(2), … , i(l’-c-1) respectively • Node n informs nodes j(l’-c), … , j(l’) to remove its id from their list

11. Unsubscription (c=0) Let E[Mn-1] = expected #arcs (links) of system with n-1 nodes E[Mn]= E[Mn] - E[Mn]/n - (c+1) ≈ (c+1)(n-1)(logn – (1/n-1)) ≈ (c+1)(n-1)log(n-1) In view 2 6 8 7 6 Local view 1 4 5 4 0 s(0->1) 1 6 0 6 7 2 s(0->4) 1 2 rm(0) 6 0 4 x 8 3 4 7 0 1 5 6 5 8 7

12. Isolation Recovery • Use Heartbeat mechanism to detect failures • Each node periodically sends heartbeat messages to every node in its PartialView • Resubscribe if the node has not received the heartbeat messages for a long time

13. SCAMP Operations • Basic membership management • Subscription • Unsubscription • Recovery Isolation • Graph rebalancing mechanisms • Indirection • Lease Mechanism

14. Indirection • In practice, some nodes might be used as contact nodes more often than the others • The average list lengths grow faster than expected • The lists of the contact nodes grow quickly • Balance the lists by Indirection mechanisms • The node which receives the subscription request forwards the “token” with a counter value • Decrement the counter every hop forwarded • The member where token with zero counter arrives acts as a contact node

15. Lease Mechanism • Each subscription has a prespecified period, called lease • Every node removes the expired member from its PartialView • Each node is responsible to resubscribe (without c additional copies) to a random chosen member from its PartialView • Subscriber’s PartialView remains the same • However, each node’s PartialView gradually changes (even there is no change to the system)

16. Full view VS Partial view

17. Impact of C

18. Impact of C (Cont.)

19. Impact of Indirection

20. Impact of Lease mechanism

21. Conclusion • SCAMP • Membership management system for gossip-based multicast • Partial View (O(log n)) per member in average • Scalable • No global system size needed • Self-reconfigurating • Used with O(log n) gossip-based multicast • Can be used in other scheme? (Can we change the paremeter?) • Achieve load balancing by using several techniques • Indirection • Distribution of contact work • Lease mechanism • Good to often change the view?