Exploiting Virtual Synchrony in Distributed Systems

1. Exploiting Virtual Synchrony in Distributed Systems Kenneth P. Birman and Thomas A. Joseph Presenter : Steve Ko

2. Outline • Overview • Motivation • Challenges • Previous Approach • Virtual Synchrony • Isis2 toolkit APIs • Conclusion

3. Overview • The goal is to introduce the “isis2” toolkit for distributed programming. • The “isis2” toolkit hides the complexity of distributed programming by providing a set of APIs. • The “isis2” toolkit provides a virtually synchronous environment to developers

4. Motivation • Distributed programming is hard • Group communication • Synchronization • Consistency • Failure detection • Recovery • Etc…

5. Challenges • Message passing is the only way to learn of the behavior of other processes. • But information propagation is asynchronous. Processes might see different orders. • A failure cannot be distinguished from a transient communication failure or an overload.

6. Challenges (cont.) Process B Process C Shared X Shared X Shared X Process A

7. Previous Approach • Atomic multicast protocols can be used. • Either all receive a message or none does so in case of a sender failure. • All recipients see the same order for events such as deliveries, failures, recoveries, and etc. • But this is expensive and sometimes too strong.

8. Virtual Synchrony • It preserves the illusion that the events happen in the same order. • However, a weaker ordering guarantee can be applied wherever possible. • It is cheaper than full atomic multicasts.

9. Virtual Synchrony (cont.) Process B Process C Replicated FIFO Queues Process A • This scenario requires full atomic multicasts.

10. Virtual Synchrony (cont.) Process B Process C Replicated FIFO Queues Process A • This scenario requires a weaker guarantee.

11. Virtual Synchrony (cont.) Process B Process C Replicated FIFO Queues With a lock Process A • This scenario requires a combination of two.

12. Virtual Synchrony (cont.) • Different communication primitives provide different ordering guarantees. • ABCAST : Everyone sees the same order. • CBCAST : Only potentially causally related events appear in the same order. • GBCAST : ABCAST + CBCAST

13. “Potentially Causally Related” • Definition • A broadcast A is potentially causally related to a broadcast B only if they were sent by the same process and B occurred after A, or if A is delivered at SENDER(B) before B was sent (or there’s a chain of such order).

14. Isis2 toolkit APIs • Group RPC • mcast(…,N,…) : multicast, collect N replies • reply(…) : reply to a multicast • reply_cc(…) : reply with copies • State transfer • join_and_xfer(…) : join, and accepts state

15. APIs (cont.) • Process groups • pg_create(…) : create process group • pg_lookup(…) : lookup group address • pg_addmember(…) : add member • pg_leave(…) : leave group • pg_join(…) : request to be added • pg_kill(…) : send UNIX signal • pg_monitor(…) : monitors membership • pg_msg_verify(…) : validates messages • pg_join_verify(…) : validates joins

16. APIs (cont.) • Coordinator-cohort • coord_cohor(…) : one leader and fault-tolerance • Replicated data • update(…) : update replicated data • read(…) : read-only access • Synchronization • P(…) : obtain mutual exclusion • V(…) : release mutual exclusion

17. APIs (cont.) • Configuration • conf_update(…) : update configuration • conf_read(…) : read configuration • News • subscribe(…) : register with service • post_news(…) : post a news message

18. Conclusion • Virtual synchrony provides application-dependant ordering guarantees. • Isis2 toolkit (and virtual synchrony) provides an easy way to develop distributed applications (e.g. NYSE infrastructure).