EEC 693/793 Special Topics in Electrical Engineering Secure and Dependable Computing

1. EEC 693/793Special Topics in Electrical EngineeringSecure and Dependable Computing Lecture 13 Wenbing Zhao Department of Electrical and Computer Engineering Cleveland State University wenbing@ieee.org

2. Outline • Reminder: • Midterm#2 April 7, Monday • Event ordering • Group communication systems • Ordered multicast • Techniques to implement ordered multicast • Membership protocols • Reference: • Reliable distributed systems, by K. P. Birman, Springer; Chapter 14-16 EEC693: Secure & Dependable Computing

3. Event Ordering • “Time, Clocks, and the Ordering of Events in a Distributed System”, by Leslie Lamport, Communications of the ACM, July 1978, Volume 21, Number 7, pp.558-565 • What usually matters is not that all processes agree on exactly what time it is, but rather, that they agree on the order in which events occur EEC693: Secure & Dependable Computing

4. Happens-Before Relation • Assumptions: • The system is composed of a collection of processes, each process consists of a sequence of events • The events of a process form a sequence, where a occurs before b in this sequence if a happens before b • The sending or receiving of a message is an event in a process EEC693: Secure & Dependable Computing

5. Happens-Before Relation • The happens-before relation “→” on the set of events of a system is the relation satisfying the following three conditions: • If a and b are events in the same process, and a comes before b, then a →b • If a is the sending of a message by one process and b is the receipt of the same message by another process, then a →b • If a →b and b →c, then a →c EEC693: Secure & Dependable Computing

6. Partial Ordering • Not all events have the happens-before relationship • Two distinct events a and b are said to be concurrent if a →b and b →a • Neither event can causally affect the other • This introduces a partial ordering of events in a system with concurrently operating processes • “a happens before b” means that information can flow from a to b • “a is concurrent with b” means that there is no information flow between a and b EEC693: Secure & Dependable Computing

7. How to Capture the Partial Ordering? • Use logical clocks to capture the partial ordering • Define a clock Ci for each process Pi. Assign a number Ci(a) to any event a in that process • The entire system of clocks is represented by the function C which assigns to any event b the number C(b), where C(b) =Cj(b) if b is an event in process Pj • The clocks Ci are logical clocks rather than physical clocks EEC693: Secure & Dependable Computing

8. Lamport Clock • A Lamport logical clock is a monotonically increasing software counter • Each process Pi keeps its own logical clock Ci to apply Lamport timestamps to events • To capture the happens-before relation→, processes must do the following: • Before each event at Pi: Ci := Ci+1 • When Pi sends a message m, it piggybacks t = Ci • When Pj receives (m,t): Cj := max(Cj,t) + 1 e→e’ C(e) < C(e’) EEC693: Secure & Dependable Computing

9. Lamport Clock: An Example EEC693: Secure & Dependable Computing

10. Group Communication System • Services provided by the GCS • Membership service: who is up and who is down • Deals with failure detection and more • Reliable, ordered, multicast service • FIFO, causal, total • Virtual synchrony service • Virtual synchrony synchronizes membership change with multicasts • GCS is often used to build fault tolerant systems EEC693: Secure & Dependable Computing

11. Reliable Multicast • Reliable multicast – the message is targeted to multiple receivers, and all receivers receive the message reliably • Positive or negative acknowledgement • Need to avoid ack/nack implosion • Distinguish receiving from delivery! Application Delivering Middleware Receiving EEC693: Secure & Dependable Computing

12. Ordered Reliable Multicast • Ordered reliable multicast – if many messages are multicast by many senders, in what order the messages are delivered at the receivers? • First in first out (FIFO) • Causal – the causal relationship among msgs preserved • Total – all msgs are delivered at all receivers in the same order EEC693: Secure & Dependable Computing

13. FIFO Ordered Multicast • FIFOor sender orderedmulticast: Messages are delivered in the order they were sent (by any single sender) a e p q r s b c d delivery of c to p is delayed until after b is delivered EEC693: Secure & Dependable Computing

14. Causally Ordered Multicast • Causalor happens-beforeordering: If send(a)  send(b) then deliver(a) occurs before deliver(b) at common destinations a p q r s b EEC693: Secure & Dependable Computing

15. Causally Ordered Multicast • Causalor happens-beforeordering: If send(a)  send(b) then deliver(a) occurs before deliver(b) at common destinations a p q r s b c delivery of c to p is delayed until after b is delivered EEC693: Secure & Dependable Computing

16. Causally Ordered Multicast • Causalor happens-beforeordering: If send(a)  send(b) then deliver(a) occurs before deliver(b) at common destinations a e p q r s b c delivery of c to p is delayed until after b is delivered e is sent (causally) after b EEC693: Secure & Dependable Computing

17. Causally Ordered Multicast • Causalor happens-beforeordering: If send(a)  send(b) then deliver(a) occurs before deliver(b) at common destinations a e p q r s b c d delivery of c to p is delayed until after b is delivered delivery of e to r is delayed until after b&c are delivered EEC693: Secure & Dependable Computing

18. Totally Ordered Multicast • Totalordering: Messages are delivered in same order to all recipients (including the sender) a e p q r s b d c all deliver a, b, c, d, then e EEC693: Secure & Dependable Computing

19. Implementing Total Ordering • Use a token that moves around • Token has a sequence number • When you hold the token you can send the next burst of multicasts • Use a sequencer to order all multicast • Message is first multicast to all, including the sequencer; then the sequencer determines the order for the message and informs all • Or send to the sequencer and the sequencer multicast with total order information • Each sender can take turn to serve as the sequencer EEC693: Secure & Dependable Computing

20. Group membership service • Input: • Process “join” events • Process “leave” events • Apparent failures • Output: • Membership views for group(s) to which those processes belong EEC693: Secure & Dependable Computing

21. Issues? • The service itself needs to be fault-tolerant • Otherwise our entire system could be crippled by a single failure! • Hence Group Membership Service (GMS) must run some form of protocol (GMP) EEC693: Secure & Dependable Computing

22. Approach • We’ll assume that GMS has members {p,q,r} at time t • Designate the “oldest” of these as the protocol “leader” • To initiate a change in GMS membership, leader will run the GMP • Others can’t run the GMP; they report events to the leader EEC693: Secure & Dependable Computing

23. GMP Example • Example: • Initially, GMS consists of {p,q,r} • Then q is believed to have crashed p q r EEC693: Secure & Dependable Computing

24. Unreliable Failure Detection • Recall that failures are hard to distinguish from network delay • So we accept risk of mistake • If p is running a protocol to exclude q because “q has failed”, all processes that hear from p will cut channels to q • Avoids “messages from the dead” • q must rejoin to participate in GMS again EEC693: Secure & Dependable Computing

25. Basic GMP • Someone reports that “q has failed” • Leader (process p) runs a 2-phase commit protocol • Announces a “proposed new GMS view” • Excludes q, or might add some members who are joining, or could do both at once • Waits until a majority of members of current view have voted “ok” • Then commits the change EEC693: Secure & Dependable Computing

26. GMP Example • Proposes new view: {p,r} [-q] • Needs majority consent: p itself, plus one more (“current” view had 3 members) • Can add members at the same time Proposed V1 = {p,r} Commit V1 p q r OK V0 = {p,q,r} V1 = {p,r} EEC693: Secure & Dependable Computing

27. Special Concerns? • What if someone doesn’t respond? • P can tolerate failures of a minority of members of the current view • New first-round “overlaps” its commit: • “Commit that q has left. Propose add s and drop r” • P must wait if it can’t contact a majority • Avoids risk of partitioning EEC693: Secure & Dependable Computing

28. What If Leader Fails? • Here we do a 3-phase protocol • New leader identifies itself based on age ranking (oldest surviving process) • It runs an inquiry phase • “The adored leader has died. Did he say anything to you before passing away?” • Note that this causes participants to cut connections to the adored previous leader • Then run normal 2-phase protocol but “terminate” any interrupted view changes leader had initiated EEC693: Secure & Dependable Computing

29. GMP Example p • New leader first sends an inquiry • Then proposes new view: {r,s} [-p] • Needs majority consent: q itself, plus one more (“current” view had 3 members) • Again, can add members at the same time Inquire [-p] Proposed V1 = {r,s} Commit V1 q r OK: nothing was pending OK V0 = {p,q,r} V1 = {r,s} EEC693: Secure & Dependable Computing