CS 775/875: Fall 2002

1. CS 775/875: Fall 2002 Chapter 11: Coordination and Agreement

2. Failure Assumptions • Each pair of processes is connected by reliable channels---eventually deliver a message • Network partition (and a later repair) is a possibility • Process failures are fail-stop • A process that crashes never recovers---remains crashed forever • Reliable failure detector (failed/unsuspected) vs. unreliable failure detector(suspected/unsuspected)

3. Distributed Mutual Exclusion • Mutual exclusion solution solely based on message passing • Ex: Three entrances to a car garage and two exits; they need to know whether or not the parking lot is full or how many are currently available • Safety, liveness, ordering properties of mutual exclusion (ME) • Evaluation criteria: bandwidth consumed, client delay, throughput • Central server algorithm • Ring-based algorithm • Ricart and Agrawala’s algorithm • Maekawa’s algorithm • http://www.cdk3.net/ig/slides/Chapter11Slides.pdf

4. Evaluation metrics • Bandwidth consumed • Client delay • Synchronization delay • Fault tolerance

5. Elections • Why election? • Calling election, participant, non-participant, identifier • Safety and liveness properties • Ring-based election algorithm (Chang and Roberts): 3N-1 messages; no failure tolerance • The bully algorithm: Synchronous system, knows all process Ids: Best N-2, worst O(N2) • http://www.cdk3.net/ig/slides/Chapter11Slides.pdf

6. Multicast Communication • Process issues one multicast operation---message is delivered to a group of processes • Efficiency and delivery guarantees • Correct process: It is a process that exhibits no failures at any point in the execution under consideration. I.e., the whole time of execution, not just a part of it. • multicast(g,m) and deliver(m) • open and closed groups • Basic multicast: B-multicast (send m to all processes), B-deliver (deliver m to a process when received): A correct will eventually deliver a message as long as the multicaster does not crash. • Reliable multicast: R-multicast, R-deliver, Integrity (at most once delivery and was the one sent), validity (eventual self delivery), agreement (if one does, all will do so): If a process that multicasts a message crashes before it has delivered it, then it is possible that the message will not be delivered to any process in the group; but if it is delivered to some correct process, then all other correct processes will deliver it. • http://www.cdk3.net/ig/slides/Chapter11Slides.pdf

7. B-multicast vs. R-multicast • In B-multicast, even if a multicaster crashes half-way during the multicast to the group (say it has sent the message to 7 out of 10 members), the 7 members will B-deliver the message to their own processes. • In R-multicast, however, each member that receives the message first B-multicasts the message to other members before delivering it to its process. This way, if the original multicaster crashed, then subsequently the message is repeated by other members who received it.

8. Multicast Communication (Contd.) • Implementing R-multicast over B-multicast: Figure 11.10 • R-multicast using IP-multicast + Piggybacked ACK+ negative ACK; Hold-back queue (closed groups) • Uniform agreement: If a process, whether it is correct or fails, delivers a message m, then all correct processes in group(m) will deliver m.

9. Multicast (cont.) • Ordered Multicast: FIFO ordering (partial), causal ordering (partial), total ordering • causal ordering implies FIFO ordering • http://www.cdk3.net/ig/slides/Chapter11Slides.pdf • FIFO: Use of sequence numbers • Total ordering: Group-unique sequence numbers, sequencer: (I) Sequencer assigns a number as well as multicasts to members • Sequence number through consensus • Causal ordering: Vector timestamps • Reliable and totally ordered delivery: impossible in an asynchronous system

10. Consensus and Related problems • System model: Set on N processes; reliable communication; processes may fail; Byzantine failures as well as crash failures • Consensus problem: Processes should agree on a value after one or more of the processes has proposed what the value should be • Byzantine failure (Page 56): Process/channel exhibits arbitrary behavior; it may send/transmit arbitrary messages at arbitrary times, commit omissions; a process may stop or take an incorrect step. • Requirements of a consensus algorithm: termination, agreement, integrity • Byzantine Generals problem: “There are three or more generals. One is a commander and the rest or lieutenants. Commander issues an order. The rest have to decide whether the commander’s order was attack or retreat and take action. One or more of the generals (including the commanders) may be traitors or faulty. The problem is for the loyal-generals to reach an agreement on an action.” termination, agreement, and integrity.

11. Consensus and Related problems(Cont.) • Interactive consistency: “Every process proposes a single value. Goal is for all correct processes to agree on a vector of values, one for each process.” termination, agreement, integrity. • Types: Arbitrary or crash failures; synchronous or asynchronous systems

12. Consensus in Synchronous Systems • See Figure 11.18http://www.cdk3.net/ig/slides/Chapter11Slides.pdf • Assumes that up to f of N processes exhibit crash failures. • Example: P1,P2,P3,P4 are processes: P1 proposes 4, P2 proposes 3, P2 proposes 2, and P4 1. Let f=1. In round 1, P1-P4 multicast their values, but P4 crashes after sending to P1. So the end of round 1, P1 has {1,2,3,4}, but P2-P3 have {2,3,4}. If we stopped here, then P1 concludes value as 1, where as P2-P3 conclude it as 2. This is incorrect. However, in round 2 (f+1 round), P1 muticasts {1,2,3} while P2 multicasts {3} and P3 multicasts {2}. End of round 2 P1-P3 will have {1,2,3,4}. So they all come to the same consensus. • Where is “Synchronous system” property used?

13. Byzantine Generals problem in Synchronous Systems • Assume arbitrary failures (not just crash) • Up to f of the N processes may be faulty: N>3*f; f+1 rounds are needed • Correct processes may detect the absence of a message via timeout. • Channels are reliable • Solution with 4 generals, only 1 of them faulty: (Figure 11.20) • http://www.cdk3.net/ig/slides/Chapter11Slides.pdf

14. Impossibility in Asynchronous Systems • In synchronous systems, we can use timeouts; but not in asynchronous systems • Fischer et al proved that no algorithm exists to guarantee solution to the Byzantine/consensus problems in asynchronous systems • Solutions? Masking faults, failure detectors, etc.