Group Communications

1. Group Communications Group communication: one source process sending a message to a group of processes: Destination is a group rather than a single process. — Broadcast – destination is everybody. — Multicast – destination is a designated group. — Unicast – destination is a single process. Useful: for many applications, the recipient of a message is a group of related process. Many services in DS are implemented by a group of processes, possibly distributed in multiple machines. Examples of use: (a) Fault tolerance based on replicated servers: The service is implemented by a group of (replicated) server processes. A client process multicast(Send-to-group) request message to the server group. Each group member performs identical operation on a request. (b) Locating objects in distributed service: For example, a distributed file service is implemented by a group of servers. The files are not replicated. A client looking for a particular file multicast the request message to the server group. The member holding the wanted file responds. (c) Updating replicated data: Data may be replicated to improve performance (response time) and reliability. The data is replicated on multiple servers which forms a server group. Update requests are multicasts to the group.

2. (d) Multiple notification: For example, a group of processes need be notified of certain event. When the event occurs, a message can be multicasted to the group. A specific example is in the multicasting of flight information to a group of display processes in an airport terminal system. A Group is a collection of processes: G = {P1, P2, …, Pn} Group management functions: Forming a group: Create-group (out gid: group-id) returns a group identifier Joining a Group: Join-group (in gid: group-id) makes the caller process a member of the group gid. Leaving a group: Leave-group (in gid: group-id) remove the caller process from the group. Delete a group: Delete-group (in gid: group-id) Only authorized process (such as the creator) can delete a group. - Dynamic group vs static group Closed Group: Processes outside the group cannot multicast to the group, although an outsider process can send to individual member of the group.

3. Implementations (of group): Centralised: Use a group management server. •All group membership function requests are handled by the server. • Server keeps database of all active groups (membership) in the system. • Pro: implementation ease. Con: server may be reliability and performance bottleneck. Distributed: Each group member keeps track of membership. membership function requests are multicast to all members. • Pro: reliable • Cons: complexity and overheads Group Communication Functions Send to group: SendToGroup (in gid: group-id, msg: message) Receive from group: ReceiveFromGroup (in gid; group-id, out msg: message) Receive any message sent to group gid. Group Send and Receive have similar issues as IPC with respect to: Blocking and non-blocking

4. Implementations (of communication functions): Unicast: A message is sent to each of the members of the group. - Number of message = group size - Inefficient If underlying communication system supports broadcast to machines (e.g. Ethernet). A broadcast message is sent to all machines. The group communication facility within the kernel on respective machine takes in the message if there are members (of the group) on that machine. If underlying communication system supports multicast to machines. Then the message can be multicasted to only those machines on which there are members. Group Atomicity Group atomicity is concerned with whether the group is always a whole or can be partitioned (with respect to group operations). Atomic multicast: When a message is sent to the group, all members of the group receive the message. Otherwise, no member receives the message. i.e., there is never the case that some members receive the message while others do not. It is all (member) or none. Example in which atomicity is needed: - Replicated data update. If not, the replicated data may get out of steps with each other. Example in which atomicity is not needed: - Locating objects in distributed service. It is sufficient that the server holding the object receives the message. If the message to this server is lost, the client can try again (say on time-out). - Multiple notification in a flight information display system. Implementation (of group atomicity)

5. No machine failure: - Message (to individual member) can be lost. - Can use a positive acknowledgment protocol (i.e., time- out retransmission). - If network failure (which may make some members unreachable) is possible, a protocol similar to the following case can be used. Machine failure possible: - For example, the sending machine (or the machine on which the group communication server runs) crashes. Some member will have received the message while others have not. - More complicated protocol. e.g., a protocol similar to 2- phase commitment protocol. - A simple (inefficient) protocol: Sender sends to each member, using positive acknowledgment protocol. When a receiver receives a message. It determines if the message has been received before. If not it sends the message to all other members of the group. Message Ordering The Order of message received by a group member can be important - FIFO order If a source process sends two messages to a group in the order of M1->M2 (i.e., M1 precedes M2), then the messages are received in that order. FIFO is easy to implement.

6. Not adequate for some application. e.g., two client processes P1 and P2 are multicasting update requests to a server group whose member are holding replicated data. Suppose P1 sends M1, then M2 ; and at about the same time P2 sends N1 then N2. Further suppose the group consists of 2 server processes S1 and S2. The message can arrive at the following order and still satisfy FIFO order: at S1: M1->N1->M2->N2 at S2: N1->M1->M2->N2 The above order may lead to inconsistent result. - Consistent order The relative order of messages received at each member process is the same. That is, the order is consistent across the group. - Causal order Assume S1 first multicasts m, upon reception of m, S3 multicasts m’, then m->m’. m and m’ are said to be causally ordered. Causal order could be violated. e.g., due to network delay. A group communication mechanism is said t observe casual order if it preserves the order.

7. Total order When a group communication mechanism satisfies both causal order and consistent order, we say the mechanism satisfies total order. That is, The message order at each member process preserves causal order and that the message ordering is the same across the entire group. Implementation: Use a logical clock global to the distributed system to timestamp messages (e.g., Lamport’s timestamps – to be discussed in a later chapter). The message are ordered by the entire group. Use a centralised server. A multicast message is first sent to the server. The server assigns sequence number to the messages (the server is also called a sequencer) and send them to all the members. The messages are delivered (to the application program) in order of the sequence number. Use a token passing (logical) ring. Only the token holder can multicast message. Note that atomicity and ordering are two different aspects of group communications. One may have total ordering but not atomicity and vise versa. Atomic totally ordered group communication is usually expensive in terms of implementation and overheads.

8. - System preserving causal order is also said to be a virtually synchronous system. An Implementation (of causal order) — Each process keeps a vector clock (V1, V2, …, Vn). — Each message carries a vector (logical) timestamp: (V1, V2, …, Vn). Where Vk increments each time process Pk multicast a message. — When message with timestamp (V1, V2, …, Vn) from Pj is received. The message is accepted if Vj = Lj + 1, and Vi <= Li Where (L1, L2, …, Ln) is the local vector clock. And the local clock is set to (V1, V2, …, Vn). Otherwise, the message is held back until the condition is satisfied.

9. Design Issues to be Considered 1. Design your client-user interface 1. Group Information Inquiring 2. Joint Group 3. 4. ……. 2. Use TCP or UDP ( I choose TCP) 3. Functional Specification 3.1 Membership related service a) create a group b) Joint group c) Leave group d) remove group ……. 3.2. Multicast related services a) send message to a group b) message ordering (consistant order) 3.3 Miscellanceous Services a) group information inquiring 4. Architectural Relationship between Clients and Server(s) server client1 client 2 … clientn

10. Design Issues to be Considered 4. Architectural Relationships among processes and threads each client owns a process and the client process may contain several threads a thread sending group related command a thread sending message related command. the server owns a process that contains several threads each primitive corresponds to a tread, e.g., join group create a group …… send a message to a group Note that, you can use other ways to organize threads. Whatever you do, give reasons in the report. 5. Client-to-Server Protocol If many functions are on server side. How does the clients call them? 1#1gname --create a group with the name gname 1#gid -- join the group with group in “gid” 6#gid#meg -- send message “msg” to the group “gid” 6. Atomicity and fault tolerance, etc will be considered in semester B. However, you have to leave some “space” for that.