Basic technology of distributed algorithms (1) Correctness of algorithms

1. Lecture 7: Synchronous Network Algorithms Basic technology of distributed algorithms (1) Correctness of algorithms (2) Efficiency of algorithms (3) Fault-tolerance

2. Complexity of distributed algorithms • Communication complexity: (1) the number of messages transmitted in whole system, (2) the number of message bits transmitted in whole system • Memory complexity: (1) the amount of memory used in whole system, (2) the amount of memory used in one process. • Computation complexity: (1) processing time in processes, (2) upper bound of transmission delay in communication links.

3. Problems 1 Leader Election in A synchronous Ring • Assumption of the network • Network graph G=(V,E) is a ring, where node set V={1,2,…,n} and edge set E={(i,i+1), where i=1,2,…,n and i iscounted mod n}. • Each process has a unique distinct identifier (UID). • Each node uses only unidirectional communication and its UID (it does not know the size of the ring, its own index and anything of its neighbors). • Only the leader performs an output. LCR algorithm (informal) (1) Each process sends its identifier around the ring. (2) When a process receives an incoming identifier, it compares that identifier to its own. If the incoming identifier is greater than its own, it keeps passing the identifier; if it is less than its own, it discards the incoming identifier; if it is equal to its own, the process declares itself the leader.

4. LCR algorithm (formal) • Suppose that each node i has • A UID u, initially i’s UID, • send, a UID or null, initially i’s UID, • A status with value in {unknown, leader}, initially unknown. Algorithm Send the current value of send to clockwise neighbor send := null if the incoming message is v, a UID, than case v>u: send :=v v=u: status := leader v<u: do nothing endcase Theorem LCR solves the leader-election problem in a synchronous ring. Proof: exact one process outputs the value leader.

5. Analysis of the Complexity of LCR Algorithm • Time complexity: n rounds until a leader is announced. • Communication complexity: Remark 1: If it is required that all the nodes halt, the time complexity is 2n and the communication complexity is still Remark 2: If it is required that the leader and nonleaders all provide output and all processes halt, the extra cost of obtaining the extra outputs and the halting is only n rounds and n messages.

6. Each process i operates in phases 0,1,2,…. In each phase l, process i send out “tokens” consists of UID in both directions. These are intended to travel distance then return to their original i. If both tokens make it back safely, process i continues with the following phase. However, the tokens might not make it back safely. While a token is proceeding in the outbound direction, each other process j on path compares with its own UID . If then j simply discards the token, whereas if , then j relays . If , then it means that process j has received its own UID before the token has turned around, so process j elects itself as the leader. All processes always relay all tokens in the inbound direction. Improved Leader-Election Algorithm In the following algorithm, the communication is supposed to be bidirectional. HS algorithm (informal)

7. i find leader relay • Analysis of the Complexity of HS Algorithm • Time complexity: O(n) rounds (the time for each phase is ). • Communication complexity: O(n log n) (at most processes altogether initiate tokens at phase l. Therefore, the total messages sent out at phase l is bounded by ).

8. FloodMax algorithm (informal) Problem2 Leader Election in a General Network • Assumption of the network • An strongly connected network digraph G=(V,E) having n nodes. • Processes do not know their indices, nor those of their neighbors, but refer to their neighbors by local names. • If a process i has the same process j for both incoming and outgoing neighbor, then i knows that the two processes are the same. • Suppose that each process has a unique distinct UID and it knows diam, the diameter of network. • Each process maintains a record of the maximum UID it has seen so far (initially its own). At each round, each process propagates its maximum on all of its outgoing edges. • After diam rounds, if the maximum value seen is the process’s own UID, the process elects itself the leader; otherwise, it is a non-leader.

9. Theorem FloodMax algorithm solves the leader-election problem in a synchronous general network. Proof: exact one process outputs the value leader. • Analysis of the Complexity of FloodMax Algorithm • Time complexity: (The time until the leader is elected and all other processes know that they are not the leader is diam rounds.) • Communication complexity: (The number of messages is , where |E| is the number of directed edges in the digraph, because a message is sent on every directed edge for each of the first diam rounds.)

10. s Bread-First Search in an undirected graph Problem 3 Breadth-First Search Bread-First Search Given a graph G=(V,E) and a distinguished source vertex i, breadth-first search explores the edges of G to discover every vertex that is reachable from s. It provides the distance from i to all such reachable vertices. It also provide a breadth-first tree with root i that contains all such reachable vertices. s Bread-First Search in a directed graph Spanning Tree A spanning tree of graph G=(V,E) is a rooted tree which contains all vertices of V and the path from the root to any vertex. • A breadth-firs tree is a spanning tree.

11. Assumption of the network • An strongly connected network digraph G=(V,E) having n nodes and a distinguished source note s. • Output is the structure of a breadth-first search tree of the network graph with root s in a distributed fashion: each process other than s should have a parent component that gets set to indicate the node that is its parent in the tree. • Processes know their neighbors’ indices. They have no knowledge of the size or diameter of the network. UIDs are not needed. • SynchBFS algorithms • At any point during execution, there is some set of processes that is “marked”, initially just s. • Process s sends out a search message at round 1, to all of its outgoing neighbors. • At any round, if an unmarked process receives a search message, it marks itself and chooses one of the processes from which the search has arrived as its parent, then it sends a search message to all of its outgoing neighbors.

12. Theorem SynchBFS algorithm solves the Breadth-First Search problem. • Analysis of the complexity of SynchBFS algorithm • Time complexity: At most diam rounds • Communication complexity: |E| Application to Message Broadcast Problem SynchBFS algorithm can be used to Message Broadcast problem: piggyback the message m in the Breadth-First Search.

13. s 2 1 4 1 3 s 2 1 1 1 4 1 1 3 1 3 1 1 1 1 3 Shortest paths from s Problem 4 Shortest Path Problem Shortest Path Problem Consider a strong directed graph G=(V,E), where ach edge e=(i,j) in E has a nonnegative real-value weight w(i,j). The problem is to find a shortest path form a distinguished node s to each other node in G.

14. BellmanFord algorithm • Each process i keeps track of parent and dist, the shortest distance from s to i it knows so far. Initially, dist(s)=0,dist(i)= for and the parent components are not defined. • At each round process i sends its dist(i) to all its outgoing neighbors. Then each process i updates its dist(i) to be • If dist(i) is updated, the parent is also updated accordingly. • After n-1 rounds, dist contains the shortest distance, and parent contains the parent in the shortest tree. • Assumption of network • Each process initially knows its neighbors’ indices and the weight of all its incident edges. • Each process knows the number n of nodes in the network. • Output: each process knows its parent in the shortest path tree, and its distance from s.

15. s s 2 1 s 2 1 s 2 1 2 1 0 0 0 4 3 1 4 3 4 3 1 1 4 3 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 1 1 3 1 3 3 s 2 1 s 2 1 0 4 3 0 1 4 3 1 1 1 1 1 1 1 1 3 1 3 • Analysis of the complexity of BellFord algorithm • Time complexity: n-1 • Communication complexity: (n-1)|E|

16. Assignment • Improve algorithm FloodMax to reduce the communication complexity. • Describe an algorithm that extends SynchBFS to allow the source process s to broadcast a message to all other processes and obtain an acknowledge that all processes have received it. Your algorithm should use O(|E|) messages and O(diam) time. You may assume that the network graph is undireted.