Model and complexity

1. Many measures Space complexity Time complexity Message complexity Bit complexity Round complexity What do these mean? Model and complexity Consider broadcasting in an n-cube (n=3) source

2. {Process 0} sends m to neighbors {Process i > 0} repeat receive m {m contains the value}; if m is received for the first time thenx[i] := m.value; send x[i] to each neighbor j > I else discard m end if forever What is the (1) message complexity (2) space complexity per process? Broadcasting using messages Each process j has a variable x[j] initially undefined

3. {Process 0} x[0] := v {Process i > 0} repeat if  a neighbor j < i : x[i] ≠ x[j] then x[i] := x[j] {this is a step} else skip end if forever What is the time complexity? (i.e. how many steps are needed?) Can be arbitrarily large! WHY? Broadcasting using shared memory Each process j has a variable x[j] initially undefined

4. Now, use “large atomicity”, where in one step, a process j reads the states of ALL its neighbors of smaller id, and updates x[j] only when these are equal, and different from x[j]. What is the time complexity? How many steps are needed? The time complexity is now O(n2) Broadcasting using shared memory Each process j has a variable x[j] initially undefined

5. Rounds are truly defined for synchronous systems. An asynchronous round consists of a number of steps where every process (including the slowest one) takes at least one step. How many rounds will you need to complete the broadcast using the large atomicity model? Time complexity in rounds Each process j has a variable x[j] initially undefined

6. Representing distributed algorithms • Why do we need these? • Don’t we already know a • lot about programming? • Well, you need to capture the notions of atomicity, non-determinism, fairnessetc. These concepts are not built into languages like JAVA, C++ etc!

7. Syntax & semantics: guarded actions <guard G><action A> is equivalent to if G then A

8. Syntax & semantics: guarded actions • Sequential actions S0; S1; S2; . . . ; Sn • Alternative constructsif . . . . . . . . . . fi • Repetitive constructsdo . . . . . . . . . od The specification is useful for representing abstract algorithms, not executable codes.

9. Syntax & semantics Alternative construct if G1 S1  G2 S2 …  GnSn fi When no guard is true, skip (do nothing). When multiple guards are true, the choice of the action to be executed is completely arbitrary.

10. Syntax & semantics Repetitive construct do G1S1  G2 S2 .  Gn Sn od Keep executing the actions until all guards are false and the program terminates. When multiple guards are true, the choice of the action is arbitrary.

11. Example: graph coloring There are four processes. The system has to reach a configuration in which no two neighboring processes have the same color. 1 0 0 {program for process i} do j  neighbor(i): c(j) = c(i)  c(i) := 1-c(i) od 1 Will the above computation terminate?

12. Consider another example program uncertain; define x : integer; initially x = 0 do x < 4  x := x + 1  x = 3  x := 0 od Question. Will the program terminate? (Our goal here is to understand fairness)

13. The adversary A distributed computation can be viewed as a game between the system and an adversary. The adversary may come up with feasibleschedules to challenge the system (and cause “bad things”. A correct algorithm must be able to prevent those bad things from happening.

14. Non-determinism Token server (Program for a token server - it has a single token} repeat if (req1 and token) then give the token to client1 elseif (req2 and token) then give the token to client2 elseif (req3 and token) then give the token to client3 forever Now, assume that all three requests are sent simultaneously. Client 2 or 3 may never get the token! The outcome could have been different if the server makes a non-deterministic choice 3 1 2 Non-determinism is abundant in the real world (examples?). Determinism is a special case of non-determinism.

15. Examples of non-determinism Non-determinism is abundant in the real world. If there are multiple processes ready to execute actions, The who will execute the action first is nondeterministic Message propagation delays are arbitrary and the order of message reception is non-deterministic Determinism caters to a specific order and is a special case of non-determinism.

16. Atomic = all or nothing Atomic actions = indivisible actions do red message  x:= 0 {red action}  blue message  x:=7 {blue action} od Regardless of how nondeterminism is handled, we would expect that the value of x will be an arbitrary sequence of 0's and 7's. Right or wrong? Atomicity (or granularity) x

17. do red message  x:= 0 {red action}  blue message  x:=7 {blue action} od Let x be a 3-bit integer x2 x1 x0, so x:=7 means x2:=1, x1:= 1, x2:=1, and x:=0 means x2:=0, x1:= 0, x2:=0 If the assignment is not atomic, then many Interleavings are possible, leading to any possible values of x Atomicity (continued) x So, the answer may depend on atomicity

18. Does hardware guarantee any form of atomicity? Yes! Transactions are atomic by definition (in spite of process failures). Also, critical section codes are atomic. We will assume that G  A is an “atomic operation.” Does it make a difference if it is not so? Atomicity (continued) if x ≠ y  y:= x fi x y if x ≠ y  y:= x fi

19. {Program for P} define b: boolean initially b = true do b  send msg m to Q ¬ empty(R,P) receive msg; b := false od Suppose it takes 15 seconds to send the message. After 5 seconds, P receives a message from R. Will it stop sending the remainder of the message? NO. Atomicity (continued) R P Q b

20. Fairness Defines the choices or restrictions on the scheduling of actions. No such restriction implies an unfair scheduler. For fair schedulers, the following types of fairness have received attention: • Unconditional fairness • Weak fairness • Strong fairness Scheduler / demon / adversary

21. Program test define x : integer {initial value unknown} do true  x : = 0  x = 0  x : = 1  x = 1  x : = 2 od An unfair schedulermay never schedule the second (or the third actions). So, x may always be equal to zero. An unconditionally fair scheduler will eventually give every statement a chance to execute without checking their eligibility. (Example: process scheduler in a multiprogrammed OS.) Fairness

22. Program test define x : integer {initial value unknown} do true  x : = 0  x = 0  x : = 1  x = 1  x : = 2 od A scheduler is weakly fair, when it eventually executes every guarded action whose guard becomes true, and remains true thereafter A weakly fair scheduler will eventually execute the second action, but may never execute the third action. Why? Weak fairness

23. Program test define x : integer {initial value unknown} do true  x : = 0  x = 0  x : = 1  x = 1  x : = 2 od A scheduler is strongly fair, when it eventually executes every guarded action whose guard is true infinitely often. The third statement will be executed under a strongly fair scheduler. Why? Strong fairness