Distributed Systems: Distributed algorithms

1. Distributed Systems: Distributed algorithms Distributed systems: distributed algorithms

2. Overview of chapters • Introduction • Co-ordination models and languages • General services • Distributed algorithms • Ch 10 Time and global states, 11.4-11.5 • Ch 11 Coordination and agreement, 12.1-12.5 • Shared data • Building distributed services Distributed systems: distributed algorithms

3. This chapter: overview • Introduction • Logical clocks • Global states • Failure detectors • Mutual exclusion • Elections • Multicast communication • Consensus and related problems Distributed systems: distributed algorithms

4. Logical clocks • Problem: ordering of events • requirement for many algorithms • physical clocks cannot be used • use causality: • within a single process: observation • between different processes: sending of a message happensbeforereceiving the same message Distributed systems: distributed algorithms

5. Logical clocks (cont.) • Formalization: happens before relation • Rules: • if x happens before y in any process p thenx  y • for any message m: send (m)  receive (m) • ifx  yandy  zthenx  z • Implementation: logical clocks x  y Distributed systems: distributed algorithms

6. Logical clocks (cont.) • Logical clock • counter appropriately incremented • one counter per process • Physical clock • counts oscillations occurring in a crystal at a definitive frequency Distributed systems: distributed algorithms

7. Logical clocks (cont.) • Rules for incrementing local logical clock • for each event (including send) in process p:Cp := Cp + 1 • when a process sends a message m, it piggybacks on m the value of Cp • on receiving (m, t), a process q • computes Cq := max (Cq, t) • applies rule 1: Cq := Cq +1 • Cq is logical time for event receive(m) Distributed systems: distributed algorithms

8. 0 5 3 2 4 1 3 1 • • • • • • • • P1 P2 P3 e a b g c f d Logical clocks (cont.) • Logical timestamps: example Distributed systems: distributed algorithms

9. Logical clocks (cont.) • C(x) logical clock value for event x • Correct usage: • ifx  ythen C(x) < C(y) • Incorrect usage: • if C(x) < C(y) thenx  y • Solution: Logical vector clocks Distributed systems: distributed algorithms

10. Logical clocks (cont.) • Vector clocks for N processes: • at process Pi: Vi[j] for j = 1, 2,…,N • Properties: • ifx  ythen V(x) < V(y) • if V(x) < V(y) thenx  y Distributed systems: distributed algorithms

11. Logical clocks (cont.) • Rules for incrementing logical vector clock • for each event (including send) in process Pi: Vi[i] := Vi[i] + 1 • when a process Pi sends a message m, it piggybacks on m the value of Vi • on receiving (m, t), a process Pi • apply rule 1 • Vi[j] := max(Vi[j] , t[j]) for j = 1, 2,…, N Distributed systems: distributed algorithms

12. Logical clocks (cont.) • Logical vector clocks : example Distributed systems: distributed algorithms

13. This chapter: overview • Introduction • Logical clocks • Global states • Failure detectors • Mutual exclusion • Elections • Multicast communication • Consensus and related problems Distributed systems: distributed algorithms

14. Global states • Detect global properties Distributed systems: distributed algorithms

15. Global states (cont.) • Local states & events • Process Pi : eik events sik state, before event k • History of Pi: hi = < ei0, ei1, ei2,…> • Finite prefix of history of Pi: hik = < ei0, ei1, ei2,…, eik > Distributed systems: distributed algorithms

16. Global states (cont.) • Global states & events • Global history H = h1 h2 h3 … hn • Global state (when?) S = ( s1p, s2q, …, snu)consistent? • Cut of the systems execution C = h1c1  h1c2  …  h1cn Distributed systems: distributed algorithms

17. Global states (cont.) • Example of cuts: Distributed systems: distributed algorithms

18. Global states (cont.) • Finite prefix of history of Pi: hik = < ei0, ei1, ei2,…, eik > • Cut of the systems execution C = h1c1  h1c2  …  h1cn • Consistent cut C e  C, f  e  f  C • Consistent global state corresponds to consistent cut Distributed systems: distributed algorithms

19. Global states (cont.) • Model execution of a (distributed) system S0 S1 S2 S3 … • Series of transitions between consistent states • Each transition corresponds to one single event • Internal event • Sending message • Receiving message • Simultaneous events order events Distributed systems: distributed algorithms

20. Global states (cont.) • Definitions: • Run = ordering of all events (in a global history) consistent with each local history’s ordering • Linearization =consistent run + consistent with  • S’ reachable from S linearization: … S  …  S’… Distributed systems: distributed algorithms

21. Global states (cont.) • Kinds of global state predicates: • Stable • Safety • Liveness • = true in S S’, S  …  S’ = true in S’  = undesirable property S0 =initial state of system S, S0 …  S = false in S  = desirable property S0 =initial state of system S, S0 …  S = true in S Distributed systems: distributed algorithms

22. Global states (cont.) • Snapshot algorithm of Chandy & Lamport • Record consistent global state • Assumptions: • Neither channels nor processes fail • Channels are unidirectional and provide FIFO-ordered message delivery • Graph of channels and processes is strongly connected • Any process may initiate a global snapshot • Process may continue their execution during the snapshot Distributed systems: distributed algorithms

23. Global states (cont.) • Snapshot algorithm of Chandy & Lamport • Elements of algorithm • Players: processes Pi with • Incoming channels • Outgoing channels • Marker messages • 2 rules • Marker receiving rule • Marker sending rule • Start of algorithm • A process acts as it received a marker message Distributed systems: distributed algorithms

24. Global states (cont.) Marker receiving rule for process pi On pi’s receipt of a marker message over channel c: if (pi has not yet recorded its state) it records its process state now; records the state of c as the empty set; turns on recording of messages arriving over other incoming channels; else pi records the state of c as the set of messages it has received over c since it saved its state. end if Marker sending rule for process pi After pi has recorded its state, for each outgoing channel c: pi sends one marker message over c (before it sends any other message over c). Distributed systems: distributed algorithms

25. Global states (cont.) • Example: Distributed systems: distributed algorithms

26. 1. Global state S 0 c (empty) <$1000, 0> p p <$50, 2000> 2 1 2 c (empty) 1 2. Global state S 1 c (Order 10, $100), M <$900, 0> p p <$50, 2000> 2 1 2 c (empty) 1 3. Global state S 2 c (Order 10, $100), M p p <$900, 0> <$50, 1995> 2 1 2 c (five widgets) 1 4. Global state S 3 c (Order 10, $100) <$900, 5> p p <$50, 1995> 2 1 2 C1=<(five widgets)> C2 = <> c (empty) 1 Global states (cont.) (M = marker message) Distributed systems: distributed algorithms

27. 1. Global state S 0 c (empty) <$1000, 0> p p <$50, 2000> 2 1 2 c (empty) 1 4. Global state S 6. Global state S 5. Global state S 5 3 4 c c c (Order 10, $100) (Order 10, $100) (Order 10, $100) <$900, 5> <$900, 5> <$900, 5> p p p p p p <$50, 1995> <$50, 1995> <$50, 1995> 2 2 2 1 1 1 2 2 2 C1=<(five widgets)> C1=<(five widgets)> C1=<(five widgets)> C2 = <> C2 = <> C2 = <> c c c M (empty) (empty) 1 1 1 Global states (cont.) (M = marker message) Distributed systems: distributed algorithms

28. Global states (cont.) • Observed state • Corresponds to consistent cut • Reachable! Distributed systems: distributed algorithms

29. This chapter: overview • Introduction • Logical clocks • Global states • Failure detectors • Mutual exclusion • Elections • Multicast communication • Consensus and related problems Distributed systems: distributed algorithms

30. No “P is here” within T + E sec No “P is here” within T + A sec Failure detectors • Properties • Unreliable failure detector: answers with • Suspected • Unsuspected • Reliable failure detector: answers with • Failed • Unsuspected • Implementation • Every T sec: multicast by P of “P is here” • Maximum on message transmission time: • Asynchronous system: estimate E • Synchronous system: absolute bound A Distributed systems: distributed algorithms

31. This chapter: overview • Introduction • Logical clocks • Global states • Failure detectors • Mutual exclusion • Elections • Multicast communication • Consensus and related problems Distributed systems: distributed algorithms

32. Mutual exclusion • Problem: how to give a single process temporarily a privilege? • Privilege = the right to access a (shared) resource • resource = file, device, window,… • Assumptions • clients execute the mutual exclusion algorithm • the resource itself might be managed by a server • Reliable communication Distributed systems: distributed algorithms

33. Mutual exclusion (cont.) • Basic requirements: • ME1: at most one process might execute in the shared resource at any time (Safety) • ME2: a process requesting access to the shared resource is eventually granted it (Liveness) • ME3: Access to the shared resource should be granted in happened-before order (Ordering or fairness) Distributed systems: distributed algorithms

34. Mutual exclusion (cont.) • Solutions: • central server algorithm • distributed algorithm using logical clocks • ring-based algorithm • voting algorithm • Evaluation • Bandwidth (= #messages to enter and exit) • Client delay (incurred by a process at enter and exit) • Synchronization delay (delay between exit and enter) Distributed systems: distributed algorithms

35. Mutual exclusion (cont.)central server algorithm • Central server offering 2 operations: • enter() • if resource free then operation returns without delayelse request is queued and return from operation is delayed • exit() • if request queue is emptythen resource is marked freeelse return for a selected request is executed Distributed systems: distributed algorithms

36. Server Queue: User Enter() P1 P4 P3 P2 Mutual exclusion (cont.)central server algorithm • Example: Distributed systems: distributed algorithms

37. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: User 3 Enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

38. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: User 3 P1 P4 P3 P2 Distributed systems: distributed algorithms

39. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: User 3 enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

40. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: 4 User 3 enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

41. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: 4 User 3 enter() enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

42. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: 4, 2 User 3 enter() enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

43. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: 4, 2 User 3 enter() exit() enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

44. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: 4, 2 User enter() enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

45. Mutual exclusion (cont.)central server algorithm • Example: Server Queue: 2 User 4 enter() P1 P4 P3 P2 Distributed systems: distributed algorithms

46. Mutual exclusion (cont.)central server algorithm • Evaluation: • ME3 not satisfied! • Performance: • single server is performance bottleneck • Enter critical section: 2 messages • Synchronization: 2 messages between exit of one process and enter of next • Failure: • Central server is single point of failure • what if a client, holding the resource, fails? • Reliable communication required Distributed systems: distributed algorithms

47. Mutual exclusion (cont.)ring-based algorithm • All processes arranged in a • unidirectional • logical • ring • token passed in ring • process with token has access to resource Distributed systems: distributed algorithms

48. Mutual exclusion (cont.)ring-based algorithm P2 P1 P3 P6 P4 P5 Distributed systems: distributed algorithms

49. Mutual exclusion (cont.)ring-based algorithm P2 P1 P3 P2 can use resource P6 P4 P5 Distributed systems: distributed algorithms

50. Mutual exclusion (cont.)ring-based algorithm P2 P1 P3 P2 stopped using resource and forwarded token P6 P4 P5 Distributed systems: distributed algorithms