Global States in a Distributed System

1. Global States in a Distributed System By John Kor and Yvonne Cheng

2. Initial Problem Example • Garbage Collector • Free’s up memory which is no longer in use • Check’s if a reference to memory still exists • What about in a distributed system

3. Initial Problem Example (cont’d) • A distributed system consists of multiple processes • Each process is located on a different computer • No sharing of processor or memory

4. Initial Problem Example (cont’d) • Each process can only determine its own “state” • Problem: How do we determine when to garbage collect in a distributed system? • How do we check whether a reference to memory still exists?

5. System Model • A distributed system consists of multiple processes • Each process is located on a different computer • Each process consists of “events” • An event is either sending a message, receiving a message, or changing the value of some variable • Each process has a communication channel in and out

6. Our Garbage Collection Problem • In order to test whether a certain property of our system is true, we cannot just look at each process individually • A “snapshot” of the entire system must be taken to test whether a certain property of the system is true • This “snapshot” is called a Global State

7. Definition The global state of a distributed system is the set of local states of each individual processes involved in the system plus the state of the communication channels.

8. Determinism • Deterministic Computation • At any point in computation there is at most one event that can happen next. • Non-Deterministic Computation • At any point in computation there can be more than one event that can happen next.

9. Deterministic Computation

10. Non-Deterministic Computation

11. Determinism • Deterministic computation • A local event would reveal everything about the global state! • The process will know other process’ state • Non-Deterministic computation • Because of branching, a local event cannot reveal what the next step will be

12. Simple Algorithm • Create a new process that collects the states of every other process • Every process will save their state at an arbitrary time and send it to this new process

13. Advantages • Very simple • Easy to implement

14. Problems? • Based on the assumption that all processes work on a synchronized global clock • Wrong assumption!

15. Problems (cont’d) State recorded by p p q m

16. Problems (cont’d) p q m

17. Problems (cont’d) State recorded by q p q m

18. Problems (cont’d) Global state recorded p q m m

19. Another view p m q

20. Another view • Process p has no record of sending m • Process q HAS record of receiving m • Problem? • Global state does not show p sending m, therefore there is confusion as to where m came from • Breaks the Consistency concept

21. Consistency • A global state is consistent if it could have been observed by an external observer • If e  e` , then both e and e` must reside within the same state • For a successful Global State, all states must be consistent

22. Solution • Need to develop an asynchronous algorithm • Cannot depend on a clock • Must ensure consistency in all global states

23. Assumptions • Distributed system: Finite set of processes and channels; described by graph • Processes • Set of states, initial state, set of events • Channels • FIFO, error-free, infinite buffers, arbitrary but finite delay

24. PART 2 Presented By: Yvonne

25. Idea of a global state recording algorithm • each process records its own state • the two processes incident by one channel cooperate in recording the channel state

26. Challenge • No global clock • Need a meaningful result • Superimposed on underlying computation

27. Meaningful: The notion of Consistency • it could have been observed by an external observer • All feasible states are consistent

28. An Example q p Sp0 Sp1 Sp2 Sp3 p m2 m1 m3 q Sq0 Sq1 Sq2 Sq3

29. A Consistent State? q p Sq1 Sp1 Sp0 Sp1 Sp2 Sp3 p m2 m1 m3 q Sq0 Sq1 Sq2 Sq3

30. Yes q p Sq1 Sp1 Sp0 Sp1 Sp2 Sp3 p m2 m1 m3 q Sq0 Sq1 Sq2 Sq3

31. A Consistent State? q p Sq3 Sp2 m3 Sp0 Sp1 Sp2 Sp3 p m2 m3 m1 q Sq0 Sq1 Sq2 Sq3

32. Yes q p Sq3 Sp2 m3 Sp0 Sp1 Sp2 Sp3 p m2 m3 m1 q Sq0 Sq1 Sq2 Sq3

33. An inconsistent State q p Sq3 Sp1 Sp0 Sp1 Sp2 Sp3 p m2 m1 m3 q Sq0 Sq1 Sq2 Sq3

34. Conducting algorithm: Using An Example • Processes: p and q • Channels: c and c’ • Token: t p q c c’

35. An Example • p records its state p q c t c’

36. An Example • q, c, and c’ record their states p q c t c’

37. An Example • The composite global state! p q c t t c’

38. An Example • n: number of messages sent along c before p’s state is recorded • n’: number of message sent along c before c’s state is recorded p q c c’

39. An Example p q - Reason of inconsistency: n<n’ c t n = 0 c’ p q c t n’ = 1 c’

40. Similar scenario • c is recorded when the token is at process p. • p sends the token through channel c, and the states of c’, p, and q are recorded. • The recorded global state : no tokens in the system. • The reason of inconsistency : n>n’

41. Conclusion from the example A consistent global state requires n = n’

42. Similar Conclusion • m : number of messages received along c before q’s state is recorded • m’ : number of messages received along c before c’s state is recorded • To be consistency: m=m’

43. Some other equations n’ >= m’ • m’ : number of messages received along c before c’s state is recorded • n’ : number of messages sent along c before c’s state is recorded • m : number of messages received along c before p’s state is recorded • n : number of messages sent along c before p’s state is recorded • n = n’ • m = m’ n >= m

44. Other Fact • The state of channel c that is recorded must be the sequence of messages sent along the channel before the sender’s state is recorded, excluding the sequence of messages received along the channel before the receiver’s state is recorded. • Two cases: • n’=m’ : c is empty • n’>m’: c must be the (m’+1)st…n’th messages sent by p along c

45. Put All Together:A brief sketch of the algorithm • p sends a marker message along all its outgoing channels after it records its state and before it sends any other messages. • On receipt of a marker message from channel c • else • state ( c ) = messages received on c since it had recorded its state excluding the marker. • if p has not recorded its state • record the state • state ( c ) = EMPTY

46. Chandy and Lamport Algorithm • Features: • Does not promise us to give us exactly what is there • But gives us consistent state!!

47. Algorithm in Action Sp0 Sp1 Sp2 Sp3 p m1 m2 m3 q Sq0 Sq1 Sq2 Sq3

48. Algorithm in Action q records state as Sq1 , sends marker to p Sp0 Sp1 Sp2 Sp3 p m1 m2 m3 q Sq0 Sq1 Sq2 Sq3

49. Algorithm in Action p records state as Sp2, channel state as empty Sp0 Sp1 Sp2 Sp3 p m1 m2 m3 q Sq0 Sq1 Sq2 Sq3

50. Algorithm in Action q records channel state as m3 Sp0 Sp1 Sp2 Sp3 p m1 m2 m3 q Sq0 Sq1 Sq2 Sq3