Where Concurrency meets Distributed Algorithms (for me)

1. Where Concurrency meets Distributed Algorithms(for me) Joint with: Kai Engelhardt Ron van der Meyden Yoram Moses Technion EE

2. Distributed behavior is complex • Rampant nondeterminism: scheduling, timing, reliability • Multitude of concurrent and co-dependent events • Decentralized information and Control Hence…

3. Hence… • Basic tasks are hard to solve • Mutual exclusion, Paxos, mail, agreement, voting, spanning trees… • Careful modeling is crucial • For verification, lower bounds • Distributed algorithms literature is fragmented • Every interesting task demands significant attention • Many different models • Problems solved one by one on a per-model basis • Similar solutions often repeated in different models • “Bottom up” but individual solutions may or may not be compatible.

4. Focus of this talk • Top-Down considerations • Refinement • Cuts vs. global states • Seq. Composition of distributed algorithms • Notions of safe composition • Sealing and message reordering

5. Main Points • Need tools for abstracting and unifying distributed solutions • Both global states and cuts can matter • Notions of safe sequential composition are interesting

6. Goal No. 1: A Program Refinement Calculus Desired: A top-down process by which a program is repeatedly replaced by a “more detailed” one, that satisfies the same requirements. P refines Q if every execution of P is an execution of Q Refinement for sequential programs is well established Back, Morgan et al.

7. Example ; ;

8. Programs and Specifications “Programs” can contain both specifications and concrete commands Specifications are described using a pair [,] of apre- and post-condition, meaning: If started in a state satisfying  the program will terminate, and its final state will satisfy  Reactive behavior can be obtained by temporal statements in spec, and state abstraction by e.g., epistemic statements.

9. Refinement Rules Splitting Rule: [,] ; [,] refines [,] Action Rules: x := 1 refines [true,x=1] Refinement allows “correctness by design,” maintainability, and design for different models

10. Review: Sequential Programs • Execute between an initial and a finalstate • Define an (initial/final) relation on states

11. A Distributed Program Should execute between distributed states A specification typically describes assumptions on the initial global state.

12. But… Distributed programs terminate along a cut Even if P starts at a global state, Q will not.

13. What is a distributed state? A sequential state serves two roles: Transformation: Actions modify the state Control: Programs execute between states Claim:In the distributed case Transformation: Actions modify global state Control: Programs execute between cuts (c1,c2,…,cn)where ci≤

14. Cuts and Specs • Sequential composition amounts to “layering” P*Q: • The cuts on which layers compose are not necessarily consistent • The formulas in a spec [,] become cut formulae. E.g., “x=0 y=2” , “consistent cut”… • What is a good language to express these? (semantics for temporal, epistemic, …) P Q

15. Goal No. 2 Account for both aspects of a distributed state? Allow for inconsistent cuts Allow for multiplexing or forking? Havelund et al. 94

16. Sequential Composition Distributed applications often have a sequential structure: • Compute MST • Elect a leader • Perform a vote using this leader • Act on the outcome of the vote… These activities may overlap and interfere. Will individual solutions compose correctly?

17. Communication-closed layers Elrad & Francez 1982, Stomp & de Roever, Janssen Zwiers & Poel, … Suppose that P= Q*L*R (so each Pi = Qi;Li;Ri ). L is communication-closed (CC) in P if, in all executions of P, all communication actions in L are internal. Communication closure in P may depend on all of P Q L R

18. Tail Communication Closure Program L is TCC if L is communication closed in all programs P = L*Q Gerth and Shrira 86 If L and R are TCC then L*R is TCC.  allows incremental CC design Work on CC and TCC focused on reliable FIFO models. What happens beyond this?

19. Example: REL –Reliable non-FIFO channels Engelhardt & M DISC 2005 Example: 2-process message transmission In REL&FIFO the program MTij is TCC

20. Safe composition in REL MTij is sealed by MTji : It will not interact with any later layer R.

21. More Formally • We define a programming language with a “phase” operator  • (Q) means Q does not communicate with other layers • A notion of “P executes over” interval r[c,d] • A refinement relation ≤ among programs • Can define CC and Sealing: L is a CC in P= Q*L*R: (Q*L*R)≤Q*(L)*R L is TCC : (L*R)≤(L)*R for all R Q seals L: (L*Q*R)≤(L)*Q*R for all R

22. Sealing Q seals L if L is a CC in L*Q*R for all R Sealing allows incremental CC design Sealing in REL is subtle: • No universal seals (barriers) exist • Only programs with no reachable communication commands are sealed by the empty program • Sealing related to Lamport causality

23. Transitive Sealing

24. An Unsealable Program Any future communication can interfere

25. Getting by with a little help… Two-process variant of P’ was unsealable

26. Sealing and Causality Thm:Q seals L iff every rcv on a channel ij in L causally precedes any snd on ij in or after Q (and L can consume all messages it sends) Prop: A straight-line program has a succinct signature that can be used to decide issues of sealing Thm: If L is sealable then it has a seal that uses O(n) messages (replacing O(n2) acknowledgements)

27. Beyond Sealing: Fitting After • Program Q fits after L if layer L never communicates with Q in L*Q: (L*Q)≤(L)*Q • Sealing implies fitting after, but the converse is false. • Fitting after allows for incremental CC design. • Fitting after is natural in FIFO models (channels are lossy and/or allow duplication) • REL is the only imperfect model that does not require headers for composition Fekete and Lynch CONCUR 90, Engelhardt & M 05

28. Conclusions • Models of concurrency are crucial for proving properties of programs and lower bounds or impossibility results • There is a role for concurrency modeling in top-down design, refinement and the study of compositionality • Sequential composition is of interest and has a rich structure