A Mile-High View of Concurrent Algorithms

1. A Mile-High View of Concurrent Algorithms Hagit AttiyaTechnion

2. A Simplistic View of Concurrent Systems A collection of processes Each a sequential thread of execution Communicating through shared data structures Concurrent algorithms @ COVA

3. Alternative Routes for Developing Concurrent Algorithms Refinement: Implementing high-level ADT from lower-level ADTs • Safety conditions, liveness properties • Hierarchical Transactional: Support for running sequential applications concurrently • Safety conditions, liveness properties Concurrent algorithms @ COVA

4. Abstract Data Types (ADT) Cover most concurrent applications At least encapsulate their data needs An object-oriented programming point of view Abstract representation of data& set of methods (operations) for accessing it Signature Specification data Concurrent algorithms @ COVA

5. Implementing High-Level ADT data data Concurrent algorithms @ COVA

6. Implementing High-Level ADT data data Using lower-level ADTs & procedures ------------------ ------------------- ------------------ ---------------- ---------------- --------------- ------------------ ------------------- Concurrent algorithms @ COVA

7. Lower-Level Operations High-level operations translate into primitives on base objects Obvious: read, write (restrictions?) Common: compare&swap (CAS) LL/SC, Double-CAS (2CAS, DCAS), kCAS, … Generic: read-modify-write (RMW), kRMW Low-level operations are often implemented from more primitive operations A hierarchy of implementations Concurrent algorithms @ COVA

8. Executing Operations invocation response P1 P2 P3 Concurrent algorithms @ COVA

9. Interleaving Operations Concurrent execution Concurrent algorithms @ COVA

10. Interleaving Operations (External) behavior Concurrent algorithms @ COVA

11. Interleaving Operations, or Not Sequential execution Concurrent algorithms @ COVA

12. Interleaving Operations, or Not Sequential behavior: invocations & response alternate and match (on process & object) Sequential specification: All the legal sequential behaviors, satisfying the semantics of the ADT E.g., for a (LIFO) stack: pop returns the last item pushed Concurrent algorithms @ COVA

13. Correctness: Sequential consistency [Lamport, 1979] For every concurrent execution there is a sequential execution that Contains the same operations Is legal (obeys the sequential specification) Preserves the order of operations by the same process Concurrent algorithms @ COVA

14. Sequential Consistency: Examples push(4) pop():4 push(7) push(4) pop():4 push(7) Concurrent (LIFO) stack  First Out Last In  Concurrent algorithms @ COVA

15. Sequential Consistency: Examples push(4) pop():7 push(7) Concurrent (LIFO) stack  First Out Last In Concurrent algorithms @ COVA

16. Example 1: Multi-Writer Registers Add logical time (Lamport timestamps) to values Write(v) read TS1,...,read TSn TSi = max TSj +1 write v,TSi Read only own value Read() read v,TSi return v Once in a while read TS1,...,read TSn write max TSjto TSi ~[Attiya, Welch TOCS 1994] Using (multi-reader) single-writer registers Need to ensure writes are eventually visible Concurrent algorithms @ COVA

17. Multi-Writer Registers: Proof Write(v,X) read TS1,...,read TSn TSi = max TSj +1 write v,TSi Read(X) read v,TSi return v Once in a while read TS1,...,read TSn write max TSjto TSi • Create sequential execution: • Place writes in timestamp order • Insert reads after the appropriate write Concurrent algorithms @ COVA

18. Multi-Writer Registers: Proof • Create sequential execution: • Place writes in timestamp order • Insert reads after the appropriate write • Legality is immediate • Per-process order is preserved since a read returns a value (with timestamp) larger than the preceding write by the same process Concurrent algorithms @ COVA

19. Sequential Consistency is not Composable enq(Q1,X) enq(Q2,X) deq(Q1,Y) enq(Q2,Y) enq(Q1,Y) deq(Q2,X) The execution is not sequentially consistent Concurrent algorithms @ COVA

20. Sequential Consistency is not Composable enq(Q1,X) deq(Q1,Y) enq(Q1,Y) enq(Q2,X) enq(Q2,Y) deq(Q2,X) Bad news for verification! The execution projected on each object is sequentially consistent Concurrent algorithms @ COVA

21. Correctness: Linearizability [Herlihy & Wing, 1990] For every concurrent execution there is a sequential execution that Contains the same operations Is legal (obeys the specification of the ADTs) Preserves the real-time order of non-overlapping operations Each operation appears to takes effect instantaneously at some point between its invocation and its response (atomicity) Concurrent algorithms @ COVA

22. Linearizable Multi-Writer Registers Add logical time to values Write(v,X) read TS1,...,read TSn TSi = max TSj +1 write v,TSi Read(X) read TS1,...,read TSn return value with max TS [Vitanyi & Awerbuch, 1987] Using (multi-reader) single-writer registers Concurrent algorithms @ COVA

23. Multi-writer registers: Linearization order Write(v,X) read TS1,...,read TSn TSi = max TSj +1 write v,TSi • Create linearization: • Place writes in timestamp order • Insert each read after the appropriate write Read(X) read TS1,...,read TSn return value with max TS Concurrent algorithms @ COVA

24. Multi-Writer Registers: Proof • Create linearization: • Place writes in timestamp order • Insert each read after the appropriate write • Legality is immediate • Real-time order is preserved since a read returns a value (with timestamp) larger than all preceding operations Concurrent algorithms @ COVA

25. Linearizability is Composable The whole system is linearizable  each object is linearizable Allows to implement and verify each object separately Good news for verification! Concurrent algorithms @ COVA

26. Example 3: Atomic Snapshot m components Update a single component Scan all the components “at once” (atomically) Provides an instantaneous view of the whole memory update scan v1,…,vm ok Concurrent algorithms @ COVA

27. Atomic Snapshots: Algorithm [Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993] double collect Update(v,k) A[k] = v,seqi,i • Scan() • repeat • read A[1],…,A[m] • read A[1],…,A[m] • if equal • return A[1,…,m] • Linearize: • Updates with their writes • Scans inside the double collects Concurrent algorithms @ COVA

28. read A[1],…,A[m] read A[1],…,A[m] write A[j] Atomic Snapshot: Linearizability Double collect (read a set of values twice) If equal, there is no write between the collects • Assuming each write has a new value (seq#) Creates a “safe zone”, where the scan can be linearized Concurrent algorithms @ COVA

29. Liveness Conditions (Eventual) • Wait-free: every operation completes within a finite number of (its own) steps • no starvation for mutex • Nonblocking: some operation completes within a finite number of (some other process) steps • deadlock-freedom for mutex • Obstruction-free: an operation (eventually) running solo completes within a finite number of (its own) steps • Also called solo termination wait-free  nonblocking  obstruction-free Concurrent algorithms @ COVA

30. Liveness Conditions (Bounded) • Wait-free: every operation completes within a bounded number of (its own) steps • no starvation for mutex • Nonblocking: some operation completes within a bounded number of (some other process) steps • deadlock-freedom for mutex • Obstruction-free: an operation (eventually) running solo completes within a bounded number of (its own) steps • Also called solo termination Bounded wait-free  bounded nonblocking  bounded obstruction-free Concurrent algorithms @ COVA

31. Wait-free Atomic Snapshot [Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993] • Embed a scan within the Update. Update(v,k) V = scan A[k] = v,seqi,i,V • Scan() • repeat • read A[1],…,A[m] • read A[1],…,A[m] • if equal • return A[1,…,m] • else record diff • if twice pj • return Vj direct scan • Linearize: • Updates with their writes • Direct scans as before • Borrowed scans with source borrowedscan Concurrent algorithms @ COVA

32. embedded scan read A[j] read A[j] read A[j] read A[j] … … … … … … … … write A[j] write A[j] Atomic Snapshot: Borrowed Scans Interference by process pj And another one… • pj does a scan inbeteween Linearizing with the borrowed scan is OK. Concurrent algorithms @ COVA

33. Alternative Routes for Developing Concurrent Algorithms Refinement: Implementing high-level ADT from lower-level ADTs • Safety conditions, liveness properties • Hierarchical Transactional: Support for running sequential applications concurrently • Safety conditions, liveness properties Concurrent algorithms @ COVA

34. The Transactional Approach [Herlihy, Moss, ISCA 1993] Systematic approach for implementing concurrent data structures Program sequentially But run concurrently Should appear to execute sequentially No high-level signature / semantics The sequential program is the specification Concurrent algorithms @ COVA

35. Transactional Synchronization A transaction aggregates a sequence of resource accesses to be executed atomically A sequence of atomic actions Like in database systems A transaction ends either by committing all its updates take effect or by aborting no update is effective Read X Write X Read Z Read Y Concurrent algorithms @ COVA

36. Safety: Serializability [Papadimitriou 79][Weikum, Vossen, 2002, Chapter 3] An analogue of sequential consistency Any interleaving of the transactions yields a result that could be achieved in a sequential execution of the same set of transactions (a serialization) Just the committed transactions?(Aborted transactions have no effect.) Final state serializability Concurrent algorithms @ COVA

37. Safety: View Serializability [Yannakakis 1984] What about intermediate values read? Could be “corrected” later But still cause harm, e.g., division by 0 Any interleaving of the transactions has an equivalent sequential execution of the same transactions Where all reads return the same value Makes no sense in the context of implementing a high-level ADT Where the internals are not exposed Concurrent algorithms @ COVA

38. Safety: Strict Serializability [Papadimitriou 79][Bernstein, Shipman & Wong, 1979] The serialization must preserve the real-time order of (non-overlapping) transactions An analogue of linearizability Called ordered serializabililty in W&V Strict view serializability Concurrent algorithms @ COVA

39. Opacity [Guerraoui & Kapalka, PPoPP 08] Essentially, strict view serializability Applied to all transactions (not only committed or completed ones) Generalized to arbitrary object types (not just reads and writes) Concurrent algorithms @ COVA

40. Liveness Properties • As in high-level implementations • But restricted to successful completions (commit) E.g., wait-freedom every transaction eventually commits,obsruction-freedom every transaction (eventually) running solo terminates Concurrent algorithms @ COVA

41. A Shift in Terminology < 2003 ≥ 2003 Lock-free Nonblocking Obstruction-free Obstruction-free Nonblocking Lock-free Wait-free Wait-free Concurrent algorithms @ COVA

42. Where’s the Confusion? [Herlihy & Wing, TOPLAS 1990] [Herlihy, TOPLAS 1991] [Herlihy, Luchangco, Moir, ICDCS 2003] Concurrent algorithms @ COVA

43. The Road Ahead • Concurrent algorithms pose a great challenge • It is “easy” to write • Correct algorithms and let efficiency take care of itself • Efficient algorithms and let correctness take care of itself • But very hard to write correct & efficient algorithms Systematically… Concurrent algorithms @ COVA