A Calculus of Atomic Actions Tayfun Elmas , Shaz Qadeer and Serdar Tasiran POPL ‘09

1. A Calculus of Atomic ActionsTayfunElmas, ShazQadeer and SerdarTasiranPOPL ‘09 236825 – Seminar in Distributed Algorithms Cynthia Disenfeld 27/05/2013 *Some slides are based on those of the authors

2. Goal • Statically verifying (partial) correctness of shared-memory multithreaded programs • Difficult: understand thread-interaction + shared memory • Single-thread programs: pre/post conditions, loop invariants • Multithreaded programs: consider the effect of all thread-interleavings (e.g. Owicki-Gries)

3. Approach • (Sound) program transformations • Abstraction • Reduction • Invariant strengthening

4. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

5. Motivating examples Lock-based atomic increment void inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } void inc() { int t; [havoc t; x := x+1;] }

6. Motivating examples Lock-free atomic increment void inc() { int t; while(true){ t := x; if (CAS(x,t,t+1) break; } } void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } * Transformation from Flanagan et al.[2005]

7. Motivating examples Lock-free atomic increment void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; }

8. Motivating examples Lock-free atomic increment void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } void inc() { int t; havoc t; havoc t; [assume x==t; x := t+1]; } void inc() { int t; [havoc t; x := x+1]; }

9. Motivating examples Client of inc void add(int n){ while (0<n) { inc() n := n-1; } } void add(int n){ while (0<n){ [x := x+1] n := n-1; } } void add(int n){ [assert 0<=n; x := x+n; n := 0]; }

10. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

11. QED approach Proof step Program Invariant Fine-grained concurrency Hard to prove! Invariant = True Coarse-grained concurrency Easy to prove If the original program may fail the new program may fail

12. Soundness For all If then exists If then exists or

13. Soundness • For each proof step • Proof steps: • Invariant strengthening • Reduction: build more coarse-grained atomic blocks • Abstraction: add (possibly failing) behaviors

14. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

15. Reduction inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } main() { x := 0; inc(); || inc(); assert(x==2); }

16. Reduction Right Mover inc() { int t; acquire(lock); R t := x; B t := t+1; B x := t; B release(lock); L } • inc() { • int t; • acquire(lock); • t := x; • t := t+1; • x := t; • release(lock); • } • inc() • x := x+1 REDUCE-SEQUENTIAL

17. Reduction main() { x := 0; • inc(); || inc(); • assert(x==2); • } main() { x := 0; B x := x+1; || x := x+1; B assert(x==2); } • main() { • x := 0; • x := x+1; • x := x+1; • assert(x==2); • } REDUCE-PARALLEL INLINE-CALL

18. Static mover check Check using the current invariant if they access different variables, or are not enabled at the same time Each statement consists of: when can it be applied? how is the state changed?

19. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

20. Abstraction rule • Replace with if For all If then exists If then exists or

21. Abstraction void inc() { int t; while(*){ t := x; assume x!=t; } t := x; [assume x==t; x := t+1]; } void inc() { int t; while (*) { havoc t; skip; } havoc t; [assume x==t; x := t+1]; } SIMULATE • Then, we can reduce • havoc t + skip • while (*){…}  havoc t

22. Abstraction • Adding non-determinism • Guards  if(*) • t := x  havoc t • assume …  skip • Adding behaviors that may go wrong • x := x+1  if (x==0) fail; x := x+1 • y := y-x  assert (y>x); y := y-x

23. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

24. Example – Sorted linked list • Hand-over-hand locking • Find, insert, delete • Local assertions • Class invariant • Atomic  easy! • But… implementation with fine-grained locking

25. Insert(5)

26. Insert(x) p := find(x); //locks p n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); //they are linked as they should and their values have increasing order UNLOCK(p);

27. Insert(x) p.val and p.next are not affected by other threads invariant: list is sorted p := find(x); n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); t.val and t.next are not affected by other threads

28. Proof p := find(x); n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); find appropriate p LOCK(p) n := p.next; t := new Node(); R t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); L

29. Proof find appropriate p LOCK(p) n := p.next; t := new Node(); R t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); L find appropriate p LOCK(p) n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p); how to continue?

30. Apparent interference Thread A Thread B p.next := t; p.next := t; n := p.next; n := p.next;

31. Apparent interference Thread A Thread B p.next := t; p.next := t; n := p.next; n := p.next; But: both p’s are locked!

32. Ruling out interference - 1 Thread A Thread B assert owner[p]==A p.next := t; assert owner[p]==A p.next := t; assert owner[p]==B n := p.next; assert owner[p]==B n := p.next;

33. Ruling out interference - 2 Thread A Thread B assert !inList[t] t.next := n; assert !inList[t] t.next := n; assert inList[p] n := p.next; assert inList[p] n := p.next;

34. Reduction after abstraction find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p); find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p);

35. Borrowed assertions find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p); find appropriate p LOCK(p) assert inList[p]&&owner[p]==tid n := p.next; t := new Node(); t.val := x; assert !inList[t] t.next := n; assert inList[p]&&owner[p]==tid p.next := t; assert (p, t, n sorted); assert owner[p]==tid UNLOCK(p);

36. Completing the proof Invariant : List is sorted find appropriate p LOCK(p) n := p.next; t := new Node(); t.val := x; t.next := n; p.next := t; assert (p, t, n sorted); UNLOCK(p);

37. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

38. Tactics • High-level strategies  multiple rule proofs • abstract from a read, write • add invariants • synchronization mechanisms

39. Example mutex P, x1, … , xn mutex (lock==true), x inc() { int t; acquire(lock); t := x; t := t+1; x := t; release(lock); } acquire(lock) { assume lock==false; lock := true; } release(lock) { lock := false; }

40. Tactic - mutex inc() { int t; acquire(lock); a=tid; t := x; t := t+1; x := t; release(lock); a=0; } AUX-ANNOTATE Invariant: lock==true iff a !=0

41. Tactic - mutex inc() { int t; acquire(lock); a=tid; t := x; t := t+1; x := t; release(lock); a=0; } inc() { int t; acquire(lock); a=tid; assert a==tid;t := x; t := t+1; • assert a==tid;x := t; • assert a==tid; • release(lock); a=0; } SIMULATE

42. Tactic - mutex inc() { int t; acquire(lock); a=tid; R assert a==tid;t := x; B t := t+1; B • assert a==tid;x := t; B • assert a==tid; • release(lock); a=0; L }

43. Tactic - mutex inc() { int t; acquire(lock); a=tid; assert a==tid; t := x; t := t+1; • assert a==tid; • x := t; • assert a==tid; • release(lock); a=0; } REDUCE & RELAX

44. Outline • Motivating examples • Approach – Soundness • Reduction • Abstraction • Borrowing assertions • Tactics • Experience / Conclusions

45. Experience • Implementation • Boogie + parallel composition • Verification conditions for validity of each step: Z3 SMT Solver • Benchmarks  without complicated global invariants • Fine-grained locking • Multiset • Hand-over-hand locking • Non-blocking algorithms

46. Future work • More tactics • More synchronization mechanisms • C / Spec# • Larger verification problems

47. Conclusions • A formal and sound proof calculus for atomicity was presented • Abstraction helps applying reduction and the other way around • Assertions can be added and checked only later • The program is simplified by obtaining coarser atomic actions • Tactics can be defined to represent different synchronization mechanisms