1 / 54

Serdar Tasiran Koç University Istanbul, Turkey

Reduction, abstraction, and atomicity: How much can we prove about concurrent programs using them?. Serdar Tasiran Koç University Istanbul, Turkey Tayfun Elmas Shaz Qadeer Ali Sezgin Koç University Microsoft Research Koç University Istanbul , Turkey Redmond, WA Istanbul, Turkey. 2.

urian
Download Presentation

Serdar Tasiran Koç University Istanbul, Turkey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reduction, abstraction, and atomicity: How much can we prove about concurrent programs using them? SerdarTasiran Koç University Istanbul, Turkey TayfunElmasShaz Qadeer Ali SezginKoç UniversityMicrosoft ResearchKoç University Istanbul, Turkey Redmond, WAIstanbul, Turkey

  2. 2 Outline • QED proof system (Cartoon illustration) • QED overview • Half-baked part: • Backward reasoning in time • Prophecy variables, “tressa” annotations

  3. Example: increment 3 x := 0; || assert (x == 2); acquire(lock); t1 := x; t1:= t1 + 1;x := t1; release (lock); acquire(lock); t2 := x; t2 := t2 + 1;x := t2; release (lock);

  4. Proof with “fine grain” actions 4 B: <A@L0=>x=0, A@L5=>x=1> L0:acquire(l); <A@L0=>x=0, A@L5=>x=1, held(l,B)> L1:t2 := x; <A@L0=>x=0, A@L5=>x=1, held(l,B), t2=x> L2:t2 := t2 + 1; <A@L0=>x=0, A@L5=>x=1, held(l,B), t2=x+1> L3:x := t2; <A@L0=>x=1, A@L5=>x=2, held(l,B)> L4:release(l) <A@L0=>x=1, A@L5=>x=2> || A: <B@L0=>x=0, B@L5=>x=1> L0: acquire(l); <B@L0=>x=0, B@L5=>x=1, held(l,A)> L1: t1 := x; <B@L0=>x=0, B@L5=>x=1, held(l,A), t1=x> L2: t1 := t1 + 1; <B@L0=>x=0, B@L5=>x=1, held(l,A), t1=x+1> L3: x := t1; <B@L0=>x=1, B@L5=>x=2, held(l,A)> L4: release(l) <B@L0=>x=1, B@L5=>x=2>

  5. QED Proof of Increment (Cartoon 1) 5 inc (): acquire (lock); t1 := x; t1 := t1 + 1; x := t1; release(lock); inc (): acquire (lock); t1 := x; t1 := t1 + 1; x := t1; release(lock); Right mover Both mover B REDUCE-SEQUENTIAL B Left mover inc (): x := x + 1;

  6. QED Proof of “increment” (Cartoon 2) 6 Main: x := 0; inc() || inc() assert (x == 2) Main: x := 0; x := x + 1; x := x + 1; assert (x == 2) Main: x := 0; x := x + 1 || x := x + 1 assert (x == 2) B B REDUCE-PARALLEL INLINE-CALL

  7. The QED approach Soundness theorem: Starting from a state siin In • P1has an assertion violationPnhas an assertion violation • P1 can go to final state sf Pn can go to final state sf or has an assertion violation. Invariant Program (P1, I1)  ...  (Pi, Ii)  ...  (Pn, In) • Difficult to prove • Fine-grain concurrency • Annotations at every • interleaving point • Easy to prove • Larger atomic blocks • Local, sequential analysis • within atomic blocks

  8. 8 Outline • QED idea • QED overview • Half-baked part: • Backward reasoning in time • Prophecy variables, “tressa” annotations

  9. 9 Programs Gated action Syntax: Syntax in code examples: S ::= assume e | assert e | x := e | havoc x | S ; S | if (e) then S else S | while (e) do S | proc(a, out b) | S || S | [ S ] Semantics: • A collection of threads and a global store • Non-deterministically pick a thread and execute one atomic step • Failed assert makes the thread and program go wrong • A distinguished state “error” • Failed assume blocks the executing thread

  10. 10 Gated actions Gate: Assertion on pre-state Transition: Two-store relation x = x + 1;

  11. 11 Gated actions – examples Gate: Assertion on pre-state Transition: Two-store relation assume (x != 0); y = y / x; assert (x != 0); y = y / x; x = x + 1; assert (x != 0); y = y / x;

  12. 12 Verifying the program • Proof succeeds when all executions of starting from states in satisfy all assertions. • Sufficient condition: For all actions in the program, • Actions “large enough” to establish assertions within themselves x := 0; x := x + 1; x := x + 1; assert (x == 2)

  13. Rule 1: Strengthen invariant I,P I’,P I’  I • All statements in P must preserve I’.

  14. Rule 2: Abstract program I,P I,P’ • P’ : Atomic statement[ S ] in P replaced with [ S’ ] • Atomic statement [S’] abstracts statement [S]

  15. Abstracting Actions 15 abstracted by   s1 If for all :   s1 s1 1. If error error then   s1 s2 s1 s2 2. If then  s1 or error • Going wrong more often is sound for assertion checking

  16. Flavors of Abstraction 16 Adding non-determinism if (x == 1) y := y + 1; if (*) y := y + 1; t := x; havoc t; assume x != t; skip; Addingassertions assert (lock_owner == tid); x := t + 1; x := t + 1;

  17. Rule 3: Reduce program I,P I,P’ [ S1 ]; [ S2 ] [ S1 ] || [ S2 ] [ S1; S2] [ S1 ; S2 ]

  18. x release S1 S2 S3 release x S1 T2 S3 acquire y S1 S2 S3 y acquire S1 T2 S3 Right and left movers (Lipton 1975)

  19. Static mover check  ...... ...... For every action  in program: (run by different thread) ...... .......  ...... ...... ......   ...... ....... ...... ...... ...... ....... Right mover: Commutes to the right of any other action run by a different thread Static right-mover check for :

  20. Static mover check   • Static right-mover check between and : • Simple cases • Mover check passes: •  and  access different variables •  and  disable each other • Fails: •  writes to a variable and  reads it •  and  both write to a variable, writes do not commute

  21. Reduction  ;  ;  right-mover: ... 12  ... n ... For each execution: Exist equivalent executions: ... 1 2  ... n ... ........... ... 1 2  ...   n ... ... 12  ...  n ;  ...

  22. Static mover check: a subtlety    s1 s1 error Static right-mover check between and : Consider such that No execution reaching s1 executes  followed by  Do not need to do mover check for state pairs starting with s1

  23. Increment: Proof by reduction 23 acquire (lock); t1:= x; t1:= t1+ 1; x := t1; release(lock); acquire (lock); t1:= x; t1:= t1+ 1; x := t1; release(lock); R B REDUCE-SEQUENTIAL B B L

  24. Static mover check fails: Apparent conflict 24 • Static mover check is local, fails! • Individual actions do not locally contain the information: • “Whenever this action executes, this thread holds the lock” • Annotate action with local assertion: • Express belief about non-interference acquire (lock); t1 := x; t1 := t1 + 1; x := t1; release(lock); acquire (lock); t2 := x; t2 := t2 + 1; x := t2; release(lock);

  25. Auxiliary variable: Which thread holds the lock? 25 • Auxiliary variable a is a history variable • Summarizes relevant part of execution history New invariant: (lock == true) (a != 0) inc (): acquire (lock); a := tid; t2= x; t2= t2+ 1 x = t2; release(lock); a := 0; inc (): acquire (lock); t1= x; t1= t1+ 1 x = t1; release(lock); AUX-ANNOTATE

  26. Annotating Actions with Assertions 26 Invariant: (lock == true) (a != 0) acquire (lock); a := tid; assert a == tid; t1= x; t1= t1+ 1 assert a == tid; x = t1; assert a == tid;release(lock); a := 0; acquire (lock); a := tid; t1= x; t1= t1+ 1 x = t1; release(lock); a := 0; ABSTRACT • Assertions indicate belief about non interference • Annotate actions locally with global information about execution

  27. 27 History Variable Annotations Make Static Mover Check Pass Thread 1 acquire (lock); a := tid1; assert a == tid1; t1 := x; t1 := t1 + 1 assert a == tid1; x := t1; assert a == tid1;release(lock); a := 0; Thread 2 acquire (lock); a := tid2; assert a == tid2; t2 := x; t2 := t2 + 1 assert a == tid2; x := t2; assert a == tid2;release(lock); a := 0; R B B B L • assert a == tid1; x := t1; andassert a == tid2; x := t2; commute • α  β β α • Because both α  β and β  α result in assertion violations.

  28. Borrowing and paying back assertions 28 Invariant: (lock == true) (a != 0) Discharges the assertions inc (): acquire (lock); a := tid; assert a == tid; t1= x; t1= t1+ 1 assert a == tid; x = t1; assert a == tid;release(lock); a := 0; inc (): acquire (lock); a := tid; assert a == tid;t1= x; t1= t1+ 1 assert a == tid;x = t1; assert a == tid;release(lock); a := 0; R B B B L REDUCE-SEQUENTIAL, DISCHARGE ASSERTIONS

  29. 29 : Example: Ruling out apparent interference • possiblyInList[t] : • False when a newly created node assigned to t. • Set to true when p.next := t for some p. Remains true afterwards. assert possiblyInList[p2]; n2:= p2.next; assert !possiblyInList[t1]; t1.next := n1;  assert !possiblyInList[t1]; t1.next := n1; assert possiblyInList[p2]; n2:= p2.next;  • p2 and t1 refer to the same node: • LHS and RHS lead to assertion violations. • Otherwise, no conflict.

  30. 30 Increment with CAS t1 := x; s1 := CAS(x,t1,t1+1); t2 := x; s2 := CAS(x,t2,t2+1); || havoc t1; s1 := CAS(x,t1,t1+1); [ if (*) { s1:=false; } else { x:=x+1; s1:= true; } ]

  31. QED-verified examples • Fine-grained locking • Linked-list with hand-over-hand locking [Herlihy-Shavit 08] • Two-lock queue [Michael-Scott 96] • Non-blocking algorithms • Bakery [Lamport 74] • Non-blocking stack [Treiber86] • Obstruction-free deque [Herlihy et al. 03] • Non-blocking stack [Michael 04] • Writer mode of non-blocking readers/writer lock [Krieger et al. 93] • Non-blocking queue [Michael-Scott 96] • Synchronous queue [Scherer-Lea-Scott 06]

  32. Outline • QED proof system • QED overview • Half-baked part: • Backward reasoning in time • Prophecy variables, “tressa” annotations

  33. Static Reduction Proofs and the Future • Challenge in some QED proofs • Different reduction proof needed for different execution futures • Example: Optimistic concurrency • Proceed assuming non-interference • Abort, undo and/or retry if interference detected • Prophecy variables and backwards reasoning in QED

  34. Example: Funny Set procedure Insert(x: data) returns success: bool; { havoc i; assume 0<=i<n; // Start from arbitrary // array slot cnt := 0; success := false; while ( cnt<n && !success) { if (q[i]==-1) { q[i] := x; success := true; } else if (q[i]== x) { success := true; } else { i := (i+1) mod n; cnt := cnt+1; } } }

  35. Set Lookup procedure Lookup(x: data) returns found: bool; { found := false; i := 0; while (i<n && !found) { found := (q[i] == x); i := i+1; } return found; }

  36. Set Lookup procedure Lookup(x: data) returns found: bool; { [ found := false; i := 0; ] while (*) { [ assume(i<n && !found); found := (q[i] == x); i := i+1; ] } [ return found;] } chk(i,x):

  37. Case 1: Lookup returns false chk(0,x) chk(1,x) chk(2,x) . . . chk(k-1,x) . . . chk(n-1,x) • chk(i,x): no x in slot q[i] • Intuition: • chk(i,x)should be a left mover for all i. • q[i]:= x may come after chk(i,x) • So, chk(i,x) not a right mover. • No q[i]:= x can come before chk(i,x) • Looks like a left mover.

  38. Case 2: Lookup returns true chk(0,x) chk(1,x) chk(2,x) . . . chk(k-1,x) q[k] := x chk(k,x) • chk(k,x): Slot q[i] has x • Cannot be a left-mover • Does not commute to the left ofq[i]:= x • Is a right mover (no deletes) • Need all earlier chk(i,x)to be right movers. • Dilemma: • What is the mover type of chk(i,x) ?

  39. Code Duplication procedure Lookup(x: data) returns found: bool; { [ found := false; i := 0; ] while (*) { chkL(i,x);// Left // mover } assume !found; [ return found;] } { [ found := false; i := 0; ] while (*) { chkR(i,x);// Right • // mover } assume found; [ return found;] } ☐

  40. Failing Lookup: chk(i,x) chk(i,x) • From an initial state with q[i] == -1 and y == x • LHS yields found == true • RHS yields found == false • chkL(i,x):is not a left mover. • Annotate chkL to say “I am part of a failing execution of Lookup.” found := (q[i] == x); assume(q[i]==-1); q[i]:= y; then is not simulated by found := (q[i] == x); assume(q[i]==-1); q[i]:= y; then chk(i,x) found := (q[i] == x);

  41. Failing Lookup: chk(i,x) { [ found := false; i := 0; ] while (*) { chkL(i,x);// Left // mover }assume !found; [ return found;] } • We would like to say “This copy of the action only occurs in executions in which found is false.” • [chk(i,x); assert !found;]does not work. • Cannot discharge assertion. • Prefix of execution does not guarantee !found.

  42. tressa: Temporal dual of assert { [ found := false; i := 0; ] while (*) { chkL(i,x);// Left // mover }assume !found; [ return found;] } • Postfix of execution justifies!found • Code split has produced artificial executions that block when they get to assume!found • Annotate chkL(i,x)to say: • “Unless !found is true, this is an artificial execution that blocks before Lookup returns.” • [ chkL(i,x); • tressa(!found);]

  43. tressa(pictorial) semantics • Tressa violation in this execution fragment if (s) is false. All future events  refers to have happened. [ …tressa ] … … … s

  44. 44 Abstraction and Mover Checks with tressa’s [assert a1; τ1; tressa p1] [assert a2; τ2; tressa p2] • Preserve assert violations: a2 a1 • Preserve tressa violations: p2 p1 • Forward simulate or replace with assert violation: τ1 (s,s’) τ2 (s,s’) ∨ a2(s) • Backward simulate orreplace with tressa violation: τ1 (s,s’) τ2 (s,s’) ∨ p2(s’) • Does α commute to the right of β ? α  β β α

  45. Failing Lookup { [ found := false; i := 0; ] while (*) { [ chkL(i,x); tressa(!found);] }assume !found; [ return found;] } { [ found := false; i := 0; ] while (*) { chkL(i,x);// Left // mover }assume !found; [ return found;] }

  46. Failing Lookup • In mover checks, can ignore scenarios where, on the LHSthe tressa is violated • Only worry about (s1,s3) if s3 satisfies !found • Then x != y • Simulation holds! found :=(q[i] == x);tressa !found q[i]:= y; then found :=(q[i] == x);tressa !found simulated by then q[i]:= y; found :=(q[i] == x);tressa !found q[i]:= y; s1 s2 s3

  47. Discharging tressa’s: Backwards Reasoning while (*) { [ chkL(i,x); tressa(!found);] } assume !found; • found := • \Exists i: 0<=i<n && q[i]==x; • tressa(!found); • assume !found; • tressa’s discharged by backwards reasoning within an atomic block.

  48. Discharging tressa’s: Backwards Reasoning • assume’s are like assignments in the reverse direction in time: • [havoc x; x := 2;] • A set of transitions (x, x’) where x is arbitrary and x’ is 2. • [assume x == 2; havoc x;] • A set of transitions (x, x’) where x’ is arbitrary and x is 2. • Denoted x =: 2 • “Reverse assignment”

  49. Prophecy Variables acquire (lock); p =: 0 t = x; tressa p == tid; t = t + 1 x = t; tressa p == tid; release(lock); p =: tid; acquire (lock); p =: 0 t = x; t = t + 1 x = t; release(lock); p =: tid; R B B B L

  50. Prophecy variables and tressas acquire (lock); p =: 0 t = x; tressa p == tid; t = t + 1 x = t; tressa p == tid; release(lock); p =: tid; acquire (lock); p =: 0; t = x; tressa p == tid; t = t + 1 x = t; tressa p == tid; release(lock); p =: tid; R B B B L

More Related