1 / 51

Refinement Verification of Concurrent Programs and Its Applications

Refinement Verification of Concurrent Programs and Its Applications. Hongjin Liang Univ. of Science and Technology of China Advisors: Xinyu Feng and Zhong Shao. Refinement. void main() { print a square; }. void main() { print a rectangle; }. .

dyanne
Download Presentation

Refinement Verification of Concurrent Programs and Its Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Refinement Verification of Concurrent Programs and Its Applications Hongjin Liang Univ. of Science and Technology of China Advisors: XinyuFeng and ZhongShao

  2. Refinement void main() { print a square; } void main() { print a rectangle; }  T  S: Thas no more observable behaviors (e.g. I/O events by print) than S.

  3. Concurrent Program Refinement • Compilers for concurrent programs S Multithreaded Java programs Correct(Compiler): S, T. T = Compiler(S)  T  S Compiler T Java bytecode

  4. Concurrent Program Refinement • Compilers for concurrent programs • Fine-grained impl. of concurrent objects (libraries) • E.g. java.util.concurrent

  5. Concurrent object O Client code C … • … • push(7); • x = pop(); • … • … • push(6); • … void push(int v) { local b:=false, x, t; x := new Node(v); while (!b) { t := top; x.next = t; b = cas(&top, t, x); } } • intpop() { • … • … • } • Whole program C[O] How to specify/prove correctness?

  6. Correctness of Concurrent Objects • Linearizability[Herlihy&Wing’90] • OlinS :correctness w.r.t. functionality • Spec S : abstract object (atomic methods) • Hard to understand/use • Equivalent to contextual refinement[Filipovic et al.] • OctxtSiffC. C[O]  C[S]

  7. OctxtSiffC. C[O]  C[S] void push(int v) { … } int pop() { … } Concrete obj. O Client C • … • x := 7; • push( x ); • … • … • y := pop(); • print(y); • … • push Abstract obj. S • pop

  8. Concurrent Program Refinement • Compilers for concurrent programs • Linearizability of concurrent objects (libraries) • Impl. of software transactional memory (STM) • Atomic block (transaction)  fine-grained impl.

  9. Concurrent Program Refinement • Compilers for concurrent programs • Linearizabilityof concurrent objects (libraries) • Impl. of software transactional memory (STM) • Impl. of concurrent garbage collectors (GC) • Impl. of operating system (OS) kernels Is such a refinement T  S general enough & easy to verify?

  10. Problems with T  S T1  S1 T2  S2  (Compositionality)  T1 || T2 S1 || S2 Existing work on verifying T  S : either is not compositional, or limits applications.

  11. Long-Standing Problems in Verifying Linearizability • Objects with Non-Fixed Linearization Points (LPs) • Future-dependent LPs(e.g. lazy set, pair snapshot) • Helping (e.g. HSY elimination-backoff stack) Most existing work : either not supports them, or lacks formal soundness.

  12. Refinement vs. Progress Properties ? • Linearizability • Correctness w.r.t. functionality • Not talk about termination/liveness properties • Progress properties • Lock-freedom (LF) • Wait-freedom (WF) • Obstruction-freedom (OF) • Deadlock-freedom (DF) • Starvation-freedom (SF) Non-blocking impl. Lock-based impl.

  13. Our Contributions (Part 1) • RGSim = Rely/Guarantee + Simulation • Compositional w.r.t. parallel composition • Flexible & applicable • optimizations in concurrent contexts • concurrent GC • fine-grained concurrent obj. • …

  14. Our Contributions (Part 2) • RGSim = Rely/Guarantee + Simulation • A program logic for linearizability • Support non-fixed LPs • Verified 12 well-known algorithms (some are used in java.util.concurrent) • Light instrumentation mechanism to help verification • Formal meta-theory: simulation (extends RGSim) • Establish a contextual refinement

  15. Our Contributions (Part 3) • RGSim = Rely/Guarantee + Simulation • A program logic for linearizability • A framework to characterize progress properties via contextual refinement (CR) • Propose different termination-sensitive CR • Equivalent to linearizability + progress • Unify all five progress properties (LF, WF, OF, DF, SF) • Make modular verification of whole program C[O] easier • Potential to have a generic verification framework for linearizability + progress

  16. Outline • Rely-Guarantee-based simulation for modular verification of concurrent refinement • Logic for linearizability • Progress properties and contextual refinement

  17. Modular Verification of T  S T1  S1 T2  S2  (Compositionality)  T1 || T2 S1 || S2

  18. is NOT compositional w.r.t. parallel composition: T1 S1 T2 S2 T: local t; t = x; x = t + 1; print( x ); S: x++; print( x ); T1 ||T2  S1 ||S2 We have T  S, since output(T)  output(S) ; but we do not have T||T  S||S .

  19. Existing Proof Methods: Simulation in CompCert [Leroy et al.] • Source state e * * … (S, ) (S’, ’) (S’’, ’’) • observable event (e.g. I/O)    • Target state e (T’, ’) (T’’, ’’) (T, ) …

  20. Simulation in CompCert[Leroy et al.]  Can verify refinement of sequential programs  NOT compositional w.r.t. parallel composition T: local t; t = x; x = t + 1; print( x ); S: x++; print( x ); We haveT  S , but notT||TS||S

  21. Simulation in CompCert[Leroy et al.]  Can verify refinement of sequential programs  NOT compositional w.r.t. parallel composition • Consider NO environments • Simulation in process calculus (e.g. CCS [Milner et al.]) • Assume arbitrary environments •  Compositional •  Too strong: limited applications

  22. Assuming Arbitrary Environments (S, ) (S’, ’) e * ’ ’ * env … (S’, ’’) (S’’, ’’’) (T’, ’) (T, ) ’ ’ e (T’’, ’’’) env … (T’, ’’) • Too strong to be satisfied, since env. can be arbitrarily bad. • Refinement applications have assumptions • about S & env.

  23. Refinement’s Assumptions • Compilers for concurrent programs • Prog. with data races has no semantics (e.g. concurrent C++) • Not guarantee correctness for racy programs • Fine-grained objects • Accesses use same primitives (e.g. stack: push & pop) • Not guarantee correctness when env. can destroy obj. • More examples are in the thesis … [Boehm et al. PLDI’08] Env. of a thread cannot be arbitrarily bad !

  24. Problems of existing simulations : • Considers noenv. in CompCert[Leroy et al.]  NOT compositional w.r.t. parallel composition • Assumes arbitraryenv. in process calculus (e.g. [Milner et al.])  Too strong: limited applications Our RGSim : • Parameterized with the interference with env. •  Compositional • More applications • Use rely/guarantee to specify the interference

  25. Overview of Rely/Guarantee [Jones'83] • r: acceptable environment transitions • g: state transitions made by the thread y = y’ x = x’ y = y’ x = x’     ’ ’ ’ ’ Thread1 Thread2 • g2: • g1: • r2: • r1: Nobody else would update y Nobody else would update x I guarantee I would not touch x I guarantee I would not touch y Compatibility (Interference Constraints): • g2  r1 and g1  r2

  26. RGSim = Rely/Guarantee + Simulation * (S’, ’’) R … (S, ) (S’, ’) (S’’, ’’’) e * * G G ≲ ≲ ≲ ≲ (T’, ’) (T’, ’’) r … e (T, ) g g (T’’, ’’’) (T, r, g) ≲ (S, R, G)

  27. Soundness Theorem If we can find r, g, R and G such that • (T, r, g) ≲ (S, R, G) then we have: T  S

  28. Parallel Compositionality (T1, r1, g1) ≲ (S1, R1, G1) (T2, r2, g2) ≲ (S2, R2, G2) g1  r2 g2  r1 G1  R2 G2  R1 (PAR) (T1||T2, r1r2, g1g2) ≲ (S1||S2, R1R2, G1G2)

  29. More on Compositionality (T1, r, g) ≲ (S1, R, G) (T2, r, g) ≲ (S2, R, G) (T1;T2, r, g) ≲ (S1;S2, R, G) (T, r, g) ≲ (S, R, G) b  B (while b doT, r, g) ≲(while B doS, R, G) … An axiomatic proof system for refinement

  30. We have applied RGSim to verify … • Optimizations in parallel contexts • Loop invariant hoisting, strength reduction and induction variable elimination, dead code elimination, … • Fine-grained impl. & concurrent objects • Lock-coupling list, counters, Treiber’s non-blocking stack, concurrent GCD algorithm, … • Concurrent garbage collectors • A general GC verification framework • Hans Boehm’s concurrent GC [Boehm et al. 91]

  31. Outline • Rely-Guarantee-based simulation for modular verification of concurrent refinement • Logic for linearizability • Progress properties and contextual refinement

  32. Linearizability of Concurrent Objects [Herlihy&Wing’90] • Correctness w.r.t. functionality • OlinS : Every concurrentexecution of object O is “equivalent” to some sequential execution of spec S

  33. Linearizability of Object O A concurrent execution of O: push(7) ret pop() ret (7) Thread 1: Linearization point (LP) push(6) ret Thread 2: Sequential execution of S time push(6), ret, push(7), ret, pop(), ret(7)

  34. t x next Example: Treiber’s Non-Blocking Stack [Treiber’86] Top next next v1 vk … Is it linearizable? v 4 while(!b){ 5 t := Top; 6 x.next := t; 7 b := cas(&Top, t, x); 8 } push(int v): 1 local b:=false, x, t; 2 x := new Node(); 3 x.data := v;

  35. ? lin Treiber’s stack O Abstract stack S Stk = v1 :: v2 :: … :: vk Top next next … v1 vk push(v): 1 local b:=false, x, t; 2 x := new Node(v); 3 while (!b) { 4 t := top; 5x.next = t; 6 b = cas(&top, t, x); 7 } PUSH(v): Stk := v::Stk; Not update the shared list “Fixed”: statically located in impl code Line 6: the only command that changes the list LP

  36. ? lin Treiber’s stack O Abstract stack S Top next next … Stk = v :: v1 :: v2 :: … :: vk v1 vk v Abstract opr PUSH(v) not done Proved it’s LP LP • 6 b := cas(&Top, t, x); • if (b) • linself; • > • 7 } < • push(v): Execute abstract opr simultaneously 1 local b:=false, x, t; 2 x := new Node(v); 3 while (!b) { 4 t := Top; 5x.next := t; Atomic block - { [PUSH(v)] … } • { [] … } Abstract opr is done

  37. Basic Approach to Verify OlinS Inspired by [Vafeiadis’ Thesis] • Instrument(O) = D with linself at LPs • Verify D in program logic with rules for linself • New assertions [S] and [] • Ensure O’s LP step corresp. to S’s single step  Not support non-fixed LPs • Future-dependent LPs • Helping

  38. Challenge 1: Future-Dependent LP Example: Pair Snapshot [Qadeer et al. MSR-TR-2009-142] t2: write(i, d) t1: readPair(i, j) m 0 1 … k write(i, d) updates m[i] to a new value d readPair(i, j) intends to return snapshot of m[i] and m[j]

  39. Pair Snapshot [Qadeer et al. MSR-TR-2009-142] • readPair(inti, j){ • 1 local s:=false, a, b, v, w; • 2 while (!s) { • 3 <a := m[i].d; v := m[i].v>; • 4 <b := m[j].d; w := m[j].v>; • 5 <if (v = m[i].v) s := true>; • 6 } • 7 return (a, b); • } write(inti, d){ 8 <m[i].d := d; m[i].v++>; } LP if line 5 succeeds m 0 1 … k •  Future-dependent LP • Not supported by linself d know: m[i] = (a,v) at line 4 v version number Where is the LP ? • Line 4? But line 5 may fail, m[i] and m[j] may be re-read

  40. Solution: Try-Commit • readPair(inti, j){ • 1 local s:=false, a, b, v, w; • 2 while (!s) { • 3 <a := m[i].d; v := m[i].v;> • 4 <b := m[j].d; w := m[j].v; • 5 <if (v = m[i].v) { s:= true; • 6 } • 7 return (a, b); • } speculateat potential LP, keep both result and original - { m[i] = (a, v) [RP, (i,j)]  … } [RP, (i,j)] > trylinself; - { m[i] = (a, v)  ( [, (a,b)] )  … }  } > commit( [, (a,b)] ); - { s [, (a,b)]  s …}

  41. Challenge 2: Helping • Example: elimination-backoff stack [Hendler et al. SPAA’04] • t1 finishes t2’s opr t2’s LP is in the code of t1 • Need to linearize a thread other than self • New auxiliary command: lin(t) • New assertions: t  S | t   • Details are in the thesis…

  42. Our Approach to Verify OlinS • Instrument(O) = D with auxiliary cmds at LPs • linself for fixed LPs • try-commit for future-dependent LPs • lin(t) for helping • Assertions to describe abstract code & states p, q ::= … | t  S | t   | p  q | p  q • Verify D in our program logic • Extend an existing logic with rules for aux cmds

  43. Our Logic for OlinS ┝ {p} S {q} ┝ {p} S {q} … ┝ {p  (t  S)} lin(t) {q * (t )} ┝ {p  (cid S)} trylinself{( p * (cid S) ) ( q * (cid) )} ┝ {p  q} commit(p) {p} More rules and soundness are in the thesis

  44. Verified Algorithms

  45. Soundness via Contextual Refinement Theorem (equivalence): •  • O ctxtS O linS Proof follows [Filipovic et al., 2009] • Intentional • Extensional • “”: all proof methods for ctxtcan verifylin • ctxtis a well-studied concept in PL community(still challenging though) • “”: modular verification (view C[O] as C[S]) • C[S] is simpler to understand/verify

  46. Outline • Rely-Guarantee-based simulation for modular verification of concurrent refinement • Logic for linearizability • Progress properties and contextual refinement

  47. Progress Properties • Describe whether methods eventually return • Defined similarly to linearizablity • Describe objects’ behaviors instead of clients’ • Intentional instead of extensional • E.g. there always exists a method call that’ll return Can we use contextual refinement to define progress properties?

  48. Our Results Linearizability OlinS Progress P(O)  •  Termination-sensitive contextual refinement OP S ( iffC. ObsBeh(C[O]) ObsBeh(C[S]))

  49. Relationships between Progress Properties WF Wait-freedom LF Lock-freedom SF Starvation-freedom equiv. to Obstruction-freedom OF Deadlock-freedom DF + Linearizability

  50. Conclusion • RGSim= Rely/Guarantee + Simulation • Idea: parameterized with interference with env. • Compositional! • Applications: optimizations, concurrent GC, … • Program logic for linearizability • Light instrumentation to help verification • linself for fixed LPs • lin(t) for helping • try-commit for future-dependent LPs • Verified 12 well-known algorithms

More Related