Oracle Semantics

1. Oracle Semantics Aquinas Hobor

2. Lipton, Perlis, DeMillo  1977 Why Proofs about Code are Hard Math proofs tend to have the form If A, then B

3. Lipton, Perlis, DeMillo  1977 Why Proofs about Code are Hard Math proofs tend to have the form If A, then B Proofs about real programs have the form If A1 and A2 and A3 and A4 and A5…

4. Lipton, Perlis, DeMillo  1977 Why Proofs about Code are Hard Math proofs tend to have the form If A, then B Proofs about real programs have the form If A1 and A2 and A3 and A4 and A5, then B1 and B2 and B3

5. Lipton, Perlis, DeMillo  1977 Why Proofs about Code are Hard Math proofs tend to have the form If A, then B Proofs about real programs have the form If A1 and A2 and A3 and A4 and A5, then B1 and B2 and B3, except when C1 or C2 or C3 or C4

6. Lipton, Perlis, DeMillo  1977 Why Proofs about Code are Hard Math proofs tend to have the form If A, then B Proofs about real programs have the form If A1 and A2 and A3 and A4 and A5, then B1 and B2 and B3, except when C1 or C2 or C3 or C4, assuming D1 and (D2 or D3)

7. Why Proofs about Code are Hard Two choices: A) Prove something on paper about a simplification of the core ideas / algorithm Problem: actual implementation will have bugs B) Prove something about the actual code Problem: too hard to do by hand

8. Machine-Checked Proofs Solution: Use computer to check your proofs (we use Coq) Ideally the computer can help prove as well Issue: Machine-checked proofs are large and very time-consuming to write Proof engineering becomes a real issue

9. Goal for Oracle Semantics We want to add concurrency to large, realistic systems in a provably correct, modular way Those systems may already exist in a sequential form; we would like to re-use existing code and machine-checked proofs wherever possible The key is to isolate the sequential and concurrent reasoning from each other.

10. Leroy, 2006 CompCert Project Sequential Source Program (C minor) Sequential C minor Operational Semantics Compiler Correctness Proof CompCert Compiler Sequential PowerPC Operational Semantics Sequential Target Program (Power PC) Sequential Translation Correctness Guarantee User Leroy

11. Leroy, 2006 CompCert Project Sequential Source Program (C minor) Sequential C minor Operational Semantics What is relationship here? Compiler Correctness Proof CompCert Compiler Sequential PowerPC Operational Semantics Sequential Target Program (Power PC) Sequential Translation Correctness Guarantee User Leroy

12. Appel & Blazy, 2007 • Separation Logic for C minor Sequential Source Program (C minor) Axiomatic Semantics (Separation Logic) Sequential C minor Operational Semantics Program Verification Soundness Proof Compiler Correctness Proof CompCert Compiler Sequential PowerPC Operational Semantics Sequential Target Program (Power PC) Sequential Translation Correctness Guarantee User Leroy Appel & Blazy

13. Changes Required for Concurrency Concurrent Source Program (C minor) Axiomatic Semantics (Concurrent Separation Logic) Concurrent C minor Operational Semantics Program Verification Soundness Proof Concurrency-Aware Compiler Correctness Proof Concurrency-Aware CompCert Compiler Concurrent PowerPC Operational Semantics Concurrent Target Program (Power PC) Concurrent Translation Correctness Guarantee User Future Work This work (Hobor, Appel, & Zappa Nardelli)

14. What this would mean We would have machine-checked end-to-end correctness proofs of concurrent programs Since everything would be machine-checked, we would have very high assurance that the actual code running on the hardware had the correct, expected behavior.

15. Changes Required for Concurrency Concurrent Source Program (C minor) Axiomatic Semantics (Concurrent Separation Logic) Concurrent C minor Operational Semantics Program Verification Soundness Proof Concurrency-Aware Compiler Correctness Proof Concurrency-Aware CompCert Compiler Concurrent PowerPC Operational Semantics Concurrent Target Program (Power PC) Concurrent Translation Correctness Guarantee User Future Work This work (Hobor, Appel, & Zappa Nardelli)

16. Additions to C minor language We add five new statements: lock elock a lock unlock eunlock a lock makelocke Rturn an address into a lock freelockerestore an address fork e el start a new thread

17. Additions to C minor language We add five new statements: lock elock a lock unlock eunlock a lock makelocke R turn an address into a lock freelockerestore an address fork e el start a new thread Resource Invariant – Describes what resources a thread acquires on lock and relinquishes on unlock

18. Example Program [l] := 0; makelock l (∃y. x↦y+y); [x] := 0; unlock l; fork child(l); lock l; [x] := [x] + 1; [x] := [x] + 1; unlock l;

19. Modularity Principle Sequential Features Concurrent Features s1 ; s2 if e then s1 else s2 while e do s break n (x1, …, xn) := e (e1, …, em) return (e1, …, en) x := e [e1] := e2 Skip … lock e unlock e fork e (e1, …, en) makelock e R freelock e …

20. Modularity Principle Sequential Features Concurrent Features s1 ; s2 if e then s1 else s2 while e do s break n (x1, …, xn) := e (e1, …, em) return (e1, …, en) x := e [e1] := e2 Skip … lock e unlock e fork e (e1, …, en) makelock e R freelock e … Connection Easy – Just Syntax

21. Changes Required for Concurrency Concurrent Source Program (C minor) Axiomatic Semantics (Concurrent Separation Logic) Concurrent C minor Operational Semantics Program Verification Soundness Proof Concurrency-Aware Compiler Correctness Proof Concurrency-Aware CompCert Compiler Concurrent PowerPC Operational Semantics Concurrent Target Program (Power PC) Concurrent Translation Correctness Guarantee User Future Work This work (Hobor, Appel, & Zappa Nardelli)

22. Modularity Principle Sequential Reasoning Concurrent Reasoning Sequential Separation Logic Soundness Sequential Compiler Correctness Type Systems … Concurrent Separation Logic Soundness Concurrent Compiler Correctness Synchronization …

23. Modularity Principle Sequential Reasoning Concurrent Reasoning Sequential Separation Logic Soundness Sequential Compiler Correctness Type Systems … Concurrent Separation Logic Soundness Concurrent Compiler Correctness Synchronization … Insert Magic Here

24. How is it done? Key: only consider well-synchronized programs • Mutual exclusion via Dijkstra Semaphores • Data-race free Our operational semantics gets stuck on ill-synchronized programs

25. Changes to Pure Sequential Semantics C minor small-step sequential semantics step relation: (1, 1) ↦ (2, 2)  = (ρ, m)  is a control stack Resource map-aware C minor sequential semantics step relation: (1, 1) ↦ (2, 2)  = (ρ, , m)  is a map from addresses to resources

26. Resources Basic resources: | NONE | VAL (data) | LOCK of invariant (invariant comes from makelock) Get stuck if you use memory without ownership Thread resource maps are disjoint – at any time, only one thread can access each address Sequential instructions and proofs ignore LOCK

27. Concurrent Operational Semantics Concurrent C minor step relation (1, K1, m1, G1) ⇒ (2, K2, m2, G2)  is scheduler K is thread list (ρ1, 1, 1) :: (ρ2, 2, 2) :: … m is memory G is global resource map – owner of unlocked locks To execute a sequential instruction, use the resource map-aware sequential step relation

28. Concurrent Instructions • The concurrent step executes concurrent instructions directly • Updates memory at the lock instruction • Maintains thread list • Transfers resources between threads and global pool • unlock: resources transferred to global pool • lock: resources transferred from global pool

29. Space Thread A Thread B Global Resource Map 1  R 2 3 Time Memory unlock 𝓁 2 1  R 3 Memory lock 𝓁 1  R 2 3 Memory R is the unique resource map that satisfies l’’s lock invariant

30. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Does not prevent compilation! Unlock

31. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Does not prevent compilation! Unlock

32. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Does not prevent compilation! Unlock

33. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Does not prevent compilation! Unlock

34. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Operational semantics is nonconstructive! Unlock

35. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Operational semantics is nonconstructive! Unlock

36. Avoiding Race Conditions in Semantics (ρ, lock, m) ⊩l⇝ P (ρ, lock, m) ⊩ ⊳P m(l) = 0 m’ = [l↦ 1] m ’ lock =  G lock = G’ Ki = (ρ, , (unlock l) • ) K’ = [i -> (ρ, ’, )] K (i :: , K, m, G) ⇒ (, K’, m’, G’) Does not prevent compilation! Unlock

37. Coroutine Interleaving Executes sequentially without interleaving Interleave when we get to concurrent operation Key: we have well-synchronized programs

38. Reasoning About Concurrency Most of the time, concurrent programs are executing sequential code Proofs about sequential features are hard enough We don’t want to add the extra complexity of concurrency when reasoning about sequential code Idea: why can’t we just pretend it is sequential?

39. Sequential Reasoning x := x + 1 [y] := 2 lock l a := [x]

40. Sequential Reasoning (1, 1) • (2, 2) • (3, 3) STUCK x := x + 1 [y] := 2 lock l a := [x] ↦ ↦

41. Oracular Reasoning (1, 1) • (2, 2) • (3, 3) STUCK (o1, 1, 1) • (o1, 2, 2) • (o1, 3, 3) • (o2, 4, 4) • (o2, 5, 5) x := x + 1 [y] := 2 lock l a := [x] ↦ ↦ ↦ ↦ ↦ ↦

42. Oracular Composition An oracle o is (, K, G)  is the scheduler K is the other threads G is the global resource map The oracle simulates running all of the other threads until the scheduler returns control to this thread

43. Soundness of Oracular Reasoning Theorem (connection of oracle and concurrent semantics) If a thread executes in a certain way on the oracular machine, then it executes in the same way when executing in the concurrent machine

44. Changes Required for Concurrency Concurrent Source Program (C minor) Axiomatic Semantics (Concurrent Separation Logic) Concurrent C minor Operational Semantics Program Verification Soundness Proof Concurrency-Aware Compiler Correctness Proof Concurrency-Aware CompCert Compiler Concurrent PowerPC Operational Semantics Concurrent Target Program (Power PC) Concurrent Translation Correctness Guarantee User Future Work This work (Hobor, Appel, & Zappa Nardelli)

45. Floyd, Hoare Hoare Logic A Hoare triple is {precondition} command {postcondition} Example: {x = 2} x++ {x = 3} In our setting, the precondition and postcondition are predicates on state (locals, memory, resource map, …) In Coq: Definition predicate : Type := state -> Prop. A Hoare logic is a set of axioms for deriving valid triples Example: {P} c1 {Q} {Q} c2 {R} {P} c1;c2 {R}

46. Reynolds, O’Hearn Separation Logic Problem: handling pointers Example: {x ↦ 0 ⋀ y ↦ 0} [x] := 1 {???} {x ↦ 1 ⋀ y ↦ 0} {x ↦ 1 ⋀ y ↦ 1} Solution: Separation Use “∗” to split propositions into two disjoint halves Example: {x ↦ 0 ∗ y ↦ 0} [x] := 1 {x ↦ 1 ∗ y ↦ 0} Enables more local reasoning for programs with pointers Key rule (Frame): {P} c {Q} {P ∗ F} c {Q ∗ F}

47. CSL 1.0 by O’Hearn, 2006 Concurrent Separation Logic 2.0 Extension of separation logic to handle concurrency Includes all of typical the rules of separation logic Associate with each lock an invariant R l⇝ R ≡ l is a lock with invariant R Rules for concurrent operations {l⇝ R} lock l {(l⇝ R) * R} {(l⇝ R) * R} unlock l {l⇝ R} Programs proved in CSL are well-synchronized! Separation Logic Concurrent Separation Logic

48. Space Thread A Thread B Thread C FB FA 𝓁⇝ R R Time 1  R 2 3 Memory lock 𝓁 unlock 𝓁 FA 𝓁⇝ R R FB 1  R 2 3 Memory {𝓁 ⇝ R} lock 𝓁 {(𝓁 ⇝ R) ∗ R} {FA∗ (𝓁 ⇝ R)} lock 𝓁 {FA∗ (𝓁 ⇝ R) ∗ R} (lock rule) (frame rule)

49. Changes Required for Concurrency Concurrent Source Program (C minor) Axiomatic Semantics (Concurrent Separation Logic) Concurrent C minor Operational Semantics Program Verification Soundness Proof Concurrency-Aware Compiler Correctness Proof Concurrency-Aware CompCert Compiler Concurrent PowerPC Operational Semantics Concurrent Target Program (Power PC) Concurrent Translation Correctness Guarantee User Future Work This work (Hobor, Appel, & Zappa Nardelli)

50. Verification of Example Program [l] := 0; makelock l (∃y. x↦y+y); [x] := 0; unlock l; fork child(l); … lock l; [x] := [x] + 1; [x] := [x] + 1; unlock l;