Satisfiability-based Program Reasoning and Synthesis

1. Satisfiability-based Program Reasoning and Synthesis Saurabh Srivastava Programming Languages at UM University of Maryland, College Park

2. phones, mp3 players [2009] Zune termination bug x-ray machines [1980] Race condition in Therac-25 • Software is everywhere cars, planes [2006] 160,000 Prius recalled – software can stall car electric/phone grid [2003] Race condition causes northeast blackout mars rover [2004] Mars RoverSpirit gets stuck in an reboot loop [2000] Mars Polar Lander misinterprets sensor signals, crashes Developers (many man years) Testers (some man years) … correctsoftware,not everywhere

3. phones, mp3 players x-ray machines cars, planes electric/phone grid mars rover Developers (many man years) Testers (some man years) Resource: Computing power Cheap and readily available! … we already have the resources to build better software … we just need to develop theory, techniques, tools to harness it

4. phones, mp3 players x-ray machines cars, planes electric/phone grid mars rover Developers (many man years) Testers (some man years) Use available computing power to (1) mechanically reason about software

5. phones, mp3 players Program reasoning x-ray machines cars, planes electric/phone grid mars rover Developers (many man years) Testers (some man years) man days + some `compute days’ Use available computing power to (1) mechanically reason about software

6. phones, mp3 players Program reasoning x-ray machines cars, planes electric/phone grid mars rover Developers (many man years) Testers (some man years) man days + some `compute days’ Use available computing power to (1) mechanically reason about software (2) automatically generate software

7. phones, mp3 players Program synthesis Program reasoning x-ray machines cars, planes electric/phone grid mars rover Developers (many man years) Testers (some man years) man days + some `compute years’ man days + some `compute days’ Use available computing power to (1) mechanically reason about software (2) automatically generate software

8. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving

9. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving

10. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving

11. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving Program Properties Program reasoning Verification Certificate Program Properties Property Inference Certificate Mechanically generated Given by human

12. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving Program Properties Program reasoning Verification Certificate Program Properties Property Inference Program Properties Verified synthesis Certificate Program synthesis Certificate Mechanically generated Program • Properties Tested synthesis ≈ Certificate Given by human

13. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving

14. We can build powerful program reasoning and program synthesis techniques that leverage the power of satisfiability solving Bring available computing power to these tasks Significant engineering advances in last decade Fast solvers (e.g., Z3) for real life SAT/SMT instances E.g., SMTCOMP: 105 clauses in a few seconds

15. Outline Theoretical foundations Practice / Applications Satisfiability-based Program Analysis and Synthesis (SPANS)[PLDI’08/PLDI’09/POPL’10] • Verification[PLDI’08/VMCAI’09/CAV’09]: • First to robustly reason about quantified invariants/properties • Property inference[PLDI’09]: • Weakest preconditions for termination, functional correctness • Proof-theoretic synthesis[POPL’10]: • Linear arithmetic, all major sorting, dynamic programming Verification Property Inference Synthesis • Path-based Inductive Synthesis (PINS)[in preparation] • Synthesizing paired programs[in preparation]: • Program Inversion • Client-server synthesis Synthesis (testing-based)

16. Part I: Satisfiability-based Program Analysis Satisfiability-based Program Analysis Program Properties Certificate Program Properties Certificate

17. Reasoning about… straight line code: simple • y>5 Intuitively, x := y+1 if y>5 holds before x:=y+1, then we can derive x>6 afterwards • x>6 Mechanically, • Forwardby variable renaming • (Symbolic execution/SSA style) y>5 ∧x’=y+1 ⇒x’>6

18. Reasoning about… straight line code: simple • y>5 Intuitively, x := y+1 if y>5 holds before x:=y+1, then we can derive x>6 afterwards • x>6 Mechanically, • Forwardby variable renaming • (Symbolic execution/SSA style) y>5 ∧x’=y+1 ⇒x’>6 Unknown Notation: Facts: Fentry, Fexit Fentry, Fexit Statements, i.e., Transitions: Tbody Tbody

19. Reasoning about… • straight linecode: simple • acyclic code: doable F0 F2 F1 F3 F’

20. Reasoning about… • straight linecode: simple • acyclic code: doable … • loops: tricky F’’’0 F’0 F’’0 Need: fixed-point F0 / loop summary / loop invariant Key difficulty in automatic program verification: Inferring loop invariants F1

21. Reasoning about… • straight linecode: simple • acyclic code: doable … • loops: tricky F’’’0 F’0 F’’0 Need: fixed-point F0 / loop summary / loop invariant Over the past 40+ years: Automating fixed-points : limited to simple invariants Need a technique that can infer complicated, quantified invariants robustly… F1

22. Satisfiability-based program analyses • reduce local program fragments to clauses, • delegate fixed-point computation to SAT solver

23. Satisfiability-based program analyses • reduce local program fragments to clauses, • delegate fixed-point computation to SAT solver Key facilitator: Invariant templates Conjunctions of (unknown) linear relations Linear Arithmetic F1 ∨F2 ∨F3 Finv ∀F1 ⇒F2 (Unknown) conjunction of predicates Predicate Abstraction

24. Satisfiability-based program analyses • reduce local program fragments to clauses, • delegate fixed-point computation to SAT solver Linear Arithmetic Farkas’ lemma F1 ∨F2 ∨F3 Values for SAT solver Program VCs boolean formula SAT solution F1, F2, F3… Finv∧Texit⇒Fexit i.e, the invariants ∀F1 ⇒F2 Predicate Abstraction Efficient “predicate cover” using SMT solvers

25. Part I: SPANS: Experiments Satisfiability-based program reasoning • Verification • … invariants for assertion checking (safety) Example: Discrete Line Drawing (12,7) (X,Y) Input: 0<Y≤X (10,6) (11,6) (8,5) (9,5) Output: |y-(Y/X)x| ≤ 1/2 (6,4) (7,4) (5,3) (3,2) (4,2) (0,0) (2,1) (1,1) (0,0)

26. Satisfiability-based program reasoning • Verification • … invariants for assertion checking (safety) Example: Discrete Line Drawing 0<Y≤X (12,7) (X,Y) Input: 0<Y≤X v := 2Y-X; y := 0; x := 0; while ( x≤X ) { print (x,y) if ( v<0 ) { v := v + 2Y; x++; } else { v := v + 2(Y-X); y++; x++; } } (10,6) (11,6) |y-(Y/X)x| ≤ 1/2 (8,5) (9,5) Output: |y-(Y/X)x| ≤ 1/2 (6,4) (7,4) (5,3) (3,2) (4,2) (0,0) Using only linear operations: Bresenham’s line drawing algorithm (2,1) (1,1) (0,0)

27. Satisfiability-based program reasoning • Verification • … invariants for assertion checking (safety) Bubble Sort 0<Y≤X 0<Y≤X v = 2(x+1)Y-(2y+1)X 2(Y-X) ≤ v ≤ 2Y Insertion Sort Selection Sort Bresenham’s Merge Sort Quick Sort |y-(Y/X)x| ≤ 1/2 ∀k : 0≤k<n⇒ A[k1] ≤A[k2]

28. Satisfiability-based program reasoning • Verification • … invariants for assertion checking (safety) 0<Y≤X 0<Y≤X v = 2(x+1)Y-(2y+1)X 2(Y-X) ≤ v ≤ 2Y Selection Sort Bresenham’s |y-(Y/X)x| ≤ 1/2 ∀k : 0≤k<n⇒ A[k1] ≤A[k2] first technique that can robustly infer such quantifiedinvariants • i < j∧i ≤ min ∧ min < n ∀k2 : 0≤k2∧i≤k2∧k2<j⇒ A[min] ≤A[k2] ∀k1,k2 : 0≤k1 ∧ k1<k2 ∧k2<n∧ k1<i⇒ A[k1] ≤A[k2]

29. Satisfiability-based program reasoning • Verification (safety) Previous best known ∞ ∞ seconds (log scale) 0. 0.0 0.00 linear arithmetic examples predicate abstraction examples

30. Satisfiability-based program reasoning • Verification (safety) Previous best known seconds (log scale) 0. 0.0 0.00 linear arithmetic examples predicate abstraction examples

31. Satisfiability-based program reasoning • Verification • … invariants for assertion checking (safety) • … loop bound computation • Property Inference • … preconditions for functional correctness • … preconditions for bugs/worst case Precondition: sorted array • … preconditions for termination Binary search over an array • First technique that can infer such quantifiedproperties Selection Sort

32. We now know how to ... • “build powerful program reasoning techniques using satisfiability solving” Core insight : Use SAT/SMT for fixed-point computation Key detail : Abstraction-specific reductions needed

33. Part II: Proof-Theoretic Synthesis Proof-theoretic Synthesis Program Properties Certificate

34. … automatically generate programs for a given specification • Program Synthesis … connected to program reasoning Finv∧ Texit⇒ Fexit Finv∧ x’=y+1⇒ x’=n+1 Finv∧ Texit⇒ Fexit statements/guards don’t look all that special! • express synthesis as generalized verification … generate certificate of correctness alongside

35. Example: Discrete Line Drawing (X,Y) (12,7) (10,6) (11,6) (8,5) (9,5) (0,0) (6,4) (7,4) (5,3) Input: 0<Y≤X (3,2) (4,2) (0,0) using only linear operations Output: |y-(Y/X)x|≤1/2 (2,1) (1,1)

36. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers

37. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers Synthesis input : Scaffold … (1) functional specification Pre- and post-conditions: e.g., pre: 0<Y≤X post: |y-(Y/X)x| ≤ 1/2 What is the program computing?

38. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers Synthesis input : Scaffold … (1) functional specification … (2) resource constraints Resources available: Looping structure e.g., acyclic;loop(acyclic) Does it contain a nested loop? Two loops in a sequence? etc. Stack template e.g., onevariable How many local variables are available for the procedure?

39. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers Synthesis input : Scaffold … (1) functional specification … (2) resource constraints e.g., linear expressions … (3) domains What are the kinds of operations in the program? Types of expressions

40. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers Synthesis input : Scaffold … (1) functional specification … (2) resource constraints … (3) domains

41. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers Synthesis input : Scaffold Synthesis approach: Synthesis conditions

42. Synthesis approach: Synthesis conditions Scaffold Solver’s answer: Trivial solution: Safety Constraints • 0<Y≤X ⋀Tinit⇒ Finv • 0<Y≤X ⋀Tinit⇒ Finv • Finv⋀Gloop⇒ |y-(Y/X)x| ≤ 1/2 • Finv⋀Gloop⇒ |y-(Y/X)x| ≤ 1/2 To get solutions translatable to real programs Well-formedness Constraints false false false false false false false Well-formedness constraints: Statement unknowns Guard unknowns • Finv⋀Tbody⇒ Finv • Finv⋀Tbody⇒ Finv To preclude non-terminating programs Termination Constraints Outputs should be reachable

43. Synthesis approach: Synthesis conditions Scaffold Safety Constraints • 0<Y≤X ⋀Tinit⇒ Finv Synthesis conditions • Finv⋀Gloop⇒ |y-(Y/X)x| ≤ 1/2 Well-formedness Constraints Well-formedness constraints: Statement unknowns Guard unknowns • Finv⋀Tbody⇒ Finv Solving synthesis conditions gives us valid, non-trivial programs Termination Constraints Outputs should be reachable

44. Synthesis approach: Synthesis conditions Scaffold Safety Constraints User input: Program User input: Scaffold • 0<Y≤X ⋀Tinit⇒ Finv Verification conditions Synthesis conditions • Finv⋀Gloop⇒ |y-(Y/X)x| ≤ 1/2 Well-formedness Constraints Well-formedness constraints: Statement unknowns Guard unknowns • Finv⋀Tbody⇒ Finv verifier verifier Termination Constraints Outputs should be reachable Proof Program + proof Synthesis Verification

45. Proof-theoretic synthesis • encode synthesis as generalized verification • solve using existing verifiers Scaffold User Input : 0<Y≤X : |y-(Y/X)x| ≤ 1/2 …discrete line drawing Program: v := 2Y-X; y := 0; x := 0; while ( x≤X ) { print (x,y) if ( v<0 ) { v := v + 2Y; x++; } else { v := v + 2(Y-X); y++; x++; } } (1) functional specification: : acyclic;loop pre-condition post-condition : one extra variable Invariant: 0<Y≤X v = 2(x+1)Y-(2y+1)X 2(Y-X) ≤ v ≤ 2Y : (2) resource constraints: Program and Proof linear arithmetic looping structure variables allowed synthesis conditions (3) domains: • expressions • guards satisfiability-based verifier

46. Part II: PTS: Experiments Proof-theoretic synthesis Experimental highlights … linear arithmetic: Strassen’s n2.78 matrix multiplication Integral square root Bresenham’s discrete line drawing A2x2 B2x2 0<Y≤X Looping:  Stack: 7 vars • Looping: ;L() • Stack: 1 vars C2x2 = A x B x>0 |y-(Y/X)x| ≤ 1/2 • Looping: ;L() • Stack: 1 vars sqrt(x) ⌊ ⌋ Strassen’s Sqrt (binary search) Bresenham’s

47. Proof-theoretic synthesis Experimental highlights … linear arithmetic: Strassen’s n2.78 matrix multiplication Integral square root Bresenham’s discrete line drawing

48. Proof-theoretic synthesis Experimental highlights true Bubble Sort Insertion Sort nested loop, 0 vars Selection Sort nested loop, 1 vars Merge Sort recursive, 0 vars … sorting: Quick Sort recursive, 1 vars Bubble Sort Insertion Sort Selection Sort Merge Sort Quick Sort ∀k : 0≤k<n⇒ A[k1] ≤A[k2]

49. Proof-theoretic synthesis seconds (log scale) 0. 0.0 0.00 linear arithmetic examples predicate abstraction examples

50. Proof-theoretic synthesis + verification seconds (log scale) 0. 0.0 0.00 linear arithmetic examples predicate abstraction examples