1 / 31

Visions of Verification

Visions of Verification. Bruce W. Weide, et al. Computer Science and Engineering The Ohio State University http://www.cse.ohio-state.edu/rsrg. Outline. Design-by-Contract Testing vs. Verification Steps Toward Verification Conclusions. Modular reasoning boundary. uses. implements.

kaspar
Download Presentation

Visions of Verification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visions of Verification Bruce W. Weide, et al. Computer Science and Engineering The Ohio State University http://www.cse.ohio-state.edu/rsrg

  2. Outline • Design-by-Contract • Testing vs. Verification • Steps Toward Verification • Conclusions

  3. Modular reasoning boundary uses implements uses implements implements uses 1. Design-by-Contract

  4. A Contract    • Mathematical models for values of all types of variables ("objects") • Preconditions and postconditions for all exported operations ("methods")

  5. Example Contract    contract List_Template (type Item) typeList is modeled by (left: string of Item, right: string of Item) exemplar p initialization ensures p = (< >, < >) // operations end List_Template

  6. Example Operation    operation Remove (updates p: List, replaces x: Item) requires |p.right| > 0 ensures #p = (p.left, <x> * p.right)

  7. List_Template Operations     Insert (updates p: List, alters x: Item) Remove (updates p: List, replaces x: Item) Advance (updates p: List) Move_To_Start (updates p: List) Move_To_Finish (updates p: List) Left_Length (restores p: List): Integer Right_Length (restores p: List): Integer

  8. A Contract Realization    • Representation data structures for all types it implements • Code for algorithms for all operations it implements • Loop invariants, representation invariants, abstraction relations as needed

  9. A Contract Enhancement    enhancement Reversal_Capability for List_Template operation Reverse (updates p: List) requires |p.left| = 0 ensures p = (reverse (#p.right), < >) end Reversal_Capability

  10. An Enhancement Realization     realization Dubious for Reversal_Capability procedure Reverse (updates p: List) decreasing |p.right| if Right_Length (p) > 0 then variable x: Item Remove (p, x) Reverse (p) Insert (p, x) end if end Reverse end Dubious

  11. Modular reasoning boundary uses Reversal_Capability implements Dubious uses List_Template implements uses 2. Testing vs. Verification Is this claim valid?

  12. Testing • Objective: to show code is incorrect • Approach: • Select a small finite number of test cases • Run code on test cases • Compare behavior to that specified by contract • Possible outcomes: • "Code is incorrect" • "Code may be correct for test cases" "Darn!"

  13. Verification Automatically? Hoare's "verifying compiler" • Objective: to show code is correct • Approach: • Generate verification conditions • Mathematically prove verification conditions • Possible outcomes: • "Code is correct in all cases" • "Code is incorrect (and here is the bug!)" • "Sorry, can't tell because of incompleteness" "Darn!"

  14. is correct Verification Condition Generator Automated Theorem Prover is incorrect (here!) Verifying Compiler

  15. Testing Verification 3. Steps Toward Verification DEET Can show code correct Can show code incorrect Can find substitution bugs Can pinpoint bugs Modular Automated Practical

  16. What is DEET? • N,N-diethyl-m-toluamide • NEJM 2002: "DEET is best bug repellant" • Detecting Errors Efficiently without Testing • Approaching all advantages of verification— except showing that code is correct—by… • Trying to show that code is incorrect

  17. Step 1: Generate VCs • Between each pair of statements in code: • What path condition can get us here? • What can we assume here? • What do we need to confirm here?

  18. 0 P: true 1 P: |p0.right| > 0 2 P: |p0.right| > 0 3 P: |p0.right| > 0 4 P: |p0.right| > 0 5 P: true if Right_Length (p) > 0 then variable x: Item Remove (p, x) Reverse (p) Insert (p, x) end if

  19. 0 A: |p0.left| = 0 1 A: INIT(x1) and p1 = p0 2 A: p2.left = p1.left and p1.right = <x2> * p2.right 3 A: x3 = x2 and p3 = (reverse(p2), < >) 4 A: p4 = (p3.left, <x3> * p3.right) 5 A: if |p0.right| > 0 then p5 = p4 else p5 = p0 if Right_Length (p) > 0 then variable x: Item Remove (p, x) Reverse (p) Insert (p, x) end if

  20. 0 C: true 1 C: |p1.right| > 0 2 C: |p2.left| = 0 and |p2.right| < |p0.right| 3 C: true 4 C: true 5 C: p5 = (reverse(p0.right), < >) if Right_Length (p) > 0 then variable x: Item Remove (p, x) Reverse (p) Insert (p, x) end if

  21. Constructing the VCs All of mathematics… • Sound and relatively complete proof system [Krone 88, Heym 95] combines P, A, and C assertions into verification conditions, each of which must be proved • For each i, VCi must be true for all values of the variables: (  (0≤j≤i(Pj Aj)))  (Pi Ci) plus all assumptions through state i… may be used to confirm what is needed in state i

  22. Step 2: Generate Error Hypotheses • Negation of VCi is error hypothesis EHi:  Pi (0≤j≤i(Pj Aj)) Ci • If there exists a satisfying assignment of values to the variables of EHi, then VCi cannot be true for all values of the variables • Such an assignment is a witness to the incorrectness of the code

  23. Finding a Witness • Problem: Proving an EH with an automated theorem prover is really no easier than proving a VC…

  24. Step 3: Restrict Scope of Variables • "Small scope hypothesis" [Jackson 00] says that many program errors are revealed by "small" witnesses Original (infinite) state space Restricted (small, finite) state space

  25. A New Version of the Problem • Restrict Item variables to a single value • Restrict string of Item variables to two values: empty string, or length one • Result of finiteness: EH can be rewritten as a propositional formula for which a satisfying assignment is a witness to incorrect code

  26. Sample Clauses • |p2.left| = 0 becomes: len_p2_left_equals_zero • Also needed (among many other facts from mathematics): len_p2_left_equals_zero  p2_left_equals_empty

  27. Another Sample • p1 = p0becomes: ((p1_left_equals_empty  p0_left_equals_empty)  (p1_right_equals_empty  p0_right_equals_empty))  ((p1_left_equals_s1  p0_left_equals_s1)  (p1_right_equals_s1  p0_right_equals_s1))  ...

  28. Step 4: Search for Witnesses • SAT solver used [Küchlin, et al. 01] directly handles non-CNF formulas • Solver shows EH for example given here to be (uniquely) satisfiable • Computation time: less than a second • Witness explicitly describes value of each variable at each state in the program, making it easy to debug incorrect code

  29. 4. Conclusions • Many variations should be explored: • How should quantifiers be handled? • Should simplification be done with original EHs, or by the SAT solver? • Can restricted scopes of variable values be incrementally enlarged without starting over? • Can a more general computer algebra system be used in place of a SAT solver?

  30. Conclusions • Verifying compiler is not beyond the realm of possibility • But theorem provers must be improved significantly to achieve it! • Combining current verification technology with a "testing psychology" may be a viable intermediate step

  31. Visions of Verification Questions? http://www.cse.ohio-state.edu/rsrg

More Related