1 / 132

Verification As a Matter Of Course

Verification As a Matter Of Course. Bertrand Meyer ETH Zurich, Eiffel Software & ITMO

toshi
Download Presentation

Verification As a Matter Of Course

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VerificationAs a Matter Of Course Bertrand Meyer ETH Zurich, Eiffel Software & ITMO With Cristiano Calcagno, Carlo Furia, Alexander Kogtenkov*, Martin Nordio, Manuel Oriol+, Yu Pei, Nadia Polikarpova, Emmanuel Stapf*, Julian Tschannen, Stephan van Staden, Yi Wei, Andreas Zeller#& others *Eiffel Software +ETH, then U. of York, now ABB #U. Saarbrücken Draft notes, LASER Summer School, Procchio (Elba), September 2011

  2. Contents • The goal: trusted components • The EVE environment • Technical developments: • 3.1 Towards full contracts • 3.2 Negative variables & the essence of object-oriented programming • 3.3 Language convention for frames • 3.4 The alias calculus • 3.5 A Calculus of Object Programs • Conclusion and assessment • Not included: concurrency

  3. - 1 - Scope & Goals

  4. Three forms of software development Simple Web sites, spreadsheets, … Enterprise computing, simulation, “soft” real-time, … Enterprise computing, simulation, “soft” real-time, … Transportation, critical infrastructure, “hard” real-time, … • 1. Casual • 2. Professional • 3. Critical

  5. Language techniques for professional development Dynamic allocation Classes, objects Inheritance Polymorphism,dynamic binding Exception handling Closures (agents) Exceptions Genericity ...

  6. Research context (1): method and language • Our work is based on Eiffel: • Method and language for quality-focused development • For professional systems, including very large ones • Continuous, seamless development • ISO standard, significant user base and code base • Libraries • Excellent development environment: EiffelStudio • Some advantages: • Built-in, deeply ingrained use of contracts: preconditions, postconditions, class invariants • Open-source, no patent or other proprietary issues • We can shape the future method, language and IDE • Java is not a Law of Nature

  7. - 2 - The EVE environment

  8. Research context (2): EVE • Eiffel Verification Environment • Open source • Developed at ETH, others’ contributions welcome • Continuous integration (weekly) with EiffelStudio • The platform for trying new ideas • All ETH and ITMO contributions included

  9. Verification As a Matter Of Course Test results AutoProof Suggestions Inter.prover Test execution EVE AutoTest Test case generation Aliasanalysis Suggestions Invariantinference Sep. logicprover Invariantinference Arbiter AutoFix

  10. Scores Carlo Furia, Julian Tschannen, Martin Nordio • Failed test: score = -1 • Successful test run: 0 < score <= 0.9 • Correctness proof: score = 1 May be scaled down if some aspects not taken into account, e.g. arithmetic overflow • In practice (AutoProof): • Successful proof: score = 1 • Out-of-memory, timeout: score = 0 • Failed proof with abstract trace: score = -1

  11. Demo

  12. Combining static and dynamic techniques • Static verification: + More modular + Scales up better - Not always feasible - Often tests idealized version of the software - Captures only specified properties • Testing: + Generally possible + Can be automated + Tests software as it is + Can integrate non-software parts - Partial (very!) - Captures only specified properties - Not always feasible

  13. Testing “Ich bin der Geist, der stets verneint”Goethe, Faust, Act I FiodorChaliapineas Mephistopheles To test a software system is to try to make it fail

  14. AutoTest Ilinca CiupaAndreas LeitnerYi WeiEmmanuel StapfArno FivaManuel Oriol… • (e.g. IEEE Computer, Sep. 2009) • Integral part of EiffelStudio • Has already uncovered hundreds of bugs in libraries and other software • Three components: • Test generation • Test extraction (from failures) • Manual tests

  15. AutoTest: Test generation • Input: set of classes • Generates instances, calls routines with automaticallyselected arguments • Oracles are contracts: • Direct precondition violation: skip(precondition satisfaction techniques minimize this) • Postcondition/invariant violation: bug found • Any test (manual or automated) that fails becomes part of the test suite

  16. Maurice Maeterlinck, La Vie des Abeilles, 1901

  17. AutoTest: Test generation Bernd Schoeller * * Test: s1, s2: SET s2  s1 *SET +SET2 +SET1 + + *: Deferred +: Effective

  18. AutoTest: Test extraction • Turn a failed execution into a test case

  19. Who finds what faults? I.Ciupa, A. Leitner,M.Oriol, A. PretschnerICST 2008 (best paper) On a small EiffelBase subset,we compared: • AutoTest • Manual testing (students) (3 classes, 2 with bugs seeded) • User reports from the field • Largely separate kinds of faults • AutoTest: 62% specification, 38% implementation • User reports: 36% specification, 64% implementation

  20. CITADEL: Daikon for Eiffel Nadia PolikarpovaIlincaCiupaISSTA 2009 How do programmer-written and inferred contracts compare? Can contract inference be used to improve the quality of programmer-written contracts?

  21. CITADEL results • A high proportion of inferred contract clauses are correct (90%) and relevant (64%) • Contract inference does not find all programmer-written contracts (only 59%) • Programmers do not write all inferred contracts (25%) • Contract inference can strengthen programmer-written postconditions and invariants and find missing precondition clauses

  22. AutoFix: programs that fix themselves Yi Wei, Yu Pei, Andreas Zeller, ISSTA 2010 Yi Wei, Yu Pei, Carlo Furia, ASE 2011 (etc.) Passing & failing test cases Difference Fix to minimize the difference • 16 faults fixed out of 42 • Some of the fixes are exactly the same as those proposed by professional programmers

  23. - 3.1 - Towardscompletecontracts

  24. Typical contract: list insertion count index 1 v -- Previous elements unchanged! put_right (v: G) -- Insert v to the right of cursor require index <= count ensure i_th (index + 1) = v count =oldcount + 1 index =oldindex end

  25. Complete contracts Bernd Schoeller, Tobias Widmer (2008)Nadia Polikarpova (VSTTE 2010) Contracts are typically incomplete (unlike those of fully formal approaches such as Z) Our solution: • Use models A model is a mathematical interpretation of the structures Model library: MML (Mathematical Model Library) Fully applicative (no side effects, attributes, assignment etc.) But: expressed in Eiffel (preserving executability)

  26. Typical contract: list insertion count index 1 v -- Previous elements unchanged! put_right (v: G) -- Insert v to the right of cursor require index <= count ensure i_th (index + 1) = v count =oldcount + 1 index =oldindex end

  27. List insertion with model-based contract note model: sequence, index classLIST [G] ... sequence: MML_SEQUENCE [G] put_right (v: G) -- Insert v to the right of cursor. require index <= sequence.count ensure sequence = old(sequence.front (index).extended (x) + sequence.tail (index + 1)) index = oldindex end ...

  28. Model-based contracts: applications • On 7 of the most popular EiffelBase classes Testing found 4 “functional” faults by violation of model-based contracts • EiffelBase2: a data structures library with full contracts • Aim is to prove the code against these contracts • Now in progress: specifying application libraries (graphics, networking...)

  29. - 3.2 - Negative Variables& the Essenceof Object-Oriented Programming

  30. O-O analysis, design & programming “Principle of general relativity”: every operation in a class is expressed relative to a “current object” (self, this, Current), known only at execution time class C feature r do x := 1 x . f (...) end ... end Need to model this property in the logic

  31. Simplified denotational rule for routines (unqualified call) Notations: • r: body of r • r : list of formal arguments of r • F [v:e] : Substitution of v for e in F (for variable v and expression e; generalized to vectors of variables and expressions) • Distributed dot (used next), for a list v =<u, v, w, …>: xv = <xu, x  v, x  v, …> (call r (c)) (s) = r (s [r : c])

  32. Classic axiomatic rule for routines • {P} r {Q} • {P [r: c]} call r (c) {Q [r: c]} (Ignoring recursion)

  33. Tentative rule for qualified (O-O) calls • {P and INV} r {Q and INV} • {xP [r: c]} callx  r (c) {xQ [r: c]} Standard solution (e.g. Peter Müller’s thesis):in r, replace all occurrences of an attribute a by Current.a (Inadequate!)

  34. Object-oriented calls Known by the target as x’ y • Negative variables: Current y • x Current= x • Current x = x • x’ x = Current • x x’ = Current • Current’ = Current x’ x Target object call x r (c)

  35. Denotational rule for calls = xr(s [r: x’c]) • Then for any property P(such that “” distributes over P): • P (call x  r (c)) = xP(call r (x’c)) = x(P(r) [r: x’c]) Unqualified calls: (call r (c)) (s) = r (s [r : c]) Qualified calls: (call x  r (c)) (s) = xcall r (x’ c) (s)

  36. Adequate Hoare rule for qualified calls • {P and INV} r {Q and INV} • {xP [r: c]} callx  r (c) {xQ [r: c]} Proper rule • {P and INV} r {Q and INV} • {xP [r: x’c]} callx  r (c) {xQ [r: x’c]} Reminder: inadequate rule

  37. - 3.3 - Framing:a language convention

  38. The frame issue: • Specifying what does not change In a class ACCOUNT: deposit (sum: INTEGER) require ... do... ensure balance = old balance + sum end What about other queries other than balance, e.g. owner, account_number, overdraft_limit...? What about descendant classes, e.g. MAFIA_ACCOUNT?

  39. “Modifies” clauses • ESC-Java, JML, Spec#: specify what a command can change • Theoretical problems: • This can be an infinite set • Requires careful design for inheritance • Practical problem: tedious to express • Pragmatic observation from survey of JMLcode: • Every item listed in a modifiesclause also appears in the postcondition!

  40. The proposed language convention • deposit (sum: INTEGER ) • require ... • do... • ensure • balance = old balance + sum • end ; cur: CURRENCY old has_penalty = (cur /= currency) involved (is_overdraft) This also avoids introducing a special language notation to express that a routine is pure A routine may only modify queries listed (outside of an oldexpression) in its postcondition

  41. - 3.4 - The Alias Calculus

  42. The question asked by the Alias Calculus Broy volume (IJSI, 2011) Given expressions e and f, and a program location p: Can e and f ever be attached to the same object at p?

  43. An example  x y 

  44. Why is this important? Alias analysis can be a key tool for proving O-O programs (as an alternative to separation logic)

  45. Applications of alias analysis b -- c = c, i.e. True a ? set_a (c) x.set_a (c) Understand as x.a := c -- y.a = b -- y.a = b x -- x.a = c -- x.a = c y c 1. Without it, cannot apply standard proof techniques to programs involving pointers 2. Concurrent program analysis, in particular deadlock 3. Program optimization

  46. Alias relations A binary relation is an alias relationif it is symmetric andirreflexive Can alias x to y and y to z but not x to z Relation of interest: “In the computation, x might become aliased to y” Definition: Not necessarily transitive: ifcthen x := y else y := z end

  47. Describing an alias relation • Set of binary relations on E; formally: P(E x E) D Identity on E = Set difference {[x, y], [y, x], [y, z], [z, y]}  {[x, y], [y, x], [x, z], [z, x], [y, z], [z, y]} • If r is a relation in E E, the following is an alias relation: rr r-1― Id [E] • Example: {[x, x], [x, y], [y, z]}= • Generalized to sets: • {x, y, z} =

  48. Canonical form and alias diagrams x, y, y, z, x, u, v {x, y}  { y, z}  {x, u, v} x , y y y • Alias diagram: y, z • (not canonical) y y x, u, v x x • Make it canonical: Canonical form of an alias relation: e.g. , meaning None of the sets of variablesis a subset of another

  49. Alias diagrams (non-O-O) Value nodes Value nodes Value nodes Source node In canonical form: no label is subset of another; each label has at least two variables x , y y, z x, u, v

More Related