1 / 134

Verification As a Matter Of Course

Verification As a Matter Of Course. Bertrand Meyer ETH Zurich, Eiffel Software & ITMO

Audrey
Download Presentation

Verification As a Matter Of Course

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VerificationAs a Matter Of Course Bertrand Meyer ETH Zurich, Eiffel Software & ITMO With Cristiano Calcagno, Carlo Furia, Alexander Kogtenkov*, Martin Nordio, Manuel Oriol+, Yu Pei, Nadia Polikarpova, Emmanuel Stapf*, Julian Tschannen, Stephan van Staden, Yi Wei, Andreas Zeller= & others *Eiffel Software +ETH, then U. of York, now ABB =U. Saarbrücken Microsoft Summer School, Moscow, July 2011

  2. SEL at ITMO • Software Engineering Laboratory | • Лаборатория Программной Инженерии • Создано в июне 2011 • Ещё остаются открытые позиции! • Аспиранты и Кандидаты (на полной ставке) • Временные гранты (“sabbaticals”) для исследователей, 2 до 6 месяцев

  3. Initiatives SEC-R conference in October 2011 ESEC 2013 will take place in Saint Petersburg New LinkedIn group: Программная Инженерия/ Software Engineering Russia Software Engineering Seminar in Saint PetersburgITMO and Polytechnical University (V. Itsykson) Every Thursday 18-21, visitors welcome http://meyerseminar.org

  4. Contents • The goal: trusted components • The EVE environment • Technical developments: • 3.1 Towards full contracts • 3.2 Negative variables & the essence of object-oriented programming • 3.3 Language convention for frames • 3.4 The alias calculus • 3.5 A Calculus of Object Programs • Conclusion and assessment • Not included: concurrency

  5. - 1 - Scope & Goals

  6. Three forms of software development Simple Web sites, spreadsheets, … Enterprise computing, simulation, “soft” real-time, … Enterprise computing, simulation, “soft” real-time, … Transportation, critical infrastructure, “hard” real-time, … • 1. Casual • 2. Professional • 3. Critical

  7. Language techniques for professional development Dynamic allocation Classes, objects Inheritance Polymorphism,dynamic binding Exception handling Closures (agents) Exceptions Genericity ...

  8. Research context (1): method and language • Our work is based on Eiffel: • Method and language for quality-focused development • For professional systems, including very large ones • Continuous, seamless development • ISO standard, significant user base and code base • Libraries • Excellent development environment: EiffelStudio • Some advantages: • Built-in, deeply ingrained use of contracts: preconditions, postconditions, class invariants • Open-source, no patent or other proprietary issues • We can shape the future method, language and IDE • Java is not a Law of Nature

  9. - 2 - The EVE environment

  10. Research context (2): EVE • Eiffel Verification Environment • Open source • Developed at ETH, others’ contributions welcome • Continuous integration (weekly) with EiffelStudio • The platform for trying new ideas • All ETH and ITMO contributions included

  11. Verification As a Matter Of Course Test results AutoProof Suggestions Inter.prover Test execution EVE AutoTest Test case generation Aliasanalysis Suggestions Invariantinference Sep. logicprover Invariantinference Arbiter AutoFix

  12. Scores Carlo Furia, Julian Tschannen, Martin Nordio • Failed test: score = -1 • Successful test run: 0 < score <= 0.9 • Correctness proof: score = 1 May be scaled down if some aspects not taken into account, e.g. arithmetic overflow • In practice (AutoProof): • Successful proof: score = 1 • Out-of-memory, timeout: score = 0 • Failed proof with abstract trace: score = -1

  13. Demo

  14. Combining static and dynamic techniques • Static verification: + More modular + Scales up better - Not always feasible - Often tests idealized version of the software - Captures only specified properties • Testing: + Generally possible + Can be automated + Tests software as it is + Can integrate non-software parts - Partial (very!) - Captures only specified properties - Not always feasible

  15. Testing “Ich bin der Geist, der stets verneint”Goethe, Faust, Act I FiodorChaliapineas Mephistopheles To test a software system is to try to make it fail

  16. AutoTest Ilinca CiupaAndreas LeitnerYi WeiEmmanuel StapfArno FivaManuel Oriol… • (e.g. IEEE Computer, Sep. 2009) • Integral part of EiffelStudio • Has already uncovered hundreds of bugs in libraries and other software • Three components: • Test generation • Test extraction (from failures) • Manual tests

  17. AutoTest: Test generation • Input: set of classes • Generates instances, calls routines with automaticallyselected arguments • Oracles are contracts: • Direct precondition violation: skip(precondition satisfaction techniques minimize this) • Postcondition/invariant violation: bug found • Any test (manual or automated) that fails becomes part of the test suite

  18. Maurice Maeterlinck, La Vie des Abeilles, 1901

  19. AutoTest: Test generation Bernd Schoeller * * Test: s1, s2: SET s2  s1 *SET +SET2 +SET1 + + *: Deferred +: Effective

  20. AutoTest: Test extraction • Turn a failed execution into a test case

  21. Who finds what faults? I.Ciupa, A. Leitner,M.Oriol, A. PretschnerICST 2008 (best paper) On a small EiffelBase subset,we compared: • AutoTest • Manual testing (students) (3 classes, 2 with bugs seeded) • User reports from the field • Largely separate kinds of faults • AutoTest: 62% specification, 38% implementation • User reports: 36% specification, 64% implementation

  22. CITADEL: Daikon for Eiffel Nadia PolikarpovaIlincaCiupaISSTA 2009 How do programmer-written and inferred contracts compare? Can contract inference be used to improve the quality of programmer-written contracts?

  23. CITADEL results • A high proportion of inferred contract clauses are correct (90%) and relevant (64%) • Contract inference does not find all programmer-written contracts (only 59%) • Programmers do not write all inferred contracts (25%) • Contract inference can strengthen programmer-written postconditions and invariants and find missing precondition clauses

  24. AutoFix: programs that fix themselves Yi Wei, Yu Pei, Andreas Zeller, ISSTA 2010 Yi Wei, Yu Pei, Carlo Furia, ASE 2011 (etc.) Passing & failing test cases Difference Fix to minimize the difference • 16 faults fixed out of 42 • Some of the fixes are exactly the same as those proposed by professional programmers

  25. - 3.1 - Towardscompletecontracts

  26. Typical contract: list insertion count index 1 v -- Previous elements unchanged! put_right (v: G) -- Insert v to the right of cursor require index <= count ensure i_th (index + 1) = v count =oldcount + 1 index =oldindex end

  27. Complete contracts Bernd Schoeller, Tobias Widmer (2008)Nadia Polikarpova (VSTTE 2010) Contracts are typically incomplete (unlike those of fully formal approaches such as Z) Our solution: • Use models A model is a mathematical interpretation of the structures Model library: MML (Mathematical Model Library) Fully applicative (no side effects, attributes, assignment etc.) But: expressed in Eiffel (preserving executability)

  28. Typical contract: list insertion count index 1 v -- Previous elements unchanged! put_right (v: G) -- Insert v to the right of cursor require index <= count ensure i_th (index + 1) = v count =oldcount + 1 index =oldindex end

  29. List insertion with model-based contract note model: sequence, index classLIST [G] ... sequence: MML_SEQUENCE [G] put_right (v: G) -- Insert v to the right of cursor. require index <= sequence.count ensure sequence = old(sequence.front (index).extended (x) + sequence.tail (index + 1)) index = oldindex end ...

  30. Model-based contracts: applications • On 7 of the most popular EiffelBase classes Testing found 4 “functional” faults by violation of model-based contracts • EiffelBase2: a data structures library with full contracts • Aim is to prove the code against these contracts • Now in progress: specifying application libraries (graphics, networking...)

  31. - 3.2 - Negative Variables& the Essenceof Object-Oriented Programming

  32. O-O analysis, design & programming “Principle of general relativity”: every operation in a class is expressed relative to a “current object” (self, this, Current), known only at execution time class C feature r do x := 1 x . f (...) end ... end Need to model this property in the logic

  33. Simplified denotational rule for routines (unqualified call) Notations: • r: body of r • r : list of formal arguments of r • F [v:e] : Substitution of v for e in F (for variable v and expression e; generalized to vectors of variables and expressions) • Distributed dot (used next), for a list v =<u, v, w, …>: xv = <xu, x  v, x  v, …> (call r (c)) (s) = r (s [r : c])

  34. Classic axiomatic rule for routines • {P} r {Q} • {P [r: c]} call r (c) {Q [r: c]} (Ignoring recursion)

  35. Tentative rule for qualified (O-O) calls • {P and INV} r {Q and INV} • {xP [r: c]} callx  r (c) {xQ [r: c]} Standard solution (e.g. Peter Müller’s thesis):in r, replace all occurrences of an attribute a by Current.a (Inadequate!)

  36. Object-oriented calls Known by the target as x’ y • Negative variables: Current y • x Current= x • Current x = x • x’ x = Current • x x’ = Current • Current’ = Current x’ x Target object call x r (c)

  37. Denotational rule for calls = xr(s [r: x’c]) • Then for any property P(such that “” distributes over P): • P (call x  r (c)) = xP(call r (x’c)) = x(P(r) [r: x’c]) Unqualified calls: (call r (c)) (s) = r (s [r : c]) Qualified calls: (call x  r (c)) (s) = xcall r (x’ c) (s)

  38. Adequate Hoare rule for qualified calls • {P and INV} r {Q and INV} • {xP [r: c]} callx  r (c) {xQ [r: c]} Proper rule • {P and INV} r {Q and INV} • {xP [r: x’c]} callx  r (c) {xQ [r: x’c]} Reminder: inadequate rule

  39. - 3.3 - Framing:a language convention

  40. The frame issue: • Specifying what does not change In a class ACCOUNT: deposit (sum: INTEGER) require ... do... ensure balance = old balance + sum end What about other queries other than balance, e.g. owner, account_number, overdraft_limit...? What about descendant classes, e.g. MAFIA_ACCOUNT?

  41. “Modifies” clauses • ESC-Java, JML, Spec#: specify what a command can change • Theoretical problems: • This can be an infinite set • Requires careful design for inheritance • Practical problem: tedious to express • Pragmatic observation from survey of JMLcode: • Every item listed in a modifiesclause also appears in the postcondition!

  42. The proposed language convention • deposit (sum: INTEGER ) • require ... • do... • ensure • balance = old balance + sum • end ; cur: CURRENCY old has_penalty = (cur /= currency) involved (is_overdraft) This also avoids introducing a special language notation to express that a routine is pure A routine may only modify queries listed (outside of an oldexpression) in its postcondition

  43. - 3.4 - The Alias Calculus

  44. The question asked by the Alias Calculus Broy volume (IJSI, 2011) Given expressions e and f, and a program location p: Can e and f ever be attached to the same object at p?

  45. An example  x y 

  46. Why is this important? Alias analysis can be a key tool for proving O-O programs (as an alternative to separation logic)

  47. Applications of alias analysis b -- c = c, i.e. True a ? set_a (c) x.set_a (c) Understand as x.a := c -- y.a = b -- y.a = b x -- x.a = c -- x.a = c y c 1. Without it, cannot apply standard proof techniques to programs involving pointers 2. Concurrent program analysis, in particular deadlock 3. Program optimization

  48. Alias relations A binary relation is an alias relationif it is symmetric andirreflexive Can alias x to y and y to z but not x to z Relation of interest: “In the computation, x might become aliased to y” Definition: Not necessarily transitive: ifcthen x := y else y := z end

  49. Describing an alias relation • Set of binary relations on E; formally: P(E x E) D Identity on E = Set difference {[x, y], [y, x], [y, z], [z, y]}  {[x, y], [y, x], [x, z], [z, x], [y, z], [z, y]} • If r is a relation in E E, the following is an alias relation: rr r-1― Id [E] • Example: {[x, x], [x, y], [y, z]}= • Generalized to sets: • {x, y, z} =

More Related