1 / 31

Teaching Software Correctness

Session 01 — 9:00-10:15, May 13. May 13-15, 2008, University of Oklahoma. http://www.cs.ou.edu/~rlpage/SEcollab/tsc. Teaching Software Correctness. Rex Page, U Oklahoma page@ou.edu Assistants Carl Eastlund (lead), Northeastern U cce@ccs.neu.edu Ryan Ralston, U Oklahoma strawdog@ou.edu

skylar
Download Presentation

Teaching Software Correctness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Session 01 — 9:00-10:15, May 13 May 13-15, 2008, University of Oklahoma http://www.cs.ou.edu/~rlpage/SEcollab/tsc Teaching Software Correctness • Rex Page, U Oklahoma page@ou.edu • Assistants • Carl Eastlund (lead), Northeastern U cce@ccs.neu.edu • Ryan Ralston, U Oklahoma strawdog@ou.edu • Zac White, U Oklahoma zacwhite@gmail.com Collaboration with Matthias Felleisen - NSF/DUE 0633664, 0813529, 0632872 TSC Workshop, May 2008, U Oklahoma 1

  2. Go through material from SE at OU • 30% of 3-credit course (4 of 15 lecture-weeks) • Coverage of coherent subsets: 1 to 3 weeks • Lab projects • To gain a student-like experience Strategy • Convincing yourself that it's important • Material is unusual … many students will resist • Instructor zeal transfers to some students • Instructor apathy transfers to all students • Devising projects that students can do • Instructors must solve assignments beforehand Hard part • New appreciation meaning of "correctness" • Prep for future programming environments • A few students ready to lead next decade Benefits Goals • New CS-course content at some schools • Predicate-based testing • Mechanical logic to verify software properties TSC Workshop, May 2008, U Oklahoma

  3. 2008 plans • DrACuLa + DblChk • + Modules • 2007 SE course • DrACuLa prog's env • with DoubleCheck • 2006 SE course • DrACuLA prog'g environment • File I/O + GUI • 2003 SE course • Programs: Scheme (DrScheme) • Verify correctness: ACL2 • 2005 SE course • ACL2, File I/O • ESC paper, FDPE'05 • 2004 SE course • Program/Verify: ACL2 • File I/O only History • Years of rumination about software education • defect control • equation-based programs TSC Workshop, May 2008, U Oklahoma

  4. PSP - Humphrey Equation-Based Programming ACL2 – J Moore Lisp with mechanical logic The Three Themes of SE-I • Low defect rate • Predictable quality • Reliable schedule • Do it right • not necessarily fast • Primarily for people • Think “Strunk and White” • Process • Design • Quality Besides a mathematical inclination, an exceptionally good mastery of one's native tongue is the most vital asset of a competent programmer. — Dijkstra TSC Workshop, May 2008, U Oklahoma 4

  5. Provably Correct Software300 Years in the Making If we had some exact language … or at least a kind of truly philosophic writing, in which the ideas were reduced to a kind of alphabet of human thought, then all that follows rationally from what is given could be found by a kind of calculus, just as arithmetical or geometrical problems are solved. - Gottfried Leibniz (about 1700) Symbolic logic - Boole, Frege, Peano, … (mid to late 1800s) Formal logic - Gentzen, Church, Curry, Gödel, … (1930s) Mechanized logic - Boyer & Moore, Milner, … (late 1900s) TSC Workshop, May 2008, U Oklahoma 5

  6. ACL2 – a mechanized logicACL2 = (ACL)2 = ACLACLA Computational Logic for Applicative Common Lisp • Lisp programs • Expressed as equations defining software properties • Restricted to "true functions" • Every invocation of f(2) delivers the same value (unlike C, C++ …) • Computation model: substitution of equals for equals • Programming model • Program specifies properties … interpretation derives process • Conventional: program specifies process … properties implicit • Derivable properties expressed as logic formulas • Turing equivalence • All computations are expressible through equations • Equation-based software (Lisp, ML, Haskell …) can model conventional software (C, C++, Java, …) and vice versa • Lisp versus C • Compiled Lisp slower … 10% to order of magnitude • C development tools more advanced • … but DrACuLa and ACL2s are moving in the right direction TSC Workshop, May 2008, U Oklahoma

  7. axiom 1 axiom 2 theorem Example Software Propertyappend (here called “cat”) is associative • Defined properites (axioms) (defun cat (xs ys) (if (null xs) ys ; (cat nil ys) = ys (cons (car xs) (cat (cdr xs) ys)))) ; (cat (list x1 x2 …) ys) = (cons x1 (cat (list x2 …) ys)) inductive definition • Derived property (theorem) (defthm append-is-associative (implies (and (true-listp xs) (true-listp ys)) (equal (cat xs (cat ys zs)) (cat (cat xs ys) zs))))) • Proof • List-induction on xs • Base case: xs = nil • Inductive case: xs = (cons (car xs) (cdr xs)) TSC Workshop, May 2008, U Oklahoma

  8. Induction hypothesis Append is Associativeinductive case • xs = (cons (car xs) (cdr xs)) • (equal (cat xs(cat ys zs)) (cat (cons (car xs) (cdr xs)) (cat ys zs))) • (equal (cat (cons (car xs) (cdr xs)) (cat ys zs)) (cons (car xs) (cat (cdr xs) (cat ys zs)))) • (equal (cons (car xs) (cat (cdr xs) (cat ys zs)) ) (cons (car xs) (cat (cat (cdr xs) ys) zs)) ) (defun cat (xs ys) (if (null xs) ys ; = (cat nil ys) (cons (car xs) (cat (cdr xs) ys)))) ; = (cat xs ys) • (equal (cons (car xs) (cat (cat (cdr xs) ys) zs)) (cat (cons (car xs) (cat (cdr xs) ys) zs))) ) • (equal (cat (cons (car xs) (cat (cdr xs) ys) zs) ) (cat (cat (cons (car xs) (cdr xs)) ys) zs) ) • (equal (cat (cat (cons (car xs) (cdr xs)) ys) zs) (cat (cat xs ys) zs) ) TSC Workshop, May 2008, U Oklahoma

  9. Mechanization Is Necessarywithout it, you get lost in details • Even simple properties lead to big proofs • Associativity proof (as presented) lacks many details • Millions of details in proofs about big programs • People can’t keep track of millions of details • But, computers can • Computers couldn’t 10 years ago – not enough capacity • People formulate properties … computers push details • Proof organized into lemmas — like software in modules • On the level of “rigorous” mathematical proof • Not fully formal • Some lemma architectures are better than others • Just as some modular decompositions of software are superior • Formulation of properties is a big task • Experience and judgment required — as in software development TSC Workshop, May 2008, U Oklahoma

  10. Things That Have Been Done with ACL2 • AMD Athlon (K7) floating point processor (1999) • AMD division circuit verified correct • After 1994/97 Pentium division bugs • Property (implies (and (floating-point-numberp p 15 64) (floating-point-numberp d 15 64) (not (equal d 0)) (rounding-modep mode)) (equal (divide p d mode) (round (/ p d) mode)))… • Other commercial applications • Motorola DSP microcode • Rockwell Collins AAMP7 avionics software • Numerous NSA applications • … “high assurance software” … TSC Workshop, May 2008, U Oklahoma

  11. Testing versus Proving • Testing • AMD floating-point • AMD test suite of 80-million cases wasn’t enough • Hardware with bugs passed all 80-million tests • Huge finite number of cases (215+64215+64 = 2158) • Cannot complete test before sun runs out of fuel • ACL2 proof covers all cases • Proofs cover all cases • Associativity proof was for any xs, ys, zs • That is, an infinite number of “test cases” • Testing cannot ensure correctness • Testing + logic can ensure that software satisfies essential properties TSC Workshop, May 2008, U Oklahoma

  12. Applicative Common Lisp the programming language part of ACL2 TSC Workshop, May 2008, U Oklahoma

  13. true lists: nil (element . true-list) Data in ACL2 All ACL2 data structures come down to these basics • Atoms • Numbers • Integer, rational, complex (decimal, binary, octal, hex) • 5, -12, 17/10, -17/10, #c(0, 1) , #b101 , #o-14 , #x11/A , #x-11/A • No floating point numbers (no intrinsic sqrt function) • Characters: #\A, #\b, #\1, #\9, #\Newline, #\Space, … • Strings: "Kauffmann, Manolios, & Moore #0-7923-7757-5" • Symbols: nil, t, bob, next-term • Conses (dotted pairs, binary trees) • (a . b) • ((a . left) . (b . right)) • (1 . nil), abbreviation: (1) • (x .(y .(z . nil))), abbreviation: (x y z) • nil, abbreviation: ( ) • (x .(y .(z . g))), abbreviation: (x y z . g) • ((x . 1) .((y . 2) . ((z . 3) . nil))), abbreviation ((x . 1) (y . 2) (z . 3)) • Abbreviation rule • When a dot is followed by left paren, delete the dot, the left paren, and the matching right paren • When a dot is followed by a nil, delete both • … or something like that TSC Workshop, May 2008, U Oklahoma

  14. Syntax of ACL2 - Formulas Prefix formulas only No infix, No postfix • Formula for evaluation • (op-symbol opnd1 opnd2 … opndn) • op-symbol must a symbol denoting an operator • Formula denotes value delivered by operator, with given operands • Operands may be formulas, defined symbols, or literals • Each operator will require a specific number of operands • Suppressed evaluation • Single-quote (apostrophe) prefixing formula suppresses evaluation • Known as a “quoted expression” (synonyms: “formula” = “expression”) • A quoted exp’n denotes the atom or dotted pair following the quote • Example: '((a b c) (d (e f) g) h) or: (quote ((a b c) (d (e f) g) h)) • This is the entire Lisp syntax • Well…okay…There are a few exceptions. Here’s one (others later): • Numbers, chars, and strings automatically denote data • No single-quote mark needed for these atoms • Nor for pre-defined symbols, such as nil and T TSC Workshop, May 2008, U Oklahoma

  15. x, ys— any formula • Discriminating atoms, conses, and empty lists • endp, null, atom, consp – defining equations • (null x) = (equal x nil) • (consp (cons xy)) = T • (atom (cons xy)) = nil • (atom x) = (null (consp x)) • (endp xs) = (atom xs) – allowed only on true lists x, y, xs— any formula Basic ACL2 Operators (intrinsics) • Building pairs and extracting parts • car, cdr, and cons operators – defining equations • (car (cons xys)) = x • (cdr (cons x ys)) = ys • (car any-formula-denoting-nil) = nil • (cdr any-formula-denoting-nil) = nil • Detecting equal values • “equal” operator • (equal xy) = T if x and y are formulas denoting the same value • (equal xy) = nil if x and y are formulas denoting different values These operators, plus a way to define new operators, are sufficient to describe any computation • Selecting between formulas • “if” operator – defining equations • (if any-formula-not-denoting-nil x y) = x • (if any-formula-denoting-nil x y) = y TSC Workshop, May 2008, U Oklahoma

  16. abbreviation of (cons (car xs) (cons (cadr xs) nil)) abbreviation of (car (cdr xs)) Defining Operators in ACL2 • defun – a “special form” • (defun f (a1 a2 … an) v) • f, a symbol, names the operator being defined • ai, a symbol, stands for the ith operand (n may be zero) • v, a formula, specifies the value the operator will deliver • After the defun, (f u1 u2 …un) denotes the formula v with each occurrence of ai replaced by the formula ui • defun has effect of attaching a name to an operator • defun is a “command” – commands alter the current ACL2 world • Ordinary formulas donot alter the world • defun doesnotdeliver a value • Ordinary formulas do deliver values • Examples • (defun drop-two (xs) (cdr (cdr xs))) • (drop-two i’(a b c d e)) = (c d e) • (drop-two (drop-two i’(a b c d e))) = (e) • (defun take-two (xs) (list (car xs) (cadr xs))) • (take-two i’(a b c d e)) = (a b) • Exercise • Define an operator to deliver the 3rd and 4th elements • Assume the operand is a list of at least four elements TSC Workshop, May 2008, U Oklahoma

  17. Commands defconst defproperty property name random value for xs - list of up to n random syms expected property of xs (predicate) random variable number of tests to run expected property to test for (predicate) Solutions • Solution 1 (defun 3rd&4th (xs) (list (caddr xs) (cadddr xs))) • Solution 2 (defun 3rd&4th-another-way (xs) (take-two (drop-two xs))) • Predicate-based testing (defconst *num-tests* 10) (defproperty 3rd&4th=3rd&4th-another-way *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (3rd&4th xs) (3rd&4th-another-way xs))) TSC Workshop, May 2008, U Oklahoma

  18. Commands defconst defproperty property name random value for xs expected property of xs (predicate) random variable number of tests to run expected property to test for (predicate) Solutions • Solution 1 (defun 3rd&4th (xs) (list (caddr xs) (cadddr xs))) • Solution 2 (defun 3rd&4th-another-way (xs) (take-two (drop-two xs))) • Predicate-based testing (defconst *num-tests* 10) (defproperty 3rd&4th=3rd&4th-another-way *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (3rd&4th xs) (3rd&4th-another-way xs))) Let's run a test TSC Workshop, May 2008, U Oklahoma

  19. Command defthm Solutions • Solution 1 (defun 3rd&4th (xs) (list (caddr xs) (cadddr xs))) • Solution 2 (defun 3rd&4th-another-way (xs) (take-two (drop-two xs))) • Predicate-based testing (defconst *num-tests* 10) (defproperty 3rd&4th=3rd&4th-another-way *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (3rd&4th xs) (3rd&4th-another-way xs))) • Theorem (defthm 3rd&4th=3rd&4th-another-way (equal (3rd&4th xs) (3rd&4th-another-way xs))) Maybe it's a theorem TSC Workshop, May 2008, U Oklahoma

  20. More Exercises • Logical operators • Define an operator that delivers a value other than nil if neither operand denotes nil, and nil if both operands denote nil • (&& formula-denoting-nil any-formula) = nil • (&& any-formula formula-denoting-nil) = nil • (&& non-nil-formulanon-nil-formula) = non-nil-formula • Define an operator that delivers nil if both operands denote nil, and a value other than nil if either operand denotes a non-nil value • (|| non-nil-formulaany-formula) = non-nil-formula • (|| any-formula non-nil-formula) = non-nil-formula • (|| formula-denoting-nil formula-denoting-nil) = nil • Define an operator that delivers a value other than nil if its operand denotes nil, and nil if its operand denotes a non-nil value • (~ non-nil-formula) = nil • (~ formula-denoting-nil) = non-nil-formula TSC Workshop, May 2008, U Oklahoma

  21. axioms negation of "or" is "and" of negations • deMorgan’s laws • ~(x || y) = (~ x) && (~ y) • ~(x && y) = (~ x) || (~ y) negation of "or" is "and" of negations derived properties aka, “theorems” Solutions • Solutions (defun && (x y) (if x y nil)) (defun || (x y) (if x x y)) (defun ~ (x) (if x nil t)) • deMorgan laws as ACL2 theorems (defthm deMorgan-or ;~(x || y) = (~ x) && (~ y) (equal (~ (|| x y)) (&& (~ x) (~ y)))) (defthm deMorgan-and ; ~(x && y) = (~ x) || (~ y) (equal (~ (&& x y)) (|| (~ x) (~ y)))) TSC Workshop, May 2008, U Oklahoma

  22. axioms • Define these operators: • Next-to-last element of list (defun rac2 (xs) … ) Exercises • Drop last two elements (defun rdc2 (xs) …) • Insert at end of list (defun snoc (x xs) …) More defun Examples (defun rac (xs) ; (rac xs) = last element of xs (if (endp (cdr xs)) (car xs) ; (rac (list x)) = x (rac (cdr xs)))) ; (rac (list x y z …) = (rac (list y z …)) inductive definitions (defun rdc (xs) (if (endp (cdr xs)) nil ; (rdc (list x)) = nil (cons (car xs) (rdc (cdr xs))))) ; (rdc (list x y z …)) = (cons x (rdc (list y z …))) How to make an inductive definition • State properties correctly • Cover all cases • Inductive part closer to non-inductive case TSC Workshop, May 2008, U Oklahoma

  23. non-inductive equation (defun snoc (x xs) (if (endp xs) (list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1 x2 …)) = (cons x1 (snoc x (list x2 …))) inductive equation Solutions (defun rac2 (xs) (rac (rdc xs))) (defun rdc2 (xs) (rdc (rdc xs))) • Predicate-based test of rdc (defconst *num-tests* 10) (defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (rdc (snoc x xs)) xs)) TSC Workshop, May 2008, U Oklahoma

  24. non-inductive equation (defun snoc (x xs) (if (endp xs) (list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1 x2 …)) = (cons x1 (snoc x (list x2 …))) inductive equation Solutions (defun rac2 (xs) (rac (rdc xs))) (defun rdc2 (xs) (rdc (rdc xs))) • Predicate-based test of rdc (defconst *num-tests* 10) (defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (rdc (snoc x xs)) xs)) Let’s run this test TSC Workshop, May 2008, U Oklahoma

  25. non-inductive equation (defun snoc (x xs) (if (endp xs) (list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1 x2 …)) = (cons x1 (snoc x (list x2 …))) inductive equation Solutions (defun rac2 (xs) (rac (rdc xs))) (defun rdc2 (xs) (rdc (rdc xs))) • Predicate-based test of rdc (defconst *num-tests* 10) (defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (rdc (snoc x xs)) xs)) Let’s run this test (defthm rdc-snoc-identity-thm (equal (rdc (snoc x xs)) xs)) Must be a theorem, eh? TSC Workshop, May 2008, U Oklahoma

  26. non-inductive equation (defun snoc (x xs) (if (endp xs) (list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1 x2 …)) = (cons x1 (snoc x (list x2 …))) inductive equation Solutions (defun rac2 (xs) (rac (rdc xs))) (defun rdc2 (xs) (rdc (rdc xs))) • Predicate-based test of rdc (defconst *num-tests* 10) (defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (rdc (snoc x xs)) xs)) Let’s run this test (defthm rdc-snoc-identity-thm (equal (rdc (snoc x xs)) xs)) Must be a theorem, eh? TSC Workshop, May 2008, U Oklahoma Maybe ACL2 can prove it

  27. non-inductive equation (defun snoc (x xs) (if (endp xs) (list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1 x2 …)) = (cons x1 (snoc x (list x2 …))) inductive equation Solutions (defun rac2 (xs) (rac (rdc xs))) (defun rdc2 (xs) (rdc (rdc xs))) • Predicate-based test of rdc (defconst *num-tests* 10) (defproperty rdc-snoc-identity *num-tests* ((n (random-int-between 0 100) (integerp n)) (x (random-symbol) (symbolp x)) (xs (random-list-of (random-symbol) n) (symbol-listp xs))) (equal (rdc (snoc x xs)) xs)) Maybe not ! Let’s run this test (defthm rdc-snoc-identity-thm (equal (rdc (snoc x xs)) xs)) Must be a theorem, eh? TSC Workshop, May 2008, U Oklahoma What if xs isn’t a true-list?

  28. non-inductive equation (defun snoc (x xs) (if (endp xs) (list x) ; (snoc x nil) = (list x) (cons (car xs) (snoc x (cdr xs))))) ; (snoc x (list x1 x2 …)) = axioms (cons x1 (snoc x (list x2 …))) • Properties derivable from axioms inductive equation ACL2 intrinsic: true-listp (true-listp xs) iff xs=nil or (true-listp (cdr xs)) theorems Solutions (defun rac2 (xs) (rac (rdc xs))) (defun rdc2 (xs) (rdc (rdc xs))) (defthm rac-snoc-identity-thm (implies (true-listp xs) (equal (rac (snoc x xs)) x)))) (defthm rdc-snoc-identity-thm (implies (true-listp xs) (equal (rdc (snoc x xs)) xs)) Need true-list hypothesis, so state theorem as implication TSC Workshop, May 2008, U Oklahoma

  29. formula denoting something closer to non-inductive case Structural induction (HtDP design recipe): when "obviously" closer to non-inductive case How to Write Inductive Definitionsaka “recursive functions” (defun snoc (x xs) ; cons, backwards (if (endp xs) (list x) (cons (car xs) (snoc x (cdr xs))))) • Three rules • Cover all cases • Specify correct result in each case • One formula for each case • Ensure definition is computational • At least one non-inductive case • All inductive cases make progress towards a non-inductive case Follow these rules … write defect-free software (defun f (x) (if (non-inductive-case x) (non-inductive-formula x) (some-other-op (f closer-to-non-inductive-case)))) TSC Workshop, May 2008, U Oklahoma

  30. Some Intrinsic Operatorshttp://www.cs.utexas.edu/users/moore/acl2/v3-3/PROGRAMMING.html Session 02 Lab:http://www.cs.ou.edu/~rlpage/tsc/ • List operators • cons, car, cdr • list, len, Nth, take, NthCdr, append, reverse • equal, true-listp, null, atom, consp , endp • Numeric operators • Integer, rational, and complex arithmetic: +, -, *, / • Integer division: mod, floor, ceiling, round, truncate • All require dividend and divisor: (floor 8 3) = 2 • Operands may be rationals, but quotients are integers • =, /= numeric operands only • <, <=, >=, >, abs, min, max integer or rational operands only • zp, posp natural number operand only • Recognizers: natp, integerp, rationalp, acl2-numberp • Character and string operators • char-upcase, char-downcase, char-code, code-char character operands • char<, char<=, …, char-equal character operands • string-append, subseq, string-upcase string operands • string<, string<=, …, string-equal string operands • Converters: (coerce list-of-chars ‘String), (coerce string ‘List) • Recognizers: characterp, stringp TSC Workshop, May 2008, U Oklahoma

  31. The End TSC Workshop, May 2008, U Oklahoma

More Related