Teaching the construction of correct programs using invariant based programming

1. Invariant Based Programming, March 2010 Ralph-Johan Back Johannes Eriksson Linda Mannila Teaching the construction of correct programs usinginvariant based programming Åbo Akademi / Dept. of Information Technologies Turku, Finland

2. Invariant Based Programming, March 2010 Formal methods in CS education • Formal methods are perceived as difficult and requiring mathematical sophistication • The CS curriculum is divided into “theory” and “practice” • Formal methods taught independently of programming courses • Students get impression that formal methods are not applicable in practice • Testing and debugging is therefore the main (only) programming method that they learn from CS studies

3. Invariant Based Programming, March 2010 Overview of talk • A short introduction to invariant based programming • The Socos tool • Teaching formal methods at Åbo Akademi using invariant based programming • Experience report on a first year course on invariant based programming

4. Invariant Based Programming, March 2010 Constructing correct programs “constructive approach” “a posteriori verification” “invariant based programming” Program code Contracts Invariants Verification conditions

5. Invariant Based Programming, March 2010 A: Int[N] A: Int[N] Sorted(A,0,N) A=A0 Permutation(A,A0) Example: Sort an array! Start with a pre-/postcondition specification

6. Invariant Based Programming, March 2010 Example: Sort an array! Structure according to invariants A: Int[N] A: Int[N] Sorted(A,0,N) A=A0 Permutation(A,A0)

7. Invariant Based Programming, March 2010 sorted un-sorted Example: Sort an array! 0 k N Construct a loop A: Int[N] less than or equal to all A[k..N-1] ! A=A0 Sorted(A,0,N) Permutation(A,A0) k: Int 0≤k≤N LOOP Sorted(A,0,k) ∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j]

8. Invariant Based Programming, March 2010 0: Int 0≤0≤N Sorted(A,0,0) ∀i,j:Int • 0≤i<0 ∧ 0≤j<N ⇒ A[i]≤A[j] Example: Sort an array! What needs to be checked? A: Int[N] Add initial transition A=A0 ⇒ A: Int[N] ✔ A: Int[N] ✔ Permutation(A,A0) Permutation(A,A0) ✔ A=A0 Sorted(A,0,N) ✔ ✔ ✔ k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] k:=0

9. Invariant Based Programming, March 2010 Trivial: Sorted(A,0,k) ∧ k=N ⇒ Sorted(A,0,N) Example: Sort an array! Add exit transition A: Int[N] Permutation(A,A0) A=A0 Sorted(A,0,N) k: Int [k=N] 0≤k≤N Sorted(A,0,k) ∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] k:=0

10. Invariant Based Programming, March 2010 A: Int[N] Permutation(A,A0) k: Int 0≤k≤N Sorted(A,0,k) ∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] A’: Int[N] Permutation(A’,A0) k+1: Int 0≤k+1≤N Sorted(A’,0,k+1) ∀i,j:Int • 0≤i<k+1 ∧ k+1≤j<N ⇒ A’[i]≤A’[j] Example: Sort an array! Add loop transition k<N m=min(A,k,N) ∧ A’= A[ k←A[m], m←A[k] ] A: Int[N] ⇒ Permutation(A,A0) A=A0 Sorted(A,0,N) k: Int [k=N] 0≤k≤N Sorted(A,0,k) ∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] [k<N] k:=0 m:=min(A,k,N); A:=A[ k←A[m], m←A[k] ]; k:=k+1

11. Invariant Based Programming, March 2010 0≤N-k Example: Sort an Array! Add a termination function A: Int[N] Variant decreases: N-(k+1) < N-k Bounded from below: 0≤k≤N ⇒0≤N-k Permutation(A,A0) A=A0 Sorted(A,0,N) k: Int [k=N] 0≤k≤N Sorted(A,0,k) ∀i,j:Int • 0≤i<k ∧ k≤j<N ⇒ A[i]≤A[j] [k<N] k:=0 m:=min(A,k,N); A:=A[ k←A[m], m←A[k] ]; k:=k+1

12. Invariant Based Programming, March 2010 Our research invariant based programming automatic and interactive verification teaching formal methods

13. Invariant Based Programming, March 2010 SOCOS: tool support for IBP

14. Invariant Based Programming, March 2010 Verification in SOCOS • Three types of verification conditions: • Consistency(for transitions) • Liveness(for situations) • Termination(for loops) • Verification conditions are sent to external proof tools • An SMT solver (automatic proofs), • PVS (interactive proof checking)

15. Invariant Based Programming, March 2010 Backends Higher assurance→ Testing Run-time evaluation of invariants Static Checking Verification conditions are sent to Simplify, a fully automatic prover Full Verification PVS is used for full verification of the final components

16. Invariant Based Programming, March 2010 Teaching invariant based programming at Abo Akademi • Spring 2005: Ph.D. course on invariant based programming -- testing out the idea • Spring 2007: a course on IBP for first year students • 2008 -- : IBP now part of standard CS curriculum • Planned next step: teaching IBP as a special math course in high school

17. Invariant Based Programming, March 2010 New first year CS curriculum • Mathematics courses • algebra • probability theory • Computer Science courses • Introduction to CS • Python programming (to show that programming is fun) • Structured derivations (to teach mathematical and logical reasoning) • Invariant based programming (to teach how to construct programs that are correct) • Java programming • Systems design course Formal methods bundle

18. Invariant Based Programming, March 2010 Invariant based programming course (spring 2007) • aimed at first or second year students • interactive, emphasizing student participation • 17 sessions a 90 min • 11 lectures • 6 practical excercises • Socos tool support only used in 4 last sessions • only automatic proofs (Simplify), no PVS proofs • 16 active participants • half with no background in formal methods

19. Invariant Based Programming, March 2010 Collecting data about the course • pre- and postcourse questionairs • observations • hand-in assignments • final exam • 8 students selected for semi-structured interview

20. Invariant Based Programming, March 2010 Main results - 1 • The students found the course • useful, • interesting, • somewhat fun and • of medium difficulty level. • On average, students found invariant based programming • rather easy to learn, • useful in practice and • made the general structure of the program more comprehensible

21. Invariant Based Programming, March 2010 Main results - 2 • Difficulties were mainly in • constructing proofs and • finding the invariant for more complex programs • The programs written by the students show that • they had understood the idea behind IBP, and • were able to construct and prove simple invariant based programs.

22. Invariant Based Programming, March 2010 Main results - 3 • The students appreciated the diagrammatic notation of IBP • most students are visual learners, • textual programming languages or pseudocode may not be the best way for expressing algorithms to these students • We had expected that identifying the invariants would be the most difficult task, but this was not the case • writing proofs by hand seemed to be most problematic, as they required much time and effort • formulating postconditions was also sometimes problematic • Students found that IBP provides good support for finding bugs during the program construction • instead of after the program is ready

23. Invariant Based Programming, March 2010 Main results - 4 • Starting with informal reasoning in the course before introducing the formal framework was not appreciated • the students would have wanted the formal proof obligations to be introduced earlier • it seems that students who are not mathematically mature do not know how to reason ”informally” but first need to learn a formal approach with a fixed set of rules • Socos • supporting a formal method with a computer based tool in the course was very well received • the students preferred SOCOS over pen and paper, as the automation increased productivity. • unfamiliarity with the SOCOS syntax was the main cause of difficulty

24. Invariant Based Programming, March 2010 Thank Youhttp://mde.abo.fi/SOCOS