Invariant Based Programming in Education

1. Invariant Based Programming in Education Tutorial, FM’08 Linda Mannila 26.5.2008

2. Background • Challenges at CS departments • First year students have weak skills in logic, formal reasoning, writing proofs etc. • Students do not see a connection between theoretical and practical courses • Goal • Giving students hands-on skills in constructing proofs and reasoning about correctness early on • Linking theory with practice

3. IBP in Education • Course for PhD students in 2005 • 15 case study sessions with faculty, students, researchers • Undergraduate course, given twice, starting in spring 2007

4. Undergraduate Course • Elective (~15 students/course) • Prerequisities: • Introductory programming • Logic • 17 interactive, example-driven sessions • 4 (7) sessions with SOCOS • Addition, ..., linear search, ..., sorting • Evaluation • questionnaires, interviews, observations, assignments, exam

5. Experienced benefits (+) Introduction to program correctness and formal verification • Good introduction to formal verification and how tools can be used in that context • Helps remove errors in the algorithm that could lead to bugs A practical method for introducing program correctness • IBP summarizes the proof conditions in a good way • IBP is intuitive Introduction to a more abstract view of programming • Learning to think about how a program works in general, without a given programming language • Learned to think about a program as states and transitions instead of merely as transitions as is usually the case More tangible overview of the program structure • Learning to draw a program makes it easier to see its structure • Makes it easy to keep the track of the various parts of a program New and useful contents

6. Experienced difficulties (-) • Syntax and notation • It’s difficult to formulate one’s programs according to the standard • Since I have programmed previously e.g. the Java way of expressing things is quite ingrained • Proofs • Proving programs by hand is very work intense • Proving complex programs is difficult • Finding the correct invariant and postcondition • Using SOCOS

7. Preliminary error analysis • Only minor problems related to the • Method (e.g. the syntax) • Algorithm • Main problem • Weak logical background • Interpreting and constructing quantified expressions • Understanding when to use quantifiers • Understanding which one to use • Problems with logic affect other aspects • Unclear invariants (not necessarily weak) • Unclear proofs

8. Conclusions • Learning IBP is not difficult • Students get the big picture rather quickly • Appreciate the diagrammatic notation • Main difficulties related to predicate logic • More focus on using logic in practice • Students need more practice in reading and constructing quantified expressions • Remedy? • Split the course into two parts: 1) manual and 2) with tool support  More time for practice

9. Assignment Results (2007) • Scores • Average 25.5 (max 40) • Seven students > 30 points • Most common errors • Syntax related (e.g. using Java like syntax or not following the given proof format) • Other errors • Incomplete invariants • Problems with the algorithm • Using undeclared variables • One “off by one” error

13. Error analysis (2008) • Jonas Erikson, Master’s Thesis • Analyzed student solutions to four assignments (all in all ~50 solutions) • Three main error categories: • Syntax (related to the diagram representation) • Reasoning (related to misconceptions and weak understanding) • Logical background (related to having a weak background in expressing logical statements)

14. Total number of errors Nested loops

15. Related to syntax

16. Related to reasoning Nested loops

17. Related to logical background Define own predicates,use quantifiers to a larger extent

18. Discussion • Could IBP be introduced at high school level? • As a course in mathematics? • Would the method need to be simplified? How?