A Pragmatic View of Formal Methods The Hi-Lite Project

1. A Pragmatic View of Formal MethodsThe Hi-Lite Project Robert B K Dewar President & CEO, AdaCore Emeritus Professor of Computer Science, New York University SSS ‘11 Feb 10th 2011

2. An observation from Yogi Berra • In theory there is no difference between theory and practice. In practice there is. • Variously attributed to Yogi Berra, Einstein etc.

4. Correctness and Reliability • Not really the same thing • Often merged in people’s thinking • The issue of large reliable programs • Correctness means adherence to some formal model • Reliability means that a program works • Many reasons why a correct program might be unreliable. • And a reliable program may well be incorrect

5. We can do Pretty Well in Practice • No one has ever died on a commercial airplane as a result of a software bug. • We have however had some hair raising close calls • But a close call is a close call • When it comes to avionics, software is not the weak link in the chain. • But we could do better • And most certainly best practices are not widely applied • Cars • Medical Equipment • Critical financial applications

6. Safety And Security • Safety means we don’t have bugs that kill people • Security means bad guys can’t cause chaos • We have lots of bad guys around who know a lot • Any safety critical system is security critical now • How vulnerable are we • TV thinks we are very vulnerable • Alias and Avionics systems • 24 and Air Traffic Control • Don’t be too quick to dismiss fiction • Clancy crashed a plane long before 9/11

7. Proving Entire Programs Correct • A nice conceptual goal • But definitely a case where theory and practice are not on the same page • Fundamental problems stand in the way

8. Producing a Full Formal Specification • No use unless we find an appropriate level of abstraction. • Every program is a full formal specification in some sense, but that’s not useful • Every generation of programming languages has been sold as high level specification eliminating the need for programmers • Fortran, mathematicians will enter formulas directly • COBOL, people will stick business models into the computer directly • 4G languages, no, honestly, they really will stick them in directly • Modeling languages, no more need for expensive programmers • Expensive, but might be worth the cost if it was a panacea, but it is not.

9. Difficulties in Producing Full Specifications • No one understands formal specifications • There are things that can’t be formally specified • Good error messages in a compiler • Intuitive user interfaces • Deliberate incompleteness • Difficulties of proving termination • Difficulties of specifying programming languages • Anyway, they will have errors • A specification is basically a large program • And often one that has never been tested

10. Proving Properties of Programs • Definitely a promising approach • Not just promises, we can deliver • Freedom from run-time errors • Proof of security properties • Security a particular important target • Secure programs do not have to work • But they do have to be secure

11. The Role of Testing • Right now, we largely rely on testing, e.g. in DO-178B • Thorough testing definitely improves our confidence • But testing is fundamentally incomplete • DO-178B has no formal model, it just works • So can we eliminate testing? • Almost no one thinks we can eliminate 100% of testing • I don’t care how much you have proved, I don’t want to fly on a plane with avionics that has never been tested. • But if we can’t eliminate ANY testing, then bleak times ahead for formal methods, certainly in the safety area. • So the idea (enshrined in DO-178C) is to figure out how to give some credit for formal methods, eliminating some testing requirements.

12. Proofs Replacing Testing • Specific Unit tests might be replaced by proofs, e.g. of the correctness of an algorithm, eliminating the need for complete domain testing. • Object Oriented Programs and Dynamic Dispatch • Thorough testing impractical (every call to every method). • Require LSP in OO hierarchies • OK, we can require it, but how to verify? • Unit testing??? • This is a place where proof techniques do make sense

13. Reasoning About Existing Programs • The use of static analysis • Finding bugs is useful, but not so useful • Guaranteeing absence of bugs is definitely useful • But eliminating all false negatives can generate too many false positives • Helps to start with an appropriate language, e.g. in Ada, range declarations add a focus for analysis. • The Microsoft Vista story • Added 500,000 annotations • Found 100,000 potential buffer overflows • Finding pre and post conditions • Incremental Analysis

14. Putting it All Together, the Hi-Lite Project • In practice we will make use of a combination of techniques and tools • Proof of correctness and formal specification • Thorough testing • Use of static analysis tools • All these tools can benefit from the addition of annotations about what should be going on. • Postconditions • Preconditions • Invariants • But different tools have different views • Is a precondition something to be proved? • Or is it something to be tested? • Or is it there to give hints to a static analyzer

15. Preconditions and Postconditions • At one level, they are just formalizations of comments. Comments are a good thing if • They are precise • They are complete • They are kept up to date • But out of date or incorrect comments are a menace • So formalizing them can only help • Not that much work once you get used to it • But what language should be used • Ada 2012 adds extensive capabilities for annotations including preconditions, postconditions, invariants, and predicates.

16. Trying to Unify the Language Requirements • If annotations/contracts are useful for proof, testing, and static analysis, how about we use the same annotation language, so we can integrate the different approaches within a common framework? • Multiple approaches are problematic • In AdaCore technology, four syntaxes for preconditions • Precondition pragma for Ada 95/2005 • Pre aspect in Ada 2012 • SPARK annotation for preconditions • CodePeer syntax for discovered precondition

17. Hi-Lite • One important goal of Hi-Lite is to create a unified language for annotations that can be used in all contexts. • Then the second step is to create a unified development environment in which different tools can use the same source base in different contexts. • The goal: to facilitate (i.e. make cheaper) the integration of formal methods and testing regimes. • We need to create industrial workflows that incorporate verification on a routine basis • These tools need to integrate different programming approaches as well (different languages, modeling …)

18. Problems in the Unification • The requirements for mathematical specifications and programming language expressions are different. • Does A+B mean A+B or does it mean apply some weird programming notion of + with overflows or wrap around? • with Pre => A + B <= Integer’Last • Does A or B mean A or B, or does it mean do some strange evaluation in which an overflow in B causes an exception even if A is True? • Not insuperable, but such issues stand in the way of this being a simple job of getting people to agree on syntax.

19. Conclusion • We have a lot of industrial experience to feed in here • We have creators of tools and users very interested in participating in creating a workable framework • AdaCore • Praxis • CEA • INRIA • We have mportant industrial partners who need this technology now, so this is much more than just research. • Altran • Astrium • Thales • Rockwell