A Practical Approach to Programming With Assertions by David Rosenblum

A Practical Approach toProgramming With Assertionsby David Rosenblum Sameh Elghzali Fall 2009

Outline • Overview of data assertions and why programming with assertions is a great idea? • History of data assertions • Impact of data assertions on software products quality • The tool under study • The data assertion process • Discussion and questions

Programming With Assertions (1/2) • What is data assertion? • Definition: Assertions are formal constraints on software system behavior that are commonly written as annotations of a source text • The primary goal in writing assertions is to specify what a system is supposed to do rather than how it is to do it

Programming With Assertions (2/2) • In addition to their formal verifications, assertions have long been recognized as a potential powerful tool for automatic runtime detection of software faults during debugging, testing and maintenance • More recently, assertion have been viewed as a permanent defensive programming mechanism for runtime fault detection in production versions of software systems

Why Programming with Assertions Is a Great Idea? (1/3) • Assertions provide run-time checks of assumptions that you would have otherwise put in code comments • An assert statement simply checks a Boolean condition, and does nothing if it is true but immediately terminates the program if it is false • Assert statements serve as test code integrated directly into your implementation

Why Programming with Assertions Is a Great Idea? (2/3) • An assertion allows you to express in code what you assume to always be true about data at a particular point in execution • You probably already write comments throughout your code to document non-obvious assumptions about the values of data structures, so why don't you instead write it into assert statements

Why Programming with Assertions Is a Great Idea? (3/3) • When you are writing code filled with assert statements, you can be more confident that when execution reaches a certain point, certain conditions are guaranteed to be met • When you are debugging code filled with assert statements, failures appear earlier and closer to the locations of the errors, which make them easier to diagnose and fix

Outline • Overview of data assertions and why programming with assertions is a great idea? • History of data assertions • Impact of data assertions on software products quality • The Tool under study • The data assertion process • Discussion and questions

History of Data Assertions (1/3) • Alan Turing advocated use of assertions in 1950 at a conference in Cambridge, he gave a short talk entitled “Checking a Large Routine” which explains the idea with great clarity • He explained that the programmer should make a number of definite assertions which can be checked individually, and from which the correctness of the whole program easily follows

History of Data Assertions (2/3) • Floyd demonstrated the need for loop assertions for verification of programs in 1967 • Luckham et al. elaborated around 1975 the basic principles outlined by Floyd into an algorithm for mechanical program verification that was based on the generation and proof of simple assertions called verification conditions • this algorithm was implemented in the Stanford Pascal Verifier around 1979

History of Data Assertions (3/3) • At the international conference on reliable software in 1975, several authors described systems for deriving runtime consistency checks from simple assertions • Today, assertion features are available as programming language extensions, as programming language features, and in complete high-level formal specification languages

Data Assertion Tools • Many data assertion tools been developed and used for multiple software languages such as Sather, Eiffel, Fortran, Ada, C, C++, Java and others • ADL (Assertion Definition Language) is an example of data assertion tool for Jave and C++ code • APP (Annotation Pre-Processor) is data assertion tool for C language is used in this presentation as an example

Outline • Overview of data assertions and why programming with assertions is a great idea? • History of Data Assertions • Impact of data assertions on software products quality • The tool under study • The data assertion process • Discussion and questions

Impact of Data Assertions on Software Products Quality • In Perry and Evangelist’s empirical study, it was shown that most software faults are interface faults • Perry and Evangelist identified 15 different kinds of interface faults in the software change request data that they analyzed • APP was initially designed to process assertions on function interfaces and simple assertions in function body • APP can help to diagnose and resolve most of the interface and other functional body faults

Perry and Evangelist's Classifications of Interface Fault

APP - Annotation Pre-Processor for C Code (1/2) • A method of generating consistency checks from annotations on variables, subprograms and exceptions • A method that uses incremental theorem providing to check algebraic specifications at run time • A method of generating consistency checks that run in parallel in respect to the execution of the underlying system

APP - Annotation Pre-Processor for C Code (2/2) • provide complete flexibility in specifying how violated assertions are handled at runtime and how much or how little checking is to be performed each time a self-checking program is executed • Does not require complete specifications for its correct operations, and the assertions one writes for APP typically are not complete specifications in any formal sense

APP Assertion Constructs (1/2) • APP recognizes assertions that appear as annotations of C source text • Assertions are written inside comment regions using the extended comment indicators /*@ ... @*/ • Informal comments can be written in an assertion region by writing each comment between the delimiter // and the end of the line

APP Assertion Constructs (2/2) • The constraint is specified using C’s expression language, with the C convention that an expression evaluating to zero is false, while a nonzero expression is true • Quantification over finite domains can be specified using a syntax that is similar to C’s syntax for for loops

APP Recognized Assertions Constructs Associated with functional interface • Assume Specifies a precondition on a function. • Promise Specifies a post condition on a function • Return Specifies a constraint on the return value of a function • Assert Specifies a constraint on an intermediate state of a function body. Associated with functional Body

Example of Data Assertions (Functional Interface) Begin of assertion clause int square_root(x) int x /*@ Assume x >=0; Return y where y>=0 Return y where y*y <=x && x< (y+1)*(y+1); @*/ { …… } Precondition constraints that input must be >0 constraints on the expected resulted Values from the function as indicated by keyword return End of assertion clause Begin of function

Example of Data Assertions (Functional Body) Void swap(x, y) Int* x; Int* y; /*@ Assume x && y && x!=y; Promise *x == in *y; Promise *y == in *x; @*/ { *x == *y ^ *x; *y == *x ^ *y; /*@ Assert *y == in *x. @*/ *x == *y ^ *x; } Examples of functional Interface assertions Begin of function Functional body assertion

Violation Actions, Predefined Macros and Severity Levels (1/2) • APP converts each assertion to a runtime check, which tests for the violation of the constraint specified in the assertion • If the check fails at runtime, then additional code generated with the check is executed in response to the failure • The default response code generated by APP prints out a simple diagnostic message • Use some preprocessor macros that are predefined by APP, can enhance the presentation and diagnoses of failures

Violation Actions, Predefined Macros and Severity Levels (2/2) • APP supports the specification of an optional severity level for each assertion, with 1 being the default and indicating the highest severity • A severity level indicates the relative importance of an assertion and determines whether or not the assertion will be checked at runtime • Severity levels can be used to control the amount of assertion checking that is performed at runtime without recompiling the program to add or remove checks

Generating and Running Self-checking Programs • APP translates an input annotated C program into an equivalent C program with embedded assertion checks • APP has the same command-line interface as cpp, the standard pre-processor pass of C compilers • Standard build tools such as make and nmake can be used to build executable self-checking programs

A Classification of Assertion • Specification of function interfaces The primary goal of specifying a function interface is to ensure that the arguments, return value and global state are valid with respect to the intended behavior of the function; the common characteristic of all function interface constraints is that they are stated independently of any implementation for the function. The constraints are special forms of traditional preconditions and post-conditions • Specification of function bodies Assertions that are stated in terms of a particular function implementation can be used as “enforced comments” to guard against such faults in the function body

Specification of Function Interfaces(1/3) • Consistency between arguments For each function in the system, specify how the value of each of its arguments depends on the values of its other arguments • Dependency of return value on arguments Function in the system, provide post-conditions that specify how its return value(s) depends on the values of its arguments • Effect on global state For each function in the system, specify what changes the function makes to the values of the global variables that are visible to it

Specification of Function Interfaces(2/3) • The context in which a function is called For each function in the system, specify how the values of its arguments and the values of the global variables visible to it govern when it is valid for the function to be called • Frame specifications For each function in the system, specify each case when the value of an argument passed to the function by reference, or the value of a global variable visible to the function, is to be left unchanged by the function • Non-null pointers For each function in the system, specify which pointer-valued arguments, return value(s) and global variables must not be null

Specification of Function Interfaces (3/3) • Sub-range membership of data For each function in the system, specify all sub-range constraints on the values of its arguments, return value(s) and global variables that are of numeric type. Also specify all sub-range constraints on the values used to index its array-valued arguments, return value(s) and global variables • Enumeration membership of data For each function in the system, specify all membership constraints on the values of its arguments, return value(s) and global variables that are of enumeration type

Specification of Function Bodies (1/2) • Condition of the else part of complex if Statements For each if statement in the system that contains a final else part, explicitly specify the implicit condition of the final else part as an initial assertion in that part • Condition of the default case of a switch statement For each switch statement in the system that contains a default case, explicitly specify the implicit condition of the default case as an initial assertion in that case. For each switch statement without a default case, provide a default case containing an assertion that always evaluates to false

Specification of Function Bodies (2/2) • Consistency between related data For each function body in the system, specify consistency constraints on mutually dependent data at frequent intervals within the code that manipulates that data • Intermediate snapshot of computation For each function body in the system, specify at frequent intervals the key constraints the function body must satisfy

Summary Classifications of Assertions

Effective Assertions for Detecting Interface Faults Effective assertions in the empirical study was found to identify and diagnose most of the faults identified by Perry and Evangelist for the interface faults.

Conclusions (1/2) • APP provides a rich collection of features for specifying not only the assertions themselves but also the responses to failed runtime assertion checks • APP fits easily into the process of developing C programs • The assertion checks generated by APP introduce negligible time and space overhead into the generated self-checking program • APP provides severity levels in order to give the specifier greater control over the amount of assertion checking that is performed at runtime

Conclusion (2/2) • APP will be extended to support other kinds of assertions and higher-level abstraction facilities • APP new features will include constraints on types and global variables, a richer abstraction of arrays and other abstract data types, and constructs for specifying interactions between program units that are larger than functions • Assertion checking is a powerful alternative for identifying and removing faults in large applications and it is powerful, practical, scalable and simple to use

Discussion and Questions

A Practical Approach to Programming With Assertions by David Rosenblum