272: Software Engineering Fall 2012

1. 272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 2: Software Verification with JPF, ALV and Design for Verification

2. Model Checking Evolution • Earlier model checkers had their own input specification languages • For example Spin, SMV • This requires translation of the system to be verified to the input langauge of the model checker • Most of the time these translations are not automated and use ad-hoc simplifications and abstractions • More recently several researchers developed tools for model checking programs • These model checkers work directly on programs, i.e., their input language is a programming language • These model checkers use well-defined techniques for restricting the state space or use automated abstraction techniques

3. Explicit-State Model Checking Programs • Verisoft from Bell Labs • C programs, handles concurrency, bounded search, bounded recursion. • Uses stateless search and partial order reduction. • Java Path Finder (JPF) at NASA Ames • Explicit state model checking for Java programs, bounded search, bounded recursion, handles concurrency. • Uses techniques similar to the techniques used in Spin. • CMC from Stanford for checking systems code written in C

4. Symbolic Model Checking of Programs • CBMC • This is the bounded model checker we discussed earlier, bounds the loop iterations and recursion depth. • Uses a SAT solver. • SLAM project at Microsoft Research • Symbolic model checking for C programs. Can handle unbounded recursion but does not handle concurrency. • Uses predicate abstraction and BDDs.

5. Java Path Finder • Program checker for Java • Properties to be verified • Properties can be specified as assertions • static checking of assertions • It can also verify LTL properties • Implements both depth-first and breadth-first search and looks for assertion violations statically • Uses static analysis techniques to improve the efficiency of the search • Requires a complete Java program • It can only handle pure Java, it cannot handle native code

6. Java Path Finder, First Version • First version • a translator from Java to PROMELA • Use SPIN for model checking • Since SPIN cannot handle unbounded data • Restrict the program to finite domains • A fixed number of objects from each class • Fixed bounds for array sizes • Does not scale well if these fixed bounds are increased • Java source code is required for translation

7. Java Path Finder, Current Version • Current version of the JPF has its own virtual machine: JVM-JPF • Executes Java bytecode • can handle pure Java but can not handle native code • Has its own garbage collection • Stores the visited states and stores current path • Offers some methods to the user to optimize verification • Traversal algorithm • Traverses the state-graph of the program • Tells MC-JVM to move forward, backward in the state space, and evaluate the assertion • The rest of the slides on the current version of JPF

8. Storing the States • JPF implements a depth-first search on the state space of the given Java program • To do depth first search we need to store the visited states • There are also verification tools which use stateless search such as Verisoft • The state of the program consists of • information for each thread in the Java program • a stack of frames, one for each method called • the static variables in classes • locks and fields for the classes • the dynamic variables (fields) in objects • locks and fields for the objects

9. Storing States Efficiently • Since different states can have common parts each state is divided to a set of components which are stored separately • locks, frames, fields • Keep a pool for each component • A table of field values, lock values, frame values • Instead of storing the value of a component in a state store an index at which the component is stored in the table in the state • The whole state becomes an integer vector • JPF collapses states to integer vectors using this idea • This strategy enables JPF to collapse and uncollapse parts of the states during the state space exploration

10. State Space Explosion • State space explosion if one of the major challenges in model checking • The idea is to reduce the number of states that have to be visited during state space exploration • Here are some approaches used to attack state space explosion • Symmetry reduction • search equivalent states only once • Partial order reduction • do not search thread interleavings that generate equivalent behavior • Abstraction • Abstract parts of the state to reduce the size of the state space

11. Symmetry Reduction • Some states of the program may be equivalent • Equivalent states should be searched only once • Some states may differ only in their memory layout, the order objects are created, etc. • these may not have any effect on the behavior of the program • JPF makes sure that the order which the classes are loaded does not effect the state • There is a canonical ordering of the classes in the memory • A similar problem occurs for location of dynamically allocated objects in the heap • If we store the memory location as the state, then we can miss equivalent states which have different memory layouts • JPF tries to remedy this problem by storing some information about the new statements that create an object and the number of times they are executed

12. Partial Order Reduction • Statements of concurrently executing threads can generate many different interleavings • all these different interleavings are allowable behavior of the program • A model checker has to check all possible interleavings that the behavior of the program is correct in all cases • However different interleavings may generate equivalent behaviors • In such cases it is sufficient to check just one interleaving without exhausting all the possibilities • This is called partial order reduction

13. state space search generates 258 states with symmetry reduction: 105 states with partial order reduction: 68 states with symmetry reduction + partial order reduction : 38 states class S1 { int x;} class FirstTask extends Thread { public void run() { S1 s1; int x = 1; s1 = new S1(); x = 3; }} class Main { public static void main(String[] args) { FirstTask task1 = new FirstTask(); SecondTask task2 = new SecondTask(); task1.start(); task2.start(); }} class S2 { int y;} class SecondTask extends Thread { public void run() { S2 s2; int x = 1; s2 = new S2(); x = 3; }}

14. Action Language and Action Language Verifier • Now, I want to talk about an infinite state model checker called Action Language Verifier (ALV) • ALV is like Spin and SMV, it has its own input language called Action Language • So, to verify something using ALV you first have to specify it in Action Language

15. Action Language • Actions specify state changes (transitions) • States correspond to valuations of variables • boolean • enumerated • integer (possibly unbounded) • heap variables (i.e., pointers) • Parameterized constants • specifications are verified for every possible value of the constant • Parameterized specifications • Enable verification of a protocols for arbitrary number of processes

16. Action Language • Transition relation is defined using actions • Atomic actions: Predicates on current and next state variables • Action composition: • asynchronous (|) or synchronous (&) • Modular • Modules can have submodules • A module is defined as asynchronous and/or synchronous compositions of its actions and submodules

17. Actions in Action Language • Atomic actions: Predicates on current and next state variables • Current state variables: reading, nr, busy • Next state variables: reading’, nr’, busy’ • Logical operators: not (!) and (&&) or (||) • Equality: = (for all variable types) • Linear arithmetic: <, >, >=, <=, +, * (by a constant) • An atomic action: !reading and !busy and nr’=nr+1 and reading’

18. What can we specify in Action Language? • For example, we can specify a Read-Write lock implementation • Then using Action Language Verifier, we can check if this read write lock satisfies a CTL property (like mutual-exclusion) integer nr; boolean busy; initial: !busy and nr=0; r_enter: [!busy] nr := nr+1; r_exit: nr := nr-1; w_enter: [!busy && nr=0] busy := true; w_exit: busy := false;

19. module main() integernr; booleanbusy; restrict: nr>=0; initial: nr=0 and !busy; module ReaderWriter() enumeratedstate {idle, reading, writing}; initial: state=idle; r_enter: state=idle and !busy and nr’=nr+1 and state’=reading; r_exit: state=reading and nr’=nr-1 and state’=idle; w_enter: state=idle and !busy and nr=0 busy’ and state’=writing; w_exit: state=writing and !busy’ and state’=idle; ReaderWriter: r_enter | r_exit | w_enter | w_exit; endmodule main: ReaderWriter*(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule Read-Write Lock in Action Language S : Cartesian product of variable domains defines the set of states I : Predicates defining the initial states R : Atomic actions of a single process R : Transition relation of a process, defined as asynchronous composition of its atomic actions R : Transition relation of main, defined as asynchronous composition of finite but arbitrary number of reader-writer modules

20. Arbitrary Number of Threads? • How do we check arbitrary number of threads? • Counting abstraction • Create an integer variable for each thread state • Each variable counts the number of threads in a particular state • Generate updates and guards for these variables based on the specification • Local states of the threads have to be finite • Shared variables can be unbounded • Counting abstraction is automated

21. Parameterized Read-Write Lock module main() integer nr; boolean busy; parameterized integer numReaderWriter; restrict: nr>=0 and numReaderWriter>=1; initial: nr=0 and !busy; module ReaderWriter() integer idle, reading, writing; initial: idle=numReaderWriter; r_enter: idle>0 and !busy and nr’=nr+1 and idle’=idle-1 and reading’=reading+1; r_exit: reading>0 and nr’=nr-1 and reading’=reading-1 and idle’=idle+1; w_enter: idle>0 and !busy and nr=0 and busy’ and idle’=idle-1 and writing’=writing+1; w_exit: writing>0 and !busy’ and writing’=writing-1 and idle’=idle+1 ReaderWriter: r_enter | r_exit | w_enter | w_exit; endmodule main: ReaderWriter(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule

22. Action Language Verifier • Action Language Verifier is an infinite state model checker that can verify properties of systems specified in Action Language • The input Action Language specification is represented as a transition system: • S : The set of states • I  S : The set of initial states • R  S  S: The transition relation • Properties of the input specification are expressed in temporal logics • Invariant(p) : is true in a state if property p is true in every state reachable from that state • Also known as AG • Eventually(p) : is true in a state if property p is true at some state on every execution path from that state • Also known as AF

23. Symbolic Model Checking Given a program and a temporal propertyp: • Either show that all the initial states satisfy the temporal propertyp • set of initial states  truth set of p • Or find an initial state which does not satisfy the propertyp • a state  set of initial states  truth set of p • We can check these in two ways: • Starting from the initial states and iteratively add states that are reachable from the current set of reachable states. We stop when there is nothing new to add. (This is called forward fixpoint computation). This computes all the reachable states. OR • Start from the bad states (p)and iteratively add states that can reach the current set of states. We stop when there is nothing new to add. (This is called backward fixpoint computation). This computes all the states that can reach a bad state.

24. Symbolic Model Checking Computes Fixpoints Pre-condition (backwardImage) of p Backward fixpoint • • • p Initial states initial states that violate Invariant(p) states that can reach p i.e., states that violate Invariant(p) Invariant(p) Forward fixpoint • • • Initial states p reachable states that violate p Post-condition (forward image) of initial states reachable states of the system

25. Symbolic Model Checking • Represent sets of states and the transition relation as logic formulas • Forward and backward fixpoints can be computed by iteratively manipulating these formulas • Pre- and Post-condition computation (aka Forward, backward image): Existential variable elimination • Conjunction (intersection), disjunction (union) and negation (set difference), and equivalence check • Requires use an efficient data structures for manipulation of logic formulas

26. Fixpoints May Not Converge • For infinite state systems fixpoint computations may not converge (there is always something new to add, so we never stop). • In fact many verification problems are undecidable for infinite state systems • So we use conservative approximations

27. Conservative Approximations • Compute a lower ( p ) or an upper ( p+ ) approximation to the truth set of the property ( p ) • Action Language Verifier can give three answers: p p p I p I 1) “The property is satisfied” 3) “I don’t know” sates which violate the property p p+  p I 2) “The property is false and here is a counter-example”

28. Action Language Verifier Action Language Tool Set Action Language Specification Action Language Parser Composite Symbolic Library Omega Library CUDD Package MONA Verified Don’t know Counter example Presburger Arithmetic Manipulator BDD Manipulator Automata Manipulator

29. Read-Write Lock Verification with ALV

30. Read-Write Lock in Java class ReadWriteLock {private Object lockObj;private int totalReadLocksGiven;private boolean writeLockIssued;private int threadsWaitingForWriteLock;public ReadWriteLock() {    lockObj = new Object();    writeLockIssued = false;  }  public void getReadLock() {synchronized (lockObj) {while ((writeLockIssued) || (threadsWaitingForWriteLock != 0)) {try {          lockObj.wait();        } catch (InterruptedException e) {        }      }      totalReadLocksGiven++;    }  }public void getWriteLock() {synchronized (lockObj) {      threadsWaitingForWriteLock++;while ((totalReadLocksGiven != 0) || (writeLockIssued)) {try {          lockObj.wait();        } catch (InterruptedException e) {          //        }      }      threadsWaitingForWriteLock--;      writeLockIssued = true;    }  }public void done() {synchronized (lockObj) {      //check for errorsif ((totalReadLocksGiven == 0) && (!writeLockIssued)) {        System.out.println(" Error: Invalid call to release the lock");return;      }if (writeLockIssued)        writeLockIssued = false;else        totalReadLocksGiven--;      lockObj.notifyAll();    }  }} How do we translate this to Action Language? Action Language Verifier Verification of Synchronization in Java Programs

31. Two Challenges in Software Model Checking • State space explosion • Exponential increase in the state space with increasing number of variables and threads • State space includes everything: threads, variables, control stack, heap • Environment generation • Finding models for parts of software that are • either not available for analysis, or • are outside the scope of the model checker

32. Modular Verification • Modularity is key to scalability of any verification technique • Moreover, it can help in isolatingthe behavior you wish to focus on, removing the parts that are beyond the scope of your verification technique • Modularity is also a key concept for successful software design • The question is finding effective ways of exploiting the modularity in software during verification

33. Interfaces for Modularity • How do we do modular verification? • Divide the software to a set of modules • Check each module in isolation • How do we isolate a module during verification/testing? • Provide stubs representing other modules (environment) • How do we get the stubs representing other modules? • Write interfaces • Interfaces specify the behavior of a module from the viewpoint of other modules • Generate stubs from the interfaces

34. Interfaces and Modularity: Basic Idea • Write interface specifications for the modules • Automatically generate stubs from the interface specifications • Automatically generated stubs provide the environment during modular verification

35. A Design for Verification Approach Our design for verification approach is based on the following principles: • Use of design patterns that facilitate automated verification • Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification • An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications • A general model checking technique for interface verification • Domain specific and specialized verification techniques for behavior verification

36. Controller GuardedCommand Shared StateMachine SharedStub ControllerStateMachine Controller +action1() +action2() +a() +b() +a() +b() -var1 -var2 +action1() +action2() Action +blocking() +nonblocking() -GuardedExecute GuardedCommand +guard() +update() Concurrency Controller Pattern ThreadA Shared ThreadB Helper classes int used at runtime used during interface verification used both times

37. Concurrency Controller Pattern • Avoids usage of error-prone Java synchronization primitives: synchronize, wait, notify • Separates controller behavior from the threads that use the controller • Supports a modular verification approach that exploits this modularity for scalable verification

38. class Action{ protected final Object owner; … private boolean GuardedExecute(){ boolean result=false; for(int i=0; i<gcV.size(); i++) try{ if(((GuardedCommand)gcV.get(i)).guard()){ ((GuardedCommand)gcV.get(i)).update(); result=true; break; } }catch(Exception e){} return result; } public void blocking(){ synchronized(owner) { while(!GuardedExecute()) { try{owner.wait();} catch (Exception e){} } owner.notifyAll(); } } public boolean nonblocking(){ synchronized(owner) { boolean result=GuardedExecute(); if (result) owner.notifyAll(); return result; } } } class RWController implements RWInterface{ int nR; boolean busy; final Action act_r_enter, act_r_exit; final Action act_w_enter, act_w_exit; RWController() { ... gcs = new Vector(); gcs.add(new GuardedCommand() { public boolean guard(){ return (nR == 0 && !busy);} public void update(){busy = true;}} ); act_w_enter = new Action(this,gcs); } public void w_enter(){ act_w_enter.blocking();} public boolean w_exit(){ return act_w_exit.nonblocking();} public void r_enter(){ act_r_enter.blocking();} public boolean r_exit(){ return act_r_exit.nonblocking();} } This helper class is provided. No need to rewrite it! Reader-Writer Controller

39. A controller interface defines the acceptable call sequences for the threads that use the controller Interfaces are specified using finite state machines public class RWStateMachine implements RWInterface{ StateTable stateTable; final static int idle=0,reading=1,writing=2; public RWStateMachine(){ ... stateTable.insert("w_enter",idle,writing); } public void w_enter(){ stateTable.transition("w_enter"); } ... } Controller Interfaces reading r_enter r_exit idle w_exit writing w_enter

40. Modular Design / Modular Verification Thread Modular Interface Verification Concurrent Program Thread 1 Thread 2 Thread n Thread 1 Thread 2 Thread n Interface Machine Interface Machine Interface Machine Interface Controller Modular Behavior Verification Shared Data Controller Behavior

41. Behavior Verification • Analyzing properties (specified in CTL) of the synchronization policy encapsulated with a concurrency controller and its interface • Verify the controller properties assuming that the user threads adhere to the controller interface • Behavior verification with Action Language Verifier • We wrote a translator which translates controller classes to Action Language • Using counting abstraction we can check concurrency controller classes for arbitrary number of threads

42. Interface Verification • A thread is correct with respect to an interface if all the call sequences generated by the thread can also be generated by the interface machine • Checks if all the threads invoke controller methods in the order specified in the interfaces • Checks if the threads access shared data only at the correct interface states

43. Interface Verification • Interface verification with Java PathFinder • Verify Java implementations of threads • Correctness criteria are specified as assertions • Look for assertion violations • Assertions are in the StateMachine and SharedStub • Performance improvement with thread Isolation • thread modular verification

44. Thread Isolation: Part 1 • Interaction among threads • Threads can interact with each other in only two ways: • invoking controller actions • Invoking shared data methods • To isolate the threads • Replace concurrency controllers with controller interface state machines • Replace shared data with shared stubs

45. Thread Isolation: Part 2 • Interaction among a thread and its environment • Modeling thread’s calls to its environment with stubs • File I/O, updating GUI components, socket operations, RMI call to another program • Replace with pre-written or generated stubs • Modeling the environment’s influence on threads with drivers • Thread initialization, RMI events, GUI events • Enclose with drivers that generate all possible events that influence controller access

46. Verification Framework Behavior Verification Action Language Verifier Controller Behavior Machine Controller Classes Counting Abstraction Concurrent Program Controller Interface Machine Interface Verification Java Path Finder Thread Thread Isolation Thread Thread Class Thread Classes

47. A Case Study: TSAFE Tactical Separation Assisted Flight Environment (TSAFE) functionality: • Display aircraft position • Display aircraft planned route • Display aircraft future projected route trajectory • Show conformance problems between planned and projected route

48. Computation Feed Parser Timer Flight Database Graphical Client TSAFE Architecture <<TCP/IP>> User Radar feed Server Client EventThread <<RMI>> 21,057 lines of code with 87 classes

49. Reengineering TSAFE • Found all the synchronization statements in the code (synchronize, wait, notify, notifyAll) • Identified 6 shared objects protected by these synchronization statements • Used 2 instances of a reader-writer controller and 3 instances of a mutex controller for synchronization • In the reengineered TSAFE code the synchronization statements appear only in the Action helper class provided by the concurrency controller pattern

50. Behavior Verification Performance P denotes parameterized verification for arbitrary number of threads