Model Checking: An introduction & overview

1. Model Checking:An introduction & overview Gordon J. Pace December 2002

2. History of Formal Methods • Automata model of computation: mathematical definition but intractable. • Formal semantics: more abstract models but proofs difficult, tedious and error prone. • Theorem proving: proofs rigorously checked but suffers from ‘only PhDs need apply’ syndrome.

3. The 1990s • Radiation therapy machine overdoses patients, • Pentium FDIV bug, • Ariane-V crash. Industry willing to invest in algorithmic based, push-button verification tools.

4. Model-Checking • Identify an interesting computation model, • For which the verification question is decidable, • And tractable on interesting problems. • Write a program to answer verification questions.

5. Formal Semantics • Operational Semantics:

6. Formal Semantics • Denotational Semantics of Timed Systems: v V’ 0 def [ delay (v’, v) ] = v’(t+1)=v(t) /\ v’(0)=low [ ]

7. Transition Systems • Q = States •  = Transition relation (Q x Q) • I = Initial states ( Q) Q, , I 

8. Constructing TSs via OS (v:=1; w:=v) || (v:=¬v) v,w=0,0 pc=0,0 v,w=1,0 pc=1,0 v,w=1,0 pc=0,1 v,w=1,0 pc=0,0 v,w=0,1 pc=0,1 v,w=0,1 pc=0,0 v,w=1,1 pc=1,0 v,w=1,1 pc=0,0

9. Constructing TSs via TDS i o m Q = Bool x Bool x Bool I = {(i,m,o) | o = i /\ m }  = {((i,m,o),(i’,m’,o’)) | m’=o, o’=i’ /\ m’ }

10. Note: • We will be ‘constructing’ TSs from a symbolic (textual/graphical) description of the system. This is a step which explodes exponentially (linear increase in description may imply exponential increase in state-space size).

11. Properties of TSs • Safety properties: ‘Bad things never happen’. eg The green lights on a street will never be on at the same time as the green lights on an intersecting street. • Liveness properties: ‘Good things eventually happen’. eg A system will never request a service infinitely often without eventually getting it.

12. Safety Property Model Are any of the red states reachable? etc

13. Safety Property Model Given a transition system M=Q,,I  and a set of ‘bad’ states B, are there any states in B which are reachable in M?

14. A Reachability Algorithm R0 = I Rn+1 = Rn (Rn) where: (P) = { s’ | sP: s  s’ } Reachable set is the fix-point of this sequence. Termination and correctness are easy to prove.

15. A Reachability Algorithm R := I; Rprev := ; while (R  Rprev) do Rprev := R; R := R  (R); if (B  R  ) then BUG; CORRECT;

16. State Space Representation • Explicit representation • Keeping a list of traversed states. • State-explosion problem. • Looking at the recursion stack will give counter-example (if one is found). • Breath-first search guarantees a shortest counter-example.

17. Typical Optimizations • On-the-fly exploration: Explore only the ‘interesting’ part of the tree (wrt property and graph). Example: Construct graph only at verification time. Finding a bug would lead to only partial unfolding of the description into a transition system.

18. Typical Optimizations • Partial order reduction: By identifying commuting actions (ones which do not disable each other), we can ignore parts of the model. Example: To check for deadlock in (a!; P  b!; Q), we may just fire actions a and b in this order rather than take all interleavings.

19. Typical Optimizations • Compositional verification: Build TS bottom up, minimising the automata as one goes along. Example: To construct (P Q), construct P and minimise to get P’, construct Q and minimise to get Q’, and then calculate (P’ Q’).

20. Typical Optimizations • Interface-Based Verification: Use information about future interfaces composands while constructing sub-components. Example: Constructing the full rhs of (10c;P + 5c;Q + …)  Huge  (5c;Tea) gives a lot of useless branches which the last process never uses.

21. State Space Representation • Symbolic state representation: Use a symbolic formula to represent the set of states. R := I; Rprev := ; while (R  Rprev) do Rprev := R; R := R  (R); if (B  R  ) then BUG; CORRECT; Requires: representation of empty set, union, intersection, relation application, and set equality test.

22. Symbolic Representation Use boolean formulae Let v1 to vn be the boolean variables in the state space. A boolean formula f(v1,…,vn) represents the set of all states (assignments of the variables) which satisfy the formula.

23. Symbolic Representation Double the variables To represent the transition relation, give a formula over variables v1,…,vn and v’1,…,v’n relating the values before and after the step.

24. Example v1 v3 Initial states: I  (v2=true) /\ (v3=v1 /\ v2) v2 1 Transition relation: T  (v3=v1 /\ v2) /\ (v’3=v’1 /\ v’2) /\ v’2=v3

25. Set Operators: Empty set:  = false Intersection: P  Q = P /\ Q Union: P  Q = P \/ Q Transition relation application: (P) = (vars: P /\ T)[vars’/vars] Testing set equality: P=Q iff P  Q

26. The Problem • Calculating whether a boolean formula is a tautology is an NP-complete problem.  • In practice representations like Binary Decision Diagrams (BDDs) and algorithms used in SAT checkers perform quite well on typical problems.

27. Counter-Example Generation Bad I=R0

28. Counter-Example Generation R1 Bad I

29. Counter-Example Generation R2 R1 Bad I

30. Counter-Example Generation R2 R1 Bad I

31. Counter-Example Generation R2 R1 Bad I

32. Counter-Example Generation R2 R1 Bad I

33. Counter-Example Generation R2 R1 Bad I Set of all shortest counter-examples obtained

34. Abstract Interpretation • Technique to reduce state space to explore, transition relation to use. • Collapse state space by approximating wrt property being verified. • Can be used to verify infinite state systems.

35. Abstract Interpretation • Example: Collapse states together by throwing away variables, or simplifying wrt formula. etc

36. Abstract Interpretation • Example: Collapse states together by throwing away variables, or simplifying wrt formula. etc

37. Abstract Interpretation • Example: Collapse states together by throwing away variables, or simplifying wrt formula. etc

38. Abstract Interpretation • Concrete counter-example generation not always easy. • May yield ‘false negatives’. etc

39. Other Techniques • Backward Analysis R0 = Bad Rn+1 = Rn  -1(Rn) If R be the fix-point of this sequence, the system is correct iff R  I = .

40. Other Techniques • Induction (depth 1): If … • The initial states are good, and • Any good state can only go to a good state, then The system is correct.

41. Other Techniques • Induction (depth n): If … • Any chain of length n starting from an initial state yields only good states, and • Any chain of n good states can only be extended to reach a good state, then, The system is correct.

42. Other Techniques • Induction By starting with n=1 and increasing, (plus adding some other constraints) we get a complete TS verification technique.

43. State-of-the-art • Explicit state traversal: No more than 107 generated states. Works well for interleaving, asynchronous systems. • Symbolic state traversal: Can reach up to 10150 (overall) states. Works well for synchronous systems. • Sometimes may work with thousands of variables … • With abstraction, 101500 states and above have been reported!

44. State-of-the-art • Combined with other techniques, microprocessor producers are managing to ‘verify’ large chunks of their processors. • Application of model-checking techniques on real-life systems still requires expert users.

45. Tools • Various commercial and academic tools available. • Symbolic: • BDD based: SMV, NuSMV, VIS, Lustre tools. • Sat based: Prover tools, Chaff, Hugo, Bandera toolset. • Explicit state: CADP, Spin, CRL, Edinburgh Workbench, FDR. • Various high-level input languages: Verilog, VHDL, LOTOS, CSP, CCS, C, JAVA.

46. Stating Properties • Safety properties are easy to specify • Intuition: ‘no bad things happen’. • If you can express a new output variable ok which is false when something bad happens, then this your property is a safety property (observer based verification). • Not all properties are safety properties.

47. Observer Verification inputs outputs Program ok Observer Advantage: Program and property can be expressed in the same language.

48. mayday  Safety Properties • The system may only shutdown if the mayday signal has been on and unattended for 4 consecutive time units. shutdown ok

49. Non-Safety Properties • Bisimulation based verification • Temporal logic based verification • Linear time logic (eg LTL) Globally (Finally bell) • Branching time logic (eg CTL) AG (ding EF dong) Globally (Globally req Finally ack)

50. Beyond Finite Systems • Example: Induction on structure: From: Prog(in,out) satisfiesProp(in,out) Prog(in,m) /\ Prop(m,out) satisfies Prop(in,out) Conclude: Any chain of Prog’s satisfies Prop.