Efficient Modular Glass Box Software Model Checking

1. Efficient Modular Glass Box Software Model Checking Michael Roberson Chandrasekhar Boyapati The University of Michigan

2. Software Model Checking • Exhaustively test programs • On all possible inputs • On all possible schedules • Up to finite bounds

3. Initial State Binary Tree State Space State Space Explosion

4. State Space Reduction • Many software model checkers • Verisoft, JPF, CMC, SLAM, Blast, Magic, … • Many state space reduction techniques • Partial order reduction • Predicate abstraction • Effective for control-orientedproperties • Our work is on Glass Box Software Model Checking • Effective for data-oriented properties • Significantly more efficient than previous model checkers

5. Modular Glass Box Checking • Check modules against abstractions • Check program replacing modules with abstractions • Modular glass box model checking is important • Further improve scalability of glass box checking • Modular glass box model checking is nontrivial

6. Modular Checking Modular checking in traditional model checkers Initial State of Module Initial State of Abstraction Check outputs at each step

7. We can't reach this transition! Modular Glass Box Checking • We cannot use reachability through transitions • Programmers must provide a class invariant • State space includes all states that satisfy the invariant • Programmers must provide an abstraction function • We use it to generate the abstraction of each state

8. Modular Glass Box Checking • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality c1 Operation c2 c1_output Abstraction Abstraction Equal a2' a1 Operation a2 a1_output

9. Outline • Motivation • Example • Approach • Experimental Results • Related Work • Conclusions

10. Integer Counter classIntegerCounter { Map map = newSearchTree(); intmax_frequency = 0; intmost_frequent = 0; void count(int i) { Integer frequency = (Integer)map.get(i); if (frequency == null) frequency = new Integer(0); map.insert(i, new Integer(frequency+1)); if (frequency >= max_frequency) { max_frequency = frequency; most_frequent = i; } } intget_most_frequent() { returnmost_frequent; } intget_max_frequency() { returnmax_frequency; } } Frequencies are stored in a Map new AbstractMap(); Modular Approach: Replace Module with Abstraction Count an integer Return most frequent integer Return frequency of most frequent integer

11. AbstractMap Implements Map get, insert, delete Linked list Simple execution Smaller state space 5 2 6 1 2 4 5 6 1 4 Module vs Abstraction • SearchTree • Implements Map • get, insert, delete • Balanced binary tree • Efficient execution • Larger state space vs

12. 5 2 6 1 4 Checking SearchTree • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality c1 Operation c2 Abstraction Abstraction Equal a2' a1 Operation a2

13. 5 2 6 1 4 1 2 4 5 6 Checking SearchTree • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality Operation c2 Abstraction Abstraction Equal a2' a1 Operation a2

14. 5 5 2 6 2 6 1 4 1 4 3 1 2 3 4 5 6 1 2 4 5 6 Checking SearchTree • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality insert(3,x) Operation c2 Abstraction Abstraction Equal a2' insert(3,x) Operation a2

15. 5 5 2 6 2 6 1 4 1 4 3 1 2 3 4 5 6 1 2 4 5 6 Checking SearchTree • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality insert(3,x) c1_output Abstraction Abstraction Equal a2' insert(3,x) a1_output

16. 5 5 2 6 2 6 1 4 1 4 3 1 1 2 2 3 3 4 4 5 5 6 6 1 2 4 5 6 Checking SearchTree • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality insert(3,x) Abstraction Abstraction Equal a2' insert(3,x)

17. 5 5 2 6 2 6 1 4 1 4 3 1 1 2 2 3 3 4 4 5 5 6 6 1 2 4 5 6 Checking SearchTree • Choose a state • Generate abstraction • Run one operation • Check outputs • Generate abstraction on post-state • Check for equality insert(3,x) Abstraction Abstraction Equal insert(3,x) 

18. insert(3,x) insert(3,x) 5 5 2 2 6 7 1 4 4 5 5 5 2 6 2 2 6 1 4 1 4 7 4 3 3 3 Glass Box Pruning • An insert operation only touches one path • Insert behaves similarly on many states insert(3,x) 5 2 6 1 4

19. insert(3,x) insert(3,x) 5 5 2 2 6 7 1 4 4 PRUNED 5 5 5 2 6 2 2 6 1 4 1 4 7 4 3 3 3 Glass Box Pruning • We don't need to check more than one of these • We can prune all others from the state space insert(3,x) 5 2 6 1 4

20. insert(3,x) insert(3,x) 5 5 2 2 6 7 1 4 4 PRUNED 5 5 5 2 6 2 2 6 1 4 1 4 7 4 3 3 3 Glass Box Pruning • We check each tree path, not each tree • This reduces the state space dramatically insert(3,x) 5 2 6 1 4

21. 5 2 6 1 2 4 5 6 1 4 2 4 4 1 4 1 5 2 5 1 2 4 5 6 5 6 2 6 1 6 5 4 5 2 6 2 6 2 6 4 1 1 5 1 4 Checking IntegerCounter vs IntegerCounter with SearchTree IntegerCounter with AbstractMap IntegerCounter' IntegerCounter Smaller state space Better state space reduction Faster analysis

22. Outline • Motivation • Example • Approach • Program specification • Search algorithm • State space representation • State space reduction • Experimental Results • Related Work • Conclusions

23. Module Implementation class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; Object get(int key) { /* ... */ } void insert(int key, Object value) { /* ... */ } void remove(int key) { /* ... */ } } c1 Operation c2 Map interface methods

24. Equal a2' a2 Abstraction class AbstractMap implements Map { class Node { Object key; Object value; Node next; } Node head; Object get(int key) { /* ... */ } void insert(int key, Object value) { /* ... */ } void remove(int key) { /* ... */ } @Declarative boolean equalTo(AbstractMap m) { /* ... */ } } • Declarative methods • Subset of Java • Free of side effects • Used for specification • Aid our analyses a1 Operation a2 Map interface methods Equality test

25. Module Specification c1 class SearchTree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; /* ... Map operations ... */ @Declarative boolean repOk() { /* ... */ } AbstractMap abstraction() { /* ... */ } } Abstraction a1 Module Invariant Abstraction function

26. Approach • Program specification • Search algorithm • State space representation • State space reduction

27. 5 2 6 1 4 Search Algorithm • Choose an unchecked valid state insert(3,x) @Declarative boolean repOk() { /* ... */ }

28. 5 2 6 1 4 1 2 4 5 6 Search Algorithm • Generate its abstraction insert(3,x) AbstractMap abstraction() { /* ... */ }

29. 5 5 2 6 2 6 1 4 1 4 3 1 2 4 5 6 1 2 3 4 5 6 Search Algorithm • Run the operation on both states insert(3,x) void insert(int key, Object value) { /* ... */ }

30. 5 5 2 6 2 6 1 4 1 4 3 1 2 4 5 6 1 1 2 2 3 3 4 4 5 5 6 6 Search Algorithm • Generate the post-state abstraction insert(3,x) AbstractMap abstraction() { /* ... */ }

31. 5 5 2 6 2 6 1 4 1 4 3 1 2 4 5 6 1 1 2 2 3 3 4 4 5 5 6 6 Search Algorithm • Check invariant and abstraction equality @Declarative boolean repOk() { /* ... */ } insert(3,x) @Declarative booleanequalTo(AbstractMap m) { /* ... */ }

32. insert(3,x) insert(3,x) 5 5 2 2 6 7 1 4 4 PRUNED 5 5 5 2 6 2 2 6 1 4 1 4 7 4 3 3 3 State Space Reduction • Identify and prune similar states insert(3,x) 5 2 6 1 4

33. Search Algorithm Let S be the states that satisfy repOk() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S = S - P Need efficient representation and operations for these sets!

34. Approach • Program specification • Search algorithm • State space representation • State space reduction

35. key = {b0,b1} value = {b2,b3} left = {b4} right = {b5} n1 key = {b6,b7} value = {b8,b9} left = {} right = {} n2 ... Representation • Represent a set as a boolean formula • Encode each field as bits (b0, b1, …) • Constrain the bits using boolean operations n1.left = null || n1.key > n2.key Øb4 Ú (b1 ÙØb7) Ú ((b1 ÚØb7) Ù b0 ÙØb6)

36. Representation • Initialize to set of states that satisfy invariant • Construct a formula describing invariant @Declarative boolean repOk() { /* ... */ } boolean formula

37. Representation • Initialize to set of states that satisfy invariant • Construct a formula describing invariant • Declarative methods • No assignment, object creation, or loops • Declarative methods allow efficient translation • Declarative methods produce compact formulas @Declarative boolean repOk() { /* ... */ }

38. Search Algorithm Let S be the states that satisfy repOk() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S = S - P Use a SAT solver Add ¬P to the SAT solver

39. Approach • Program specification • Search algorithm • State space representation • State space reduction • Dynamic analysis • Static analysis

40. Dynamic Analysis • Discover and prune states that are similar • Symbolic execution • Generates a path constraint, P • P holds for states that traverse the same code path • P is the set of similar states to be pruned insert(3,x) 5 2 6 1 4

41. op(key,value) Operation is insert n1 Node exists, is greater/less than key n2 n3 n4 n5 Final node does not exist (yet) Dynamic Analysis • Discover and prune states that are similar • Symbolic execution • Generates a path constraint, P • P holds for states that traverse the same code path • P is the set of similar states to be pruned op = insert && root = n1 && key < n1.key && n1.left = n2 && key > n2.key && n2.right = n5 && key < n5.key && n5.left = null

42. a = false b = true a = true a = false b = false b = false Static Analysis • Dynamic analysis finds P, the similar states • Pruning these states is not always correct!  class WhyStaticAnalysis { boolean a, b; void operation() { if (a) a = false; else a = true; } @Declarative boolean repOk() { return a || b; } } a = true a = true P := b = true  • We use a static analysis to ensure correctness • All pruned transitions checked together • Any error within finite bounds is caught

43. a = false b = true a = true a = false b b Static Analysis  class WhyStaticAnalysis { boolean a, b; void operation() { if (a) a = false; else a = true; } @Declarative boolean repOk() { return a || b; } } a = true a = true P := b = true Prestate of a pruned transition Poststate of a pruned transition Invariant repOk = (a || b) Prestate Invariant repOkPre= (a || b)a=true = true Not valid when b = false! Poststate Invariant repOkPost= (a || b)a=false = b repOkPre®repOkPost = b Correct Transition

44. pre Operation post Abstraction Abstraction Equal abs post' abs pre Operation abs post Static Analysis • For every valid prestate in P, the following hold • The invariant is maintained in the poststate • Equality of abstractions repOkpre® repOkpost && abs_post.equalTo(abs_post') • Use a SAT solver to check • If it holds then pruning is sound • If not, we have a counterexample boolean formula

45. Outline • Motivation • Example • Approach • Experimental Results • Related Work • Conclusions

46. Checking Modules vs Abstractions • TreeMap • Implemented with a red-black tree • HashMap • Implemented with a hash table • AbstractMap • Implemented with a linked list of (key, value) pairs

47. Checking Modules vs Abstractions • TreeSet • Implemented with a TreeMap • HashSet • Implemented with a HashMap • AbstractSet • Implemented with a linked list of set items

48. Maps vs AbstractMap We check over 235 trees in under 15 minutes

49. Sets vs AbstractSet

50. Checking Clients