1 / 31

Model Checking Java Programs using Structural Heuristics

Model Checking Java Programs using Structural Heuristics. Alex Groce Carnegie Mellon University Willem Visser NASA Ames Research Center. Model Checking. Explores graph of reachable system states Checking for local assertions, invariants and general temporal (logic) properties

fynn
Download Presentation

Model Checking Java Programs using Structural Heuristics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Model Checking Java Programs using Structural Heuristics Alex Groce Carnegie Mellon University Willem Visser NASA Ames Research Center

  2. Model Checking • Explores graph of reachable system states • Checking for local assertions, invariants and general temporal (logic) properties • Symbolic model checking • Explicit-state model checking

  3. Java Code Bytecode void add(Object o) { buffer[head] = o; head = (head+1)%size; } Object take() { … tail=(tail+1)%size; return buffer[tail]; } JAVAC JVM 0: iconst_0 1: istore_2 2: goto #39 5: getstatic 8: aload_0 9: iload_2 10: aaload Model Checker Java PathFinder Special JVM

  4. Depth-first Search push initial state on Stack while (Stack not empty) s = top(Stack) if s has no more successors pop the Stack else s’ = next successor of s if s’ not already visited mark s’ visited if s’ is a goal state then terminate push s’ on Stack

  5. Problems with DFS • Produces lengthy counterexamples • If state-space is too large to fully explore • May expend all resources on a single path when shallow counterexamples exist • Failed runs give little information because states explored may be very “similar”

  6. Directed Model Checking • Model checking as a search in a state space • Why not use heuristics to guide the search? • Need to know what we’re looking for • Can we find good heuristics for model checking? • Bug-finding rather than verification

  7. Best-first Search priority queue Q = {initial state} while (Q not empty) s = state in Q with lowest f remove s from Q for each successor state s’ of s if s’ not already visited mark s’ visited if s’ is a goal state then terminate f = h(s’) store (s’, f) in Q

  8. Two Kinds of Heuristics • Property-specific heuristics • Directed at a specific error • Number of unblocked threads as a measure of distance to deadlock • Static analysis for distance to an assertion check • Focus of most previous work in field

  9. Two Kinds of Heuristics • Structural heuristics • Designed to explore the structure of a program in a systematic fashion • But what do we mean by structure?

  10. Structural Heuristics • One obvious kind of structure in a program: • Control flow • Reachable control flow rather than just CFG • Motivation for branch coverage metrics used in software testing

  11. Branch Coverage • Instrument model checker to calculate branch coverage • Using a simple coverage measure as a heuristic doesn’t work well • Easily falls into local minima (once any branches are taken, every state on that path has “better” coverage) • Doesn’t distinguish between branches explored once and branches explored many times

  12. The Branch Counting Heuristic • Count the number of times each branch has been taken • Heuristic value is then: • Branches never before taken get lowest value • Non-branching transitions are next lowest • Otherwise, score is equal to the count (lower values are explored first)

  13. Three Searches DFS Branch Counting CFG Each CFG state is a basic block that increments some variable x. ERROR BFS

  14. Three Searches DFS Branch Counting CFG BFS

  15. Three Searches DFS Branch Counting CFG BFS

  16. Three Searches DFS Branch Counting CFG BFS

  17. Three Searches DFS Branch Counting CFG Heuristic avoids taking BFS

  18. Three Searches DFS Branch Counting CFG BFS

  19. Three Searches DFS Branch Counting CFG Expands 15 states BFS Terminates only with depth limit Expands 25 states

  20. Experimental Results • DEOS real-time operating system example • This version uses an integer valued counter, without abstraction

  21. Results for DEOS All experiments performed on a 1.4GHz Athlon, limiting Java heap size to 512MB, all times are in seconds

  22. The Interleaving Heuristic • An important (and very hard to find) class of errors in Java is concurrency errors • What kind of structure could we explore to catch these? • Thread-interdependency

  23. The Interleaving Heuristic • Not clear how to heuristically define actual thread-interdependence • So we use an approximation: • Executions in which context is switched more often are given better heuristic values • Explores executions unlikely to appear in testing (JVM/JITs schedule quite differently)

  24. The Interleaving Heuristic • Keep track on each path of which threads are executed at each transition • Give lower (better) heuristic score to paths in which the most recently executed thread has been run less frequently • Slightly more complicated in practice, counting live threads

  25. Limiting the Queue • With heuristics we are more interested in finding bugs than in verification • So, we apply a technique from heuristic search literature: • Limit the size of the priority queue! • When queue has more than k states in it, remove all but k states with best heuristic values

  26. Experimental Results • Dining Philosophers • Comparison to other results: • Godefroid and Khurshid in TACAS ’02 paper apply genetic algorithms to dining philosophers • Best result reported is 17 philosophers, 177 seconds, 50% success rate (on a slower machine) • HSF-SPIN • Not clear how to compare (times not given) • Best result they show is 16 philosophers, and SPIN (using partial order reduction) itself fails with 14 philosophers

  27. Experimental Results

  28. One Last Heuristic • The choose-free heuristic: • Works only for abstracted Java programs • Rewards transitions that do not involve nondeterminism introduced by the abstraction • Prefers counterexamples that do not result from loss of precision introduced by the abstraction • Structure of abstraction, not program

  29. Previous Work • Edelkamp, Lafuente, and Leue • HSF-SPIN: SPIN + heuristic search framework • Bloem, Ravi, and Somenzi • Symbolic Guided Search: BDDs + heuristics • With BDDs heuristics can aid verification • Cobleigh, Clarke, and Osterweil • FLAVERS verification work

  30. Conclusions • Structural heuristics: a useful class of heuristics • When model checking is used for debugging, we may not know what kinds of bugs we are hunting • Property-specific heuristics are also useful; approach is complementary, not replacement • Most-blocked can perform as well or better than interleaving in the Remote Agent example, depending on the k limit and search method

  31. Future Work • Experiment with other, larger examples • Static analysis for property-specific heuristics • Language for properties/search/heuristics • Discover how heuristics work when symbolic execution is introduced into JPF • Counterexample analysis for “bug causality” • What other kinds of structure can be exploited with heuristics? • Counting occurrences of data values, perhaps

More Related