Sparse Coding for Specification Mining and Error Localization

1. Sparse Coding for Specification Mining and Error Localization • Wenchao Li, Sanjit A. Seshia • University of California - Berkeley • wenchaol@eecs.berkeley.edu • Runtime Verification • September 26, 2012

2. Assertion-Based Verification Generate stimulus to patch coverage holes Coverage Tests Circuit/Program Simulator Assertions Find bugs with assertions Problem: assertions are created manually “…typically 20% of specifications pass vacuously during the first formal verification runs of a new hardware design…” [IBM Haifa]

3. Error Localization 01010101010101101101010101011111101010101 Fatal Error • Challenges: • Limited observability • Long error detection latency • Transient and hard-to-reproduce bugs Where? Idea: assertions can provide local observability and correctness checks

4. Related Work Our technique is template-free and does not require having the system model • Specification Mining: • Programs: single-state invariants, pre-/post-conditions, automata learning, alternating patterns • Circuits: fixed-delay pairs, temporal logic patterns • Require templates • Error Localization: • Programs: model checking, predicates • Circuits: instruction footprints, SAT-based, mined assertion-based • Require system model and good observability • Require templates

5. What can you tell by just observing a trace? Obj1.m1() Obj1.m1() Obj1.m2() Obj2.m1() Obj2.m1() Cloud • Hardware trace • Program trace • Human interaction/behavior • Sensor network • Distributed system

6. A Sparse Coding Approach » 0.8 * + 0.3 * + 0.5 * » 0.8 * + 0.3 * + 0.5 * x»0.8 * f3 + 0.3 * f30 + 0.5 * f61 Sparsity helps to uncover latent structure of the data • Key idea:Express each subtrace as a Boolean combination of a few “basis subtraces”–a (sparsity-constrained) Boolean matrix factorization problem.

7. Contributions and Outline • A new formalism for discovering structure in a trace • A definition of the sparsity-constrained Boolean matrix factorization problem and an algorithm for solving it • Applications to specification mining and error localization • Does not rely on redefined templates • Simultaneous perform error localization and explanation • Outline: • Problem formulation • Algorithm • Error localization and explanation • Results

8. Problem Formulation columns are sparse basis coefficient ○ = Multiplication as “AND” Addition as “OR” Subtrace

9. Sparsity-Constrained Boolean Factorization Given a data matrix and a positive integer , the sparsity-constrained Boolean factorization problem is to find , and such that and and is maximized. C = 2

10. Algorithm Idea v u Observe that the data matrix X can be viewed as the adjacency matrix for a bipartitie graph. Idea: factorization →biclique cover (biclique↔ basis subtrace)

11. Algorithm Overview A D A D A B E E C C C A D A D E E C C • Incrementally generate maximal bicliques • Consensus-based algorithm • Extend to a maximal biclique • Keep track of closeness to sparsity constraint • Heuristically optimize for basis sharing

12. Algorithm Overview A A A D D D E E E B C C C A D A D … A X F F F B E G E Y C C C Z Step 1: start with the set of v-rooted star bicliques Step 2: Pick two stars and form a consensus Step 3: Extend the consensus to a maximal biclique Step 4: Add the biclique to the cover if possible Step 5: update sparsityconstraint at the covered nodes

13. An Arbiter Example A 2-input 2-output arbiter with round-robin scheme Number of subtraces 12 Sample mined assertions (basis subtrace):

14. Error Localization and Explanation • Minimize • Subject to • Error localization and explanation based on reconstruction: A subtrace has an error if it cannot be reconstructed from the basis subtraces • A subtrace is error-free if • If not, a (minimum) error explanation is , where is the solution to the minimization problem above.

15. Example Illustration Error trace Error subtrace Error explanation Error All subtraces Space spanned by the learned basis Correct subtraces Alternative error Explanation Error localization and explanation (arbiter example):

16. Experimental Results A CMP Router in a NoC • Error Localization: • Inject a single bit flip at a random cycle for each of 99 error traces • Localize the error to the subtrace(out of 999) where it was injected • Comparisons: • Baseline approach (1): hash all distinct subtraces – report error even before an error is injected for the 99 traces • Baseline approach (2): use unit basis – 0% localization • Sparse Coding: 55.6% localization • Chip Multiprocessor Router: • Observe 14 control signals • Subtracewidth of 2 cycles • Learn the basis from a single error-free trace of 1000 cycles: 0.243 seconds to obtain 189 basis subtraces from 93 distinct subtraces

17. Conclusion THANK YOU A template-free assertion miner that can explore embedded patterns in digital circuit traces Effective assertion-mining based error localization and explanation Potential applications to other domains, e.g. programs or distributed systems