Backward Reasoning with Formal Properties: A methodology for bug isolation on simulation traces

1. Backward Reasoning with Formal Properties: A methodology for bug isolation on simulation traces Anvesh Komuravelli1 Srobona Mitra1 Ansuman Banerjee2 Pallab Dasgupta1 1Dept. of Comp. Sc. & Engg., IIT Kharagpur 2Adv. Comp. & Microelectronics Unit, ISI Kolkata Presented by: Srobona Mitra 2011 Asian Test Symposium

2. Outline • Background and Objective • Example • The Approach • Running on the example • Experimental Results • Conclusion and Future Work

3. Background • When a logical bug is sensitized, it may take several cycles for the bug to propagate to the design interface and manifest itself in the trace • Automated bug localization techniques for hardware designs typically work on the design implementation to root-cause the given bug • Advantage: complete approach • Disadvantage: scalability issues • Assertions are scattered throughout the design, at the interface of each component module and globally

4. Objective • To pinpoint the location of bug with the help of the assertions given the buggy trace • Advantage: • highly scalable • Works for incompletely specified designs, where the RTL code of some components is not available • new properties can be added in hindsight to perform what-if analysis • Disadvantage: • incomplete approach

5. The Problem • Given: • A buggy simulation trace where the simulation output differs from the golden output • The set of assertions scattered all over the design • Goal: • To use causal deduction using the assertions to locate the bug • Challenges: • To use the properties to propagate the cause-effect relationships through internal signals of design not visible in the trace • To do the deduction backwards starting from the bug manifestation

6. Example Buggy Trace Cycle: a b 1: 0 0 2: 0 1 3: 0 1 ……. …….. 11: 1 1 12: 1 1 13: 1 1 14: 0 1 15: 0 1 16: 0 1 17: 0 1 18: 0 1 19: 0 1 20: 0 0 a c f d b e b20 = FALSE d16 = FALSE c14 = TRUE f12 = TRUE e11 = FALSE a11 = FALSE Refutation! G(a → e) G(e → ¬Xf ) G(f ↔ F[2,2]c) G(c V XXd) G(d → XXXXb)

7. Bounded LTL Semantics Let π = < s0, s1, … > be the infinite state sequence corresponding to an infinite execution of the system beginning at s0 So πk = < sk, sk+1, … >

8. Subformulae of an LTL formula

9. Rewriting of LTL formulae

10. Overall Methodology • Extract all the subformulae of all the properties. • G(pi) → pi → sf(pi) • Form the bi-implications corresponding to all the subformulae (using rewrite rules) • Form the conjuncts corresponding to the properties • a & Xb → (a) & (Xb) • Create new variables for all subformulae at each time step while going back • Add the signal valuations at each cycle • Form a SAT expression and check for unsatisfiability

11. On the example - Subformulae G(a → e) {a, e} G(e → ¬Xf ) {e, f, Xf} G(f ↔ F[2,2]c) {f, F[2,2]c, F[1,1]c, F[0,0]c, c} G(c V XXd) {c, XXd, Xd, d} G(d → XXXXb) {d, XXXXb, XXXb, XXb, Xb, b} E = {a, b, c, d, e, f, XXXXb, XXXb, XXb, Xb, F[2,2]c, F[1,1]c, F[0,0]c, XXd, Xd, Xf}

12. Bi-implications Xb (Xb)t ↔ bt+1 XXb (XXb)t ↔ (Xb)t+1 XXXb (XXXb)t ↔ (XXb)t+1 XXXXb (XXXXb)t ↔ (XXXb)t+1 F[0,0]c (F[0,0]c)t ↔ ct F[1,1]c (F[1,1]c)t ↔ (F[0,0]c)t+1 F[2,2]c (F[2,2]c)t ↔ (F[1,1]c)t+1 Xd (Xd)t ↔ dt+1 XXd (XXd)t ↔ (Xd)t+1 Xf (Xf)t ↔ ft+1

13. Conjuncts for formulae a → e ¬at V et e → ¬Xf ¬et V ¬(Xf)t f ↔ F[2,2]c (¬ft V (F[2,2]c)t) Λ (¬(F[2,2]c)t V ft) c V XXd ct V (XXd)t d → XXXXb ¬dt V (XXXXb)t

14. Running of the algorithm ¬b20(Xb)19 ↔ b19+1 ¬(Xb)19 (XXb)18 ↔ (Xb)18+1 ¬(XXb)18(XXXb)17 ↔ (XXb)17+1 ¬(XXXb)17 (XXXXb)16 ↔ (XXXb)16+1 ¬(XXXXb)16 ¬d16 V (XXXXb)16 ¬d16 (Xd)15 ↔ d15+1 ¬(Xd)15(XXd)14 ↔ (Xd)14+1 ¬(XXd)14 c14 V (XXd)14 c14 (F[0,0]c)14 ↔ c14 (F[0,0]c)14 (F[1,1]c)13 ↔ (F[0,0]c)13+1

15. Running of the algorithm (F[1,1]c)13 (F[2,2]c)12 ↔ (F[1,1]c)12+1 (F[2,2]c)12 (¬f12 V (F[2,2]c)12) Λ (¬(F[2,2]c)12V f12) f12 (Xf)11 ↔ f11+1 (Xf)11 ¬e11 V ¬(Xf)11 ¬e11 ¬a11 V e11 ¬a11 a11 (from trace) Refutation!

16. Experimental Results • Experiments with randomly generated safety properties and randomly generated traces • Varied the average length of the properties, visibility, total number of signals, length of refutation trace • To analyze, formed a digraph with properties as vertices and an edge from p1 to p2 iff p1 affects p2. • G(a Λ Xb) affects G(b → c)

17. Experimental Results • Despite wide variations in the properties of the digraph like number of components, component size, longest path in a component, significant factors affecting the time and space are: • Number of properties • Length of property • Length of counterexample

18. Experimental Results Tabulated • Average Property Length = 6 and Properties# = 10 • Average Property Length = 6 and Properties# = 50

19. Experimental Results Tabulated • Average Property Length = 20 and Properties# = 10 • Average Property Length = 20 and Properties# = 50

20. Conclusion and Future Work • We propose a methodology for property-assisted debugging of a hardware design • Future direction is to explore how to use the refutation provided by the methodology • Can a subset of the total set of properties be reasoned which accounts for the refutation? • Constrain the search space of the design to locate and fix the bug • Combine property assisted debugging with design assisted debugging

21. Thank You! Questions? E-mail ID: srobona@cse.iitkgp.ernet.in