A Theory of Mutations with Applications to Vacuity, Coverage, and Fault Tolerance

1. A Theory of Mutations with Applications to Vacuity, Coverage, and Fault Tolerance Orna Kupferman1 Wenchao Li2 Sanjit A. Seshia2 1 Hebrew University 2 UC Berkeley FMCAD 2008 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA

2. FMCAD 2008

3. This system is correct evenunder faults (e.g. flips in latches) Why? Convince me. It satisfies its specificationunderthese faults. So is my specification not good enough or is my system fault-tolerant? Adam Bob Need fault-tolerance! Doesn’t this mean the specificationcoverage is low? But also need to certify it! FMCAD 2008

4. Problem • Current mutation-based metrics are inadequate to reason about specification coverage for fault-tolerant circuits in model checking. FMCAD 2008

5. Preliminaries • Coverage • Introduce ∆ to an implementation I and check I’ ² S. • Fault Tolerance • I with fault f still satisfies S. • Vacuity • Introduce ∆ to a specification S and check I ² S’. All three involve introducing mutations in the verification process! FMCAD 2008

6. Contributions A theory of mutations: • formally ties together coverage and vacuity in model checking; • enables reasoning coverage for fault-tolerant circuits. FMCAD 2008

7. Agenda • Related Work • Coverage • Vacuity • A Theory of Mutations • Coverage and Vacuity are dual • Aggressiveness amongst mutations • Applications • Conclusion FMCAD 2008

8. state path Coverage • Is my specification complete? • Coverage metrics for model checking [HKHZ 99; KGG 99; CKV 01,03] FSM Coverage FMCAD 2008

9. Coverage • Functional Coverage in BMC [GKD 07] • Detect “forgotten cases” [Claessen 07] • Coverage for fault-tolerant systems [FPFRT 03, DBBDCMF 05] • Single stuck-at fault model FMCAD 2008

10. Is my specification satisfied trivially? Vacuity detection [KV 99, 03; BBER 01; AFFGP 03; CG 04; BFGKM 05; BK 08] Replace a sub-formulae in the most challenging way. Vacuity G (req → F grant) G (req → false) Trivially true in a system where req is never sent. FMCAD 2008

11. Agenda • Related Work • Coverage • Vacuity • A Theory of Mutations • Coverage and Vacuity are dual • Aggressiveness amongst mutations • Applications • Conclusion FMCAD 2008

12.  100X 100X 1000 1000 1001 1001 1001 1000 1000  old new Examples of Mutations • Can mutate inputs, outputs, or latches • Stuck-at • Restricting a signal to a value • Freeing (abstracting) a signal Modifies behaviors Removes behaviors Adds behaviors FMCAD 2008

13. A Theory of Mutations • Properties: • Invertability: (Cμ)ν= C • Monotonicity: I ² S → Iμ ² Sμ • Duality • Interesting Mutations: • Conditional stuck-at • Conditional add/remove transitions • Permuting events FMCAD 2008

14. Duality Iμ ² S ↔ I ² Sν ,where ν and μare dual mutations. low coverage vacuity FMCAD 2008

15. z Circuit with input = {z}, control signals = {x, y}, output = {x}, described by the state representation on the right. xy x 01 0 00 0 0,1 I S 0 1 01 0 0,1 0 1 add behavior remove behavior 11 1 10 1 0,1 0,1 11 1 01 0 I’ S’ 0 1 0,1 0,1 01 0 00 0 S simulates I’ and S’ simulates I FMCAD 2008

16. Aggressiveness • Mutation  is more aggressive than  if applying  makes it harder for the design to satisfy its specification. • I² S → I ² S or I ² S→ I ² S ≥imp  ≥spec  FMCAD 2008

17. Some Aggressive Orders • Free(x) ≥ k-SEU(x) • Free(x) ≥ Stuck_at_0(x) • Free(x) ≥ Flip(x) • Delay_k+1 ≥ Delay_k • k-SEU(x) ≥ m-SEU(x) ≥ for k ≥ m • More interesting ones can be found in the paper. FMCAD 2008

18. Coverage for Fault-tolerance • For a fault-tolerant system I and a set of mutations {j} such that • Ij² S for all 1≤j≤k. • The fault-tolerant system loosely satisfiesS if there is a mutation  such that • j ≤imp  for all 1≤j≤k; • I² S. FMCAD 2008

19. Agenda • Related Work • Coverage • Vacuity • A Theory of Mutations • Coverage and Vacuity are dual • Aggressiveness amongst mutations • Applications • Conclusion FMCAD 2008

20. Applications • Useful vacuity information can be obtained for free from coverage checks. • Analyze coverage for fault-tolerant systems. • Improving specifications • Catch bugs • Strengthen environmental assumptions FMCAD 2008

21. Vacuity from Coverage • S: G (sp[2..0] = 3’b110 → X (sp[2..0] = 3’b111) • In our experiment, applying the “Flip(x)” mutation to sp[0] still satisfies S. • S’: G (sp[2..0] = 3’b110 → X (sp[2..0] = 3’b110) • S & S’ → G ¬(sp[2..0] = 3’b110) FMCAD 2008

22. System behaviors System behaviors System behaviors High-coverage spec. certifies system’s target resilience Original low-coverage spec. Certifying Fault-Tolerance 1-SEU 2-SEU FMCAD 2008

23. Experiments VIS benchmarks, results obtained with Cadence SMV model checker FMCAD 2008

24. Simplied model S’: G (ξ → X (grant = 2’b10) Improving Specifications • Chip Multiprocessor Router [Peh 01] • However, the process still requires some user assistance. S: G (ξ → X ¬(grant = 2’b11) FMCAD 2008

25. Conclusion • A theory of mutations that • Unifies coverage and vacuity • Can be used to certify the correctness of fault-tolerant circuits • A new technique to tighten specifications • The ideas here can be applied to other verification techniques. FMCAD 2008

26. Q & A Thank you! FMCAD 2008

27. References FMCAD 2008