MOGENTES 3 rd and Final Review Reporting Period January 2010 – March 2011 Cologne, 26 May 2011

1. Scientific Results – Simulink Track Daniel Kroening, Nannan He / ETH (UOXF) MOGENTES 3rd and Final Review Reporting Period January 2010 – March 2011 Cologne, 26 May 2011

2. Simulink • Matlab Simulink is a graphical dataflow language • Predominant modelling formalism in automotive software • Synchronous semantics • Resembles RTL-level hardware designs • Easy impact analysis • Demonstrators: CAS, SAC, RELAB

3. BMC based TCG for Simulink Simulink model Unwinding depth k Goto Binaries Translate model Unroll loops k times Check for counterexample Test suite T t [found] Generate test-case t No counterexample

4. Mutation testing for Simulink Detected! • Mutant is a small syntactic modification • Test-case Generation (TCG): Find a test-case to detect mutant 1, -2, -2,… 1, 0, -3,… 1, 0, 1,… Miter model assumes: (In1==In1’)&&(In2==In2’) 1, 0, 1,… Inject an ABS operator Miter model of S and S’

5. Floating-point Arithmetic (FPA) • FPA is critical for embedded software systems • E.g, approximate real-valued quantities in SAC model as floating-point numbers. (not infinite-precision rational numbers) • In FPA, a + b = a if a is a very large b is a very small number. • Automated and precise verification of floating-point models is extremely challenging. • Unavoidable high computational cost • Lack efficient SMT solvers that can handle FPA • Lack benchmarks to conduct comparative studies of FP verification techniques

6. FPA Verification • To achieve both accurate reasoning of FPA and much better performance, CBMC/Cover implements a mixed abstraction framework [FMCAD-2009](3). • Uses both under- and over- approximations simultaneously. • Combines this with a novel abstraction refinement method. • Proposed additional standard theories to SMT-Lib emerging from MOGENTES work [SMT-2010] (10). • For the theory of sets, lists and maps • For floating-point arithmetic (following IEEE 754 FP standard) • Require assessment of the compatibility of this standard with existing domain-related industrial standards like ISO 11783, etc.

7. General TCG framework for Simulink • Objectives • Minimize the size of T and the runtime of entire TCG • Maximize the mutation coverage • Q: Which mutant m is selected? Simulink model s mutants M Pick mutant m from M M ≠ Ø Done N Y Test suite T Generate test t detecting m [t found] Remove m from M Remove relevant mutants from M t

8. Formal Concept Analysis (FCA) • A concept is a maximalgrouping of all objects that share common attributes. • FCA computes the concept lattice from the context. • Partial order defines the lattice structure on concepts. ({1,2,3,4}, Ø) e:even o: odd s: square p: prime ({2,4}, {e}) ({2,3}, {p}) ({1,3}, {o}) ({1,4}, {s}) ({4}, {e, s}) ({1}, {o, s}) ({3}, {o, p}) ({2}, {e, p}) (Ø, {e,s,p,o}) (b) A Example Concept Lattice [DAC-2011] (16)

9. Mutant Selection and Removal • Mutants occur in multiple concepts • Choose the concept where M has the largest number of attributes. • Select M while traversing the lattice • Method1: Remove all mutants detected by generated t. • Further avoid TCG efforts via FCA? • Method2: Remove all mutants in the visited concept C IF TCG has • Failed finding t for one or multiple mutants in C • Potential unobservable mutants • Generated t for one or multiple mutants in C Mutantk..l ... Mutantj ... ... … Top-down Breadth-first Bottom-up Breadth-first Concept Lattice of (M, A, I)

10. Experimental results (1) • 7 instances extracted from SAC model • Inject between 66 and 621 mutants into each Simulink model • Performance comparison • Three FCA guided mutant selections • Three simple non-FCA mutant selections • Random TCG. • Bottom-up + Opt. M and Bottom-up overall achieve the highest degree of coverage. • Bottom-up results in the best coverage within timeout. • Bottom-up + Opt. M generates the smallest number of test-cases with same or similar coverage.

11. Experimental Results (2) Mutations far from observation points Mutations close to observation points Using Method1 *: TCG procedure not finished in given time threshold. Bot-U + Method2

12. Equivalent Mutants • Equivalent mutants: No observable different behaviours. • K-induction in mutation-based testing • Build miter model • Base case: Prove the equivalence in depth d. • Step case: Prove that the base case implies the equivalence in depth d+1. • Instrument assertions of internal signal properties. assert(sel>0&&sel_m>0)

13. Experimental Results

14. Summary of Scientific results • Bounded model checking based TCG for Simulink • New abstraction-based algorithm for precise and automatic floating-point verification • Generation of small test suites with high coverage via formal concept analysis • Detection of equivalent mutants using induction techniques http://www.cprover.org/cover/

15. Publications (1)

16. Publications (2)

17. Thanks L. Haller Induction + Abstraction P. Farries Project Administrator G. Weissenbacher BMC+TCG Interpolation P. Ruemmer FPA+TCG Theorem Proving M. Purandare Coverage + Abstraction T. Wahl FPA + Concurrency V. D’Silva Abstract Interpolation N. He SimulinkTCG A. Donaldson k-induction A. Brillout FPA + Interpolation