Event-B in a Nutshell

1. 13th CREST Open Workshop 12th-13th of May 2011, London Search-Based Software Engineering for Model-Based Testing Event-B in a Nutshell Test Generation Approaches SBT Challenges Test Data Generation Finite Model Learning Conclusions Alin Stefanescu - University of Pitesti, Romania *) joint work with colleagues from DEPLOY project

2. [Event-B in a Nutshell]

3. Event-B history • Jean-Raymond Abrial (1938- )   • Inventor of the Z and B formal methods. • Z – developed in the 70s • B – developed in the 90s, successfully deployed in industry • Event-B – born with the 21st century • Evolution of B for system level specification • Developement supported by French and European projects: • FP6 RODIN and FP7 DEPLOY

4. DEPLOY project (2008-2012) – funded by FP7 • DEPLOY :: Industrial deployment of advanced systemengineering methods for high productivity and dependability using formal methods • 4 industrial partners • Bosch, Siemens, SAP, SSF • 3 industrial service providers • Systerel, ClearSy, Cetic • 7 academic partners • Newcastle, Aabo, Düsseldorf, ETH Zurich, • Southampton, Pitesti, Bucharest • http://www.deploy‐project.eu

5. Rodin platform for Event-B • Extension of Eclipse IDE (Java-based) • Theorem proving as core technology • Many other Rodin Plug-ins • ProB:animation, consistency and model-checking • Animators (AnimB) • Decomposition • Modularisation • Team-work • Code generation • UML-B • etc.

6. Event-B in a nutshell Event-B model • State-transition model(like ASM, B, Z) • set theory as mathematical language • refinement as basic modeling approach • Contexts • carrier sets (domains) • constants • axioms • Machines • global variables • invariants • events that update the variables • Events • local parameters • guards • actions • ITEMS := CONTEXT {{it1}, {it34}, {it36}, {it67}, {it89}, {it11}, {it354}, {it876},{it321}, {it333}, {it78}, {it787}, {it7878}, {it2342}, {it3453}, {it6786}, {it1232}, {it7765}, {it7098}) items : Powerset(ITEMS)

7. [Test Generation Approaches]

8. Test generation based on Event-B Model-Based Testing (MBT) • Model-based testing (MBT) is a newlyintroduced topic in DEPLOY • priority topic for industrial partners like SAP • challenges due to the sheer size of the state space of real-life scenarios Weinvestigate search-based testing (SBT) techniques for Event-B.

9. Future MBT plugin in RODIN Event-B model MBT Plug-in MBT Users { Model-checking } { Constraint-based } Extra test information { Search-based } Test cases University of Pitesti and University of Dusseldorf Tool developers:

10. Test generation from Event-B SBT Opportunity! 1. Generate a set of tests (sequence of events with concrete param.) 2. Optimize test suite (according to some criteria) – if still needed Event-B model ev2(..),ev5(..),...ev3() ev4(..),ev2(..),...ev4(..) ... ... ... ... ... ... ... ... ev3(..),ev7(..),...ev5(..) ... ... ... ... ... ev6(..),ev5(..),...ev8 Global variables: var1, var2, var3, ... Events ev1(p11,...), ev2(p21,...), ev3(p31,...), ... ev2(..),ev5(..),...ev3() ev4(..),ev2(..),...ev4(..) ... ev3(..),ev7(..),...ev5(..)

11. What is the explicit state space State Space of the Event-B model (3,4,{a,b},...) Abstract machine Event-B model ev3(5) Global variables: var1, var2, var3, ... Events ev1(p11,...), ev2(p21,...), ev3(p31,...), ... ... ... ... ... • States given by the values of global variables • Transitions labeled by events with concrete parameters

12. Test generation from Event-B State Space of the Event-B model (3,4,{a,b},...) Approach 1: Explore the state space using the ProB model checker • state space explosion mainly due to data Try: guide the search ev3(5) ... ... ... SBT Opportunity!

13. Test generation from Event-B – part II State Space of the Event-B model (3,4,{a,b},...) Approach 2: Explore state space ignoring the data (i.e. local parameters) • Problem 1: still large state space Then: construct approximations of state space up to depth K using finite automata Try 1: using machine learning and static analysis Try 2: using evolutionary algorithms? • Problem 2: infeasible sequences Try 1: constraint solving for path feasibility Try 2: test data generation with metaheuristics ev3 ... ... ... SBT Opportunities!

14. [Search-Based Testing Challenges]

15. Let’s take a look at some of specific challenges for Event-B... • More details in: • A. Stefanescu, F. Ipate, R. Lefticaru, C. Tudose.  • Towards Search-Based Testing for Event-B Models. • To appear in Proc. of 4th International Workshop on Search-Based Software Testing (SBST), 2011.

16. No explicit state space Fact • Event-B has no explicit states like the EFSMs • no control state (as in EFSMs) Problem • Large (possibly infinite) state space • testing coverage criteria must be defined • only recent work addressing SBT for EFSMs Possible ideas: • coverage of all events (or a given subset of them) or coverage of all test paths of length < K • many other coverages possible, so industrial guidance is needed • consider the class of Event-B models with a special state variable (see industrial use cases from SAP, SSF, Bosch and UML-B models)

17. Non-numerical types Fact • Event-B is based on set theory • set relations, powersets, functions, set comprehensions, products, records, etc. • Complex structured data (e.g. business domain) Problem • fitness functions in literature mostly defined for numerical types Possible solutions • design new fitness functions for set-based (non-numerical) types • efficient encoding of mixed non-numerical/numerical test data

18. Hierarchical models Fact • Event-B supports different types of hierarchy • refinement from abstract to concrete levels • model decomposition • modularity • most industrial models use some sort of hierarchy (due to size) Problem • no much previous work on SBT addressing hierarchical models Possible ideas: • adapt existing work on test selection for hierarchical state machines • use the existing ProB model checker that can partially deal with hierarchy

19. Non-determinism Fact • Event-B has different types of non-determinism • :| or :∈ operators (e.g. x :∈ {item1, ..., item20}) • non-deterministic choice of the event to be executed when several enabled • non-deterministic choice of parameters (ANY construct) • non-deterministic initialisation of variables satisfying the set of invariants Problem • no much previous work on SBT addressing non-deterministism Possible ideas: • devise fitness functions that improve the chance of choosing a given path in a non-deterministic model (under certain assumptions) • make the non-determinism visible (model instrumentation)

20. [Test Data Generation]

21. Generating test data for a path Problem • Given one path of events, provide the test data (event parameters) that enables the execution of the path. Approach • genetic algorithms • encoding of sets into binary genes • mixed choromosomes (numerical and binary genes) • More details in: • I. Dinca, A. Stefanescu, F. Ipate, R. Lefticaru, C. Tudose.  • Test Data Generation for Event-B Models using Genetic Algorithms. • InProc. of 2nd International Conference on Software Engineering and Computer Systems (ICSECS'11). CCIS Series, vol. 181, pp. 76-90, Springer, 2011.

22. Test data generation with genetic algorithms Event-B model Encoding of variables … 0 1 1 1 1 1 0 0 1 0 1 0 0 1 1 1 1 1 1 1 0 0 0 0 1 0 1 0 0 items • ITEMS := CONTEXT {{it1}, {it34}, {it36}, {it67}, {it89}, {it11}, {it354}, {it876},{it321}, {it333}, {it78}, {it787}, {it7878}, {it2342}, {it3453}, {it6786}, {it1232}, {it7765}, {it7098}) items : Powerset(ITEMS) “Chromosome” Crossover 0 Selection Mutation Fitness evaluation End? Fitness functions Simulator (ProB)

23. Fitness functions for one path fitness := approach level + normalized branch level New objective functions for set types Classical Tracey’s objective for numerical types

24. Examples from the benchmark

25. Statistical results • Statistical comparison of • Genetic Algorithms (GA) and • Random Testing (RT) • on 18 paths covering 5 Event-B models • using statistical test like t-test and U-test • And (of course) the winner is: • GA performs significantly better than RT on most paths • Note: We are currently evaluating constraint-solving (mature for Event-B). It It seems to be quicker for small to medium path (with exceptions).

26. [Finite Model Learning]

27. Generating finite models from Event-B Problem • There is no explicit state space of an Event-B model Approach • Finite automata learning (adapted L* algorithm ) • Aproximation through cover automata • K-bound on the length of executions • Use finite automata for conformance test generation • More details in: • F. Ipate, I. Dinca, A. Stefanescu: • Model Learning and Test Generation for Event-B using Cover Automata. • Submitted to SEFM’11.

28. First experiments SBT Opportunity?! Preliminary approach • Approximation through cover automata for bound l • Incremental -> fits very well with model refinements • Minimal finite automata • Sometimes difficult to find counterexamples (to the approximation) • Scales for medium size models: Boundl ev3 ... ... ...

29. [Conclusions]

30. Opportunities for Search-Based Techniques • To wrap-up opportunities: • Test suite minimisation with multi-objective optimisation • Test data generation for one path with search-based algorithms • Construct finite models with evolutionary algorithms • Combine ProB model-checker with meta-heuristics • Combine ProB constraint-solver with meta-heuristics • Experiment with different search algorithms (PSO, ACO, SA,...) • To be answered until end of project (April 2012): • Which of the above work good in practice?