160 likes | 272 Views
Survey – Test Generation Willibald Krenn, TUG. Models – In a Nutshell (1). Model-Based Specification VDM, Z, B, JML, Spec#, CIRCUS, ALLOY, OCL, PiSPEC Challenge: Sequence Finding (Build FSM) Examples in Survey: Z, B Abstract Data-Types (algebraic)
E N D
Survey – Test Generation Willibald Krenn, TUG
Models – In a Nutshell (1) • Model-Based Specification • VDM, Z, B, JML, Spec#, CIRCUS, ALLOY, OCL, PiSPEC • Challenge: Sequence Finding (Build FSM) • Examples in Survey: Z, B • Abstract Data-Types (algebraic) • Raise, OBJ, Larch, CASL, ELOTOS (data part) • Test whether implementation fulfills axioms (Challenge: Equality!) • Examples in Survey: Container for Natural Numbers, Sort • Process Algebras • CCS, CSP, LOTOS • Specify behavior • Currently no example in survey • Test-case generation, e.g. based on LTS
Models – In a Nutshell (2) • Labeled Transition Systems • LTS, IOSTS • LTS: Transition system; Labels (e.g. input/output symbols) on edges; • Test case generation: e.g. test purpose multiplied with Spec • Kripke Structures, Temporal Logic • Example: LTL properties for a brake • Test case generation: e.g. by model checking (counter example) • EFSM, State Charts • State Charts similar to hybrid automata („do“ keyword) • Example in Survey: UML state diagram for ATM withdrawl • Test: State/Transition Equivalence • Hybrid • Hybrid Automata • Hybrid I/O Transition Systems
Tools – Model Based Spec. • Random Testing • JET • JML annotations • test data generation • Directed Testing (White Box) • DART, EXE, CUTE / jCUTE, Pex • Implementation gets instrumented • Do symbolic and concrete execution • From symbolic execution: Calculate input data to get high coverage (branches!) • Multi Strategy • Spec Explorer • Model-Program • State exploration
Tools – Testing from LTS • Testing with Purposes • TGV, STG • Purpose some LTS • Multiply specification with purpose and generate tests • AGATHA • Symbolic execution of STS • Purpose is a sub-tree • Random • TorX • On-The-Fly • Fault based (mutation) • Mutate specification, extract discriminating sequence, use sequence as test purpose
Tools – Testing with Model Checkers • Coverage Criteria • UPPAAL • Testing real-time systems • Supports generation of test suites that satisfy certain coverage criteria • FShell (white box) • White-Box testing of C programs • Uses CBMC • Control Flow Graph coverage by employing SAT solver
Tools – Testing from EFSM, State Charts • Purposes • Autolink • Creates TTCN test suites • Spec given in SDL, Purpose in MSC • TDE/UML • Takes UML behavior models annotated with additional test data. E.g., coverage requirements, constraints • Generates test cases for C++/Java • Eclipse Plug-In
Fault Injection Techniques • Provided by SP • Gives an overview of different techniques to inject faults into embedded designs • Fault injection is used as experimental dependability validation method.
Case Studies • GSM 11-11 • B-Testing-Tool used for test case generation • Successful • AGEDIS • TGV, GOTCHA used for test case generation • Most interesting: „Lessons learnt“ • ASML EUV machine • Model-based integration and testing method • Addresses costly integration problems • TorX used for test case generation • Successful
LTS - Example !c ?b !a !c