530 likes | 663 Views
Dependence Analysis in Reduction of Requirement Based Test Suites Boris Vaysburg Luay Tahat Bogdan Korel
E N D
Dependence Analysis in Reduction of Requirement Based Test Suites Boris Vaysburg Luay Tahat Bogdan Korel Computer Science Department Bell Labs Innovations Computer Science Department Illinois Institute of Technology Lucent Technologies Illinois Institute of Technology Chicago, IL USA Naperville, IL USA Chicago, IL USA
Outline • Model-based testing • Selective model-based testing • Selective test suite reduction using EFSM analysis • EFSM dependencies • Test Reduction based on EFSM analysis • Static approach • Dynamic approach • Conclusions
Model Based Testing • Model based testing is used to test software systems based on a system model • Test cases are generated from the system model • Testing state-based software systems
Model Based Testing • Systems can be modeled using: • EFSM: Extended Finite State Machine • SDL: Specification Description Language
Model Based Testing • Systems can be modeled using: • EFSM: Extended Finite State Machine • SDL: Specification Description Language
Extended Finite State MachineEFSM • EFSM consists of: • States • Transitions • The following elements are associated with each transition: • Event • Enabling condition • Action(s)
State 1 State 2 EFSM Transition Event(p)[Condition]/Action
Model Based Testing Strategies • State coverage • Transition coverage • Path coverage • Constrained path coverage: each transition can be traversed at most “n” times in a test
Model-Based Testing • These strategies are used to test the whole system • Model-based testing strategies may generate large test suites
Selective Model-Based Testing • Testers are frequently interested in testing only selected functionality (requirement) • In a system model requirements frequently correspond to transitions • The problem of selective testing of a requirement can be converted to a problem of testing a transition in the model (selective testing of a transition)
Selective Testing Strategies with respect to a transition • State coverage • Transition coverage • Path coverage • Constrained path coverage Each test must contain a transition (requirement) under test
Selective Model-Based Testing • The size of a selective model-based test suite is frequently significantly smaller than the size of a complete system test suite • However, the size of selective test suites still may be large
Reduction of Selective Test Suites • The goal is to develop an approach of reduction of selective test suites • EFSM dependence analysis is used to reduce the size of selective test suites
Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.
Test Suite Reduction Interactions? Dependencies between EFSM elements.
EFSM dependence analysis • Dependencies between transitions (“active” elements of EFSM) • Two types of dependencies between transitions in the EFSM are identified: • Data dependence • Control dependence
Data Dependence Data dependence captures the notion that one transition defines a value to a variable and another transition may potentially use this value.
Control Dependence • Control dependence captures the notion that one transition may affect traversal of another transition • It is defined based on the concept of post-dominance
EFSM Dependence Graph • Nodes represent transitions • Arcs represent control and data dependencies
T5 T6 T8 T1 T4 T9 T2 T3 T7 Static EFSM Dependence Graph data dependence control dependence
Test Suite Reduction Two tests are considered to be “equivalent” with respect to a transition under test if, during traversal of the EFSM model, these tests exhibit the same “pattern” of interactions with respect to the transition under test.
Test suite reduction Interactions: • Data dependencies • Control dependencies Pattern of interactions: • Dependence sub-graph • Static interaction pattern • Dynamic interaction pattern
Test #1: T1, T4, T6, T8, T6, T8, T5, T8, T9 Test #2: T1, T2, T4, T6, T8, T6, T8, T5, T8, T9
Selective test suite reductionStatic Interaction Pattern • All dependencies that occurred during traversal of the test (path) are mapped to the EFSM dependence sub-graph • Dependencies that do not “influence” a transition under test are removed • The resulting dependence sub-graph is called Static Interaction Pattern • If two tests have the same Static Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite
Test #1: T1 T4 T6 T8 T6 T8 T5 T8 T9
T5 T6 T8 T1 T4 T9 Dependence Sub-graph
T6 T8 T1 T4 T9 Dependence Sub-graph T5
T6 T8 T1 T4 T9 Dependence Sub-graph T5
T6 T1 T4 Static Interaction Patternfor Test #1 T5
Test #2: T1 T2 T4 T6 T8 T6 T8 T5 T8 T9
T5 T6 T8 T1 T4 T9 T2 Dependence Sub-graph
T6 T8 T1 T4 T9 T2 Dependence Sub-graph T5
T6 T8 T1 T4 T9 T2 Dependence Sub-graph T5
T6 T1 T4 Static Interaction Patternfor Test #2 T5 Test #1 and Test #2 have the same Static Interaction Patterns
Test suite reduction • Significant test reduction can be achieved using Static Interaction Patterns
n Size of Selective Test Suite Size of Reduced Test Suite Using Static Interaction Pattern 2 18 3 3 100 7 4 450 14 5 1,806 16 6 6,762 16 7 24,300 16 Test Suite Size Reduction
Selective test suite reduction • Test reduction using Static Interaction Patterns is relatively “coarse” • Static Interaction Patterns are not able to capture intricate interactions • Dynamic Interaction Pattern is used
Dynamic EFSM dependencies • Dynamic dependencies are dependencies that occur during traversal of the test (path) • Dynamic EFSM dependence graph
Dynamic EFSM Dependencies for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9 Dynamic EFSM Dependence Graph
Selective test suite reductionDynamic Interaction Pattern • All dependencies that occurred during traversal of the test (path) are used to create a dynamic EFSM dependence graph. • Dependencies that do not “influence” a transition under test are removed. • The resulting dependence sub-graph is called Dynamic Interaction Pattern. • If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite.
Dynamic EFSM Dependence graph for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9
Dynamic EFSM Dependence graph for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9
Dynamic EFSM Dependence graph for Test #2 T1 T2 T4 T6 T8 T6 T8 T5 T8 T9
Dynamic Interaction Pattern for Test #2 T1 T2 T4 T6 T8 T6 T8T5T8 T9 If two tests have the same Dynamic Interaction Pattern with respect to a transition under test, one of these tests is removed from the test suite