Domain testing

1. Domain testing Tor Stålhane

2. Domain testing revisited We have earlier looked at domain testing as a simple strategy for selecting test cases. We will now take this a step further. Testers have frequently observed that domain boundaries are particularly fault prone and should therefore be carefully checked.

3. Predicates We will assume that all predicates are simple. This implies that they contain only one relational operator. Allowable operators are =, ><, >=, <=, < and >. Predicates that are not simple can be split into two or more simple predicates.

4. Path conditions Each path thorough a system is defined by a path condition – the conjunction of all predicates encountered along this path. An input traverses a path if and only if the path condition is fulfilled. The path condition defines a path domain, which is the set of all inputs that cause the path to be executed.

5. Path domains – 1 A path domain is surrounded by a boundary, consisting of one or more borders where each border corresponds to a predicate. A border is • Closed if defined by a predicate containing =, =< or >=. Closed borders belong to the path domain. • Open if defined by a predicate containing ><, > or >. Open borders do not belong to the path domain.

6. Path domains – 2 Note the difference between open and closed domains. Closed: what is on the line belongs to the domain Open: what is off the line – below or above – belongs to the domain X X X

7. Domain error A domain error occurs if an input traverses the wrong path through the program. We have no way to know the correct borders and there is no unique correct version of the program. When domain error occurs along a path, it may be thought of as being caused by one of the given borders being different from the correct one.

8. Path domains P1 P1 and P2: {S1, S3,S4} P1 and not P2: {S1, S3, S5} not P1 and P2: {S2, S3, S4} not P1 and not P2: {S2, S3, S5} S1 S2 S3 P2 S4 S5

9. ON and OFF points – 1 The test strategy is a strategy for selecting ON and OFF points, defined as follows: • ON point for a • Closed border lies on the border • Open border lies close to the border and satisfies the inequality relation • OFF point lies close to the border and on the open side or – alternatively – does not satisfy the path condition associated with this border

10. ON and OFF points – 2 The ON and OFF points are used as follows: • For testing a closed border we use • Two ON points to identify the border • One OFF point, to test that the correct border does not lie on the open side of the border • For testing an open border the role of the ON and OFF points are reversed. The strategy can be extended to N-dimensional space by using N ON points

11. ON and OFF points – 3 If the border line has V vertices, we will need • One ON point close to each vertex. • One OFF point per vertex at a uniform distance from the border. In all cases, it is important that the OFF points are as close as possible to the ON points

12. OFF ON Given border ON Example – two-dimensional space Correct border Example for an open border: ><, < or >. Border outside the line

13. Example – two-dimensional space Correct border OFF ON Given border ON OFF Example for an closed border: “=“ predicate. Border on the line

14. The problem of size The main problem with this strategy is the cost. Let us assume we have 20 input variables. Then the suggested strategy would need: • For each > or <: 20 ON points and one OFF point • For each = or ><: 20 ON points plus two or three OFF points Ten predicates would require 210 – 230 test cases.

15. The problem of precision The strategy described require the ON points to lie exactly on the border. For any pair of real numbers, there is always a third real number that lies between them. For a computer, however, this is not the case, due to limited precision. Thus, there exist borders for which no ON point can be represented in the computer.

16. A simplified strategy We will drop the requirement that the border can be exactly identified. Then we can also drop the requirement that the ON point lies exactly on the border. This remove the precision problem. In addition we can reduce the number of points by one per border. The only error that will not be detected is if the real border passes between an ON and an OFF point. Thus, these two points need to be close.

17. Simplified use of On and OFF points OFF OFF =, >< >, < ON ON OFF

18. Effectiveness • Assume that • m1 is the smallest real value • m2 is the largest real value • The length of L, called M, is • M = m2 – m1 +1 • P(not detect) = d / M OFF d >, < ON L

19. Code containing array references Code segment. The array b has 10 elements. i = x + 3; IF b[i] <=5 THEN ,,, ELSE … We need three predicates: • b[x + 3] <= 5 – the original predicate • x + 3 >= 1 – not below lower bound • x + 3 >= 10 – not above upper bound

20. Non-linear borders – 1 Everything discussed till now has been based on the assumption that the real border is linear. If this is not true, the domain strategy might fail. In the example on the next slide, the ON points are on the border, OFF point is off the border but nevertheless the given border is wrong.

21. Non-linear borders – 2 Correct border OFF ON Given border ON

22. A simple algorithm We can apply domain testing as follows: • Select a test case and run it. This will cause one path to be traversed. • Identify the borders of this path and test them with ON and OFF points • If the OFF point belongs to a new path then this path is selected for testing otherwise check another OFF point. • Terminate when no new path can be found

23. A 1 2 B C 3 Simple algorithm - example IF x in A THEN S1 ELSE IF x in B THEN S2 ELSE S3

24. When to use domain testing Domain testing, as it is described here, requires that we know how the input partition the input space into input domains. Thus, it is only possible to use it for small chunks of code.

25. Acknowledgement This set of slides is based on the paper “A Simplified Domain-Testing Strategy” by B. Jeng and E.J. Weyuker.