White Box Testing and Symbolic Execution

1. White Box Testing and Symbolic Execution Written by Michael Beder

2. Agenda • What is White Box Testing? • Flow Graph and Coverage Types • Symbolic Execution: • Formal Definition • Examples • Questions White Box Testing and Symbolic Execution

3. What is White Box Testing? • Software testing approach that uses inner structural and logical properties of the program for verification and deriving test data • Also called: Clear Box Testing, Glass Box Testing and Structural Testing • Manual: Inspection, Walkthrough • Automatic: Syntax Parser, Symbolic Execution White Box Testing and Symbolic Execution

4. Pros and Cons • Pros: • Usage of more information on the tested object • (than BlackBox) • Inference of real Equivalence Partitioning • Structural Coverage Assurance • Cons: • Expensive • Limited Semantic Coverage White Box Testing and Symbolic Execution

5. Example I Sin(x) { if (|x| < eps) return x; if (|x| < 5*eps) return x – (x^3)/6; … } • Black Box Testing: Test 0, Pi/2, -Pi/2 • White Box Testing: Test 0.5*eps, eps, 3*eps, 5*eps, 7*eps, … Usage of logical properties makes better coverage White Box Testing and Symbolic Execution

6. Example II: Unit Testing f(x) g(x) { h(x) { { … … if (x > 0) return g(x); } } return h(x); } • Black Box Testing: Test f(x), g(x), h(x) for every x • White Box Testing: Test g(x) for x > 0, h(x) for x <= 0 and verify f(x) Usage of structural properties makes fewer, qualitative tests White Box Testing and Symbolic Execution

7. Flow Graph • Abstraction of the program • Defines the data and control flow in the program • Uniform representation of the program, language independent • Simple basic elements: assignment and condition • Further analysis is performed using Graph Algorithms White Box Testing and Symbolic Execution

8. start 1, 2 6 3 4 5 end Flow Graph – cont. • G = (V, E) when V is a set of basic blocks. start, end in V E is a set of control branches Example: 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } F T Input: b = 2 F Output: a = 0, c = 2 T White Box Testing and Symbolic Execution

9. White Box Coverage Types • Statement Coverage: Every statement is executed • Branch Coverage: Every branch option is chosen • Path Coverage: Every path is executed • Basic Path Coverage: Every basic path is executed • All definition-use path Coverage: All paths between the definition and a usage of a variable are executed Loops? White Box Testing and Symbolic Execution

10. start 1, 2 6 3 4 5 end Basic Path Coverage • Let p1, p2 be paths from start to end. Then, p1 < p2 if exists vertex v such that p2 contains v and p1 doesn’t contain v • Find maximal set of paths p1, p2, …, pk such that pi < pj for i < j • Such set of paths is of size E – N + 2 (Linear Complexity) • Each path is called “basic path” • Example: p1 = start – 1,2 – 3 – end p2 = start – 1,2 – 3 – 4 – 6 – 3 – end p3 = start – 1,2 – 3 – 4 – 5 – 6 – 3 – end E – N + 2 = 8 – 7 + 2 = 3 F T F T White Box Testing and Symbolic Execution

11. Path Function • A function , when D is the working domain • Represents the current values of the variables as function of their initial values • Each variable X is represented by a projection function • Function composition: • For example: White Box Testing and Symbolic Execution

12. start 1, 2 6 F 3 T F 4 5 T end Path Condition For Example: p = start – 1,2 – 3 – end. 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } The path condition is B <= 1, when B is b’s value at start • A condition that should be fulfilled for going along the path • A constraint on the initial values of the variables White Box Testing and Symbolic Execution

13. Symbolic Execution • A method for deriving test cases which satisfy a given path • Performed as a simulation of the computation on the path • Initial path function = Identity function, Initial path condition = true • Each vertex on the path concludes a symbolic composition on the path function and a logical constraint on the path condition: If an assignment was made: If a conditional decision was made: path condition path condition branch condition Output: path function and path condition at the final vertex of the path White Box Testing and Symbolic Execution

14. Example III x = x + y y = y + x end The final path function represents the values of X, Y, Z after both assignments as a function of their initial values White Box Testing and Symbolic Execution

15. V1 Vn Vk Symbolic Execution is a special case when Concatenation and Associativity If is the path function of path and is the path function of path then is the path function of path The composition is associative: White Box Testing and Symbolic Execution

16. start 1, 2 6 F 3 T F 4 5 T end Example IV 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } Find test case for path: p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end input = b output = c White Box Testing and Symbolic Execution

17. start 1, 2 6 3 4 5 end Example IV 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a - 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertex path function path condition start: (A, B, C) true 1,2 (A, B, C) true 3 (B, B, 0) true 4 (B, B, 0) (true Λ B>1) ↔ B>1 5 (B, B, 0) (B>1 Λ B^2>0) ↔ B>1 input = b output = c F T F T White Box Testing and Symbolic Execution

18. start 1, 2 6 3 4 5 end Example IV 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 If (a^2 > c) 5 c = c + a 6 a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertex path function path condition 6 (B, B, B) B>1 3 (B-2, B, B) B>1 4 (B-2, B, B) (B>1 Λ B-2>1) ↔ B>3 5 (B-2, B, B) (B>3 Λ (B-2)^2>B) ↔ B>4 6 (B-2, B, 2B-2) B>4 3 (B-4, B, 2B-2) B>4 end (B-4, B, 2B-2) (B>4 Λ B-4<=1) ↔ B=5 input = b output = c F T F T White Box Testing and Symbolic Execution

19. start 1, 2 6 F 3 T F 4 5 T end Example IV 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4 if (a^2 > c) 5 c = c + a 6 a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end end (B-4, B, 2B-2) B=5 Hence the test case is B = 5 and the expected result is 2B-2 = 8. input = b output = c Is there a test case for p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end ? White Box Testing and Symbolic Execution

20. Question (from exam) • 1 d = b + c; • 2 if (d > 20) • 3 a = 3 * a + d; • if (b < a) { • 5 a = 1; • 6 if (d < 2 * b) • 7 b = 2; • 8 } • Draw program’s Flow Graph • Find minimal number of test cases for the following coverage types: • Statement Coverage • Path Coverage • Branch Coverage • Basic Path Coverage White Box Testing and Symbolic Execution

21. White Box Testing vs. Black Box Testing Given a function f(X1, X2, …, X10) with the following preconditions: • Every parameter is odd • Every parameter is less or equal to M • Some parameter is equal to M The function should report about every precondition that is not fulfilled f examines each parameter in turn using if statements (without else) and handles differently the following cases: • Exactly one parameter is higher than M • Two or more parameters are higher than M Check f’s correctness using White/Black Box Testing methods White Box Testing and Symbolic Execution