CS 217 Software Verification and Validation

1. CS 217 Software Verification and Validation Week 8, Summer 2014 Instructor: Dong Si http://www.cs.odu.edu/~dsi

2. REVIEW OF BB-testing

3. Black-box Testing • Input Space Partitioning • Boundary Value Analysis

4. Example 1: compare two numbers – p50 of week3 • Function ‘Compare (x, y)’ • Inputs: Two numbers – x and y • Outputs: A larger number between x and y (x, y) z Compare (x, y) = z

5. – p51 of week3 • Equivalence Classes: { (x, y) | x < y } { (x, y) | x > y } { (x, y) | x = y } { input other than a pair of numbers, “as&%dfget^$(&w” } Valid inputs Invalid inputs

6. – p52 of week3 Valid (x, y) Input Space Three test cases: (1, 2) --- 2 (8, 8) --- 8 (100, 30) --- 100 Plus one test cases: (^&%*) --- ERROR x = y x < y x > y

7. Example 2: Loan application-p53 of week3 2-64 chars. Customer Name Account number Loan amount requested Term of loan Monthly repayment Term: Repayment: Interest rate: Total paid back: 6 digits, 1st non-zero $500 to $9000 1 to 30 years Minimum $10 Choosing (or defining) partitions seems easy, but is easy to get wrong…

8. Number of characters: 1 2 64 65 invalid valid invalid A-Z Valid characters: Anyother a-z -’ space Customer name

9. 499 500 9000 9001 invalid valid invalid Loan amount

10. Black-box Testing • Input Space Partitioning • Boundary Value Analysis

11. Example 3 - Bonus • Consider a program IsAnOddNumber(x)which output “Yes” or “No” to the user to tell if the input integer x is a odd number or not. If you are planning to classify all possible user inputs into two classes - valid and invalid inputs • How do you partition the valid input space (the integer number space)? • Can you give an example of invalid test data?

12. Example 3 - Bonus • If your software manager asks the program MUST ONLY take input real number x as: -100<x<100 • How do you partition all possible user inputs into two classes - valid and invalid inputs now? • If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis?

13. Example 4 - Bonus • Consider a program Absolute(x)which calculate the absolute value of a input real number x. If you are planning to classify all possible user inputs into two classes - valid and invalid inputs • How do you partition the valid input space (the real number space)? • Can you give an example of invalid test data?

14. Example 4 - Bonus • If your software manager asks the program MUST ONLY take input real number x within the range of: -321<x<-123, 123<x<321 • How do you partition all possible user inputs into two classes - valid and invalid inputs now? • If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis?

15. Today’s topicWHITE-BOX TESTING

16. Styles of Testing • Testing traditionally can be conducted in two styles • Black-Box testing Try to choose "smart" tests based on the requirements, without looking at the code. • White-Box testing Try to choose "smart" tests based on the structure of the code, with minimal reference to the requirements.

17. Definition of White-Box Testing • Testing based on analysis of internal logic (design, code, etc.). (But expected results still come from requirements.) • Use different techniques to check every visible path of the source code to minimize errors. • It is the ability to know which line of the code is being executed and being able to identify what the correct output should be.

18. Characters of WB testing • Exploits the structure, control or data flow of programs in order to design test cases. • Typically performed during coding. • Allows to test parts of the program, since you know the structure. (with black‐box is not possible) • Allows to cover systematically every part of the program.

19. WB Testing Techniques • Logic coverage: (learned in previous classes) • Statement coverage • Branch coverage • Condition coverage • … • Dataflow based testing / Path coverage: all program paths have been traversed at least once

20. Pseudo code & Control/Dataflow Graphs “nodes” “edges” Input x Absolute (x) { IF (x>=0) THEN y = x; ELSE IF (x<0) THEN y = -x; Output y; } IF (x>=0) ELSE IF (x<0) y = -x; y = x; y output

21. Can you draw a graph? Absolute (x) { IF (x>0) THEN y = x; ELSE IF (x==0) THEN y = x; ELSE IF (x<0) THEN y = -x; Output y; }

22. Example Fun (x, y) { z = 0; IF ((x>z) or (y>z)) THEN z = 2; ELSE z = 3; Output z; } Test cases: Fun (0, 0) Fun (2, 0) Fun (0, 2) Fun (2, 2) Fun (8, 9)

23. Control/Dataflow of the example Test cases: Fun (0, 0) Fun (2, 0) Fun (0, 2) Fun (2, 2) Fun (8, 9) Input x, y Fun (x, y) { z = 1; IF ((x>z) or (y>z)) THEN z = 2; ELSE z = 3; Output z; } z = 1; IF ((x>z) or (y>z)) ELSE z = 3; z = 2; Output z;

24. Statement coverage • Has each statement in the program been executed? √ √ √

25. Branch coverage • Has each branchof each control structurebeen executed? • For example, given an if statement, haveboth the T and F branchesbeen executed? • Another way of saying this is, hasevery edgein the Control/Dataflow graph been executed?

26. Condition coverage • Has each Boolean sub-expressionevaluated both to true (T) and false (F) ? Fun (x, y) { z = 1; IF ((x>z) or (y>z)) THEN z = 2; ELSE z = 3; Output z; } Test cases: Fun (0, 0) Fun (2, 0) Fun (0, 2) Fun (2, 2) Fun (8, 9) Y>1 X>1 T , F ? T , F ?

27. Exercise • Consider a program ‘Compare’ which compare the value between two numbers. The inputs are two real numbers x and y, the program outputs the larger number between x and y. Compare (x, y) { IF (x>y) THEN z = x; ELSE IF (x==y) THEN z = x; ELSE IF (x<y) THEN z = y; Output z; }

28. Exercise - Control/Data Flow Graphs • I have provided you the Pseudo code, • Can you draw a Control/Dataflow Graphs? • Try it! This is a bonus question with +5 points towards to FINAL!!

29. Exercise - Statement Coverage • Statement Coverage requires that each statement will have been executed at least once. • What is the minimum number of test casesrequired to achieve statement coveragefor the program Compare (x, y) ?

30. Exercise - Branch Coverage • Branch Coverage requires that each branch will have been traversed at least once. • What is the minimum number of test cases required to achieve branch coveragefor the program Compare (x, y) ?

31. Condition Coverage • Condition Coverage requires that each condition (sub-expression)will have been True at least once and False at least once. • What is the relationship between Branch and Condition Coverage?

32. Condition / branchcoverage IF ( and ) THEN … ELSE … Y>0 X>0 T , F ? T , F ? T F

33. Condition Coverage example – Combination ! IF (Aor B) THEN s1; ELSE s2; Combinations of condition values: TT, TF, FT, FF

34. Dataflow based testing -> Path coverage • All program paths have been traversed at least once. Input x, y z = 1; IF ((x>z) OR (y>z)) ELSE Two paths ! z = 3; z = 2; Output z; 34

35. Path coverage - Example 1 if a then s1 else if b then s2 else if c then s3 else s4 end_if_then_else end_if_then_else end_if_then_else # Paths: ___ # Branches: ___

36. WB Testing Techniques • Logic coverage: (learned in previous class) • Statement coverage • Branch coverage • Condition coverage • … • Dataflow based testing / Path coverage: all program paths have been traversed at least once

37. Coming Up Next week… • Lab 2: White-box Testing • Preparation: • Go over the slides of WB testing • Understand “Code coverage” & “Data flow / Path”