CS4311 Spring 2011 Unit Testing Dr. Guoqiang Hu Department of Computer Science UTEP

1. CS4311 Spring 2011 Unit Testing Dr. GuoqiangHuDepartment of Computer ScienceUTEP

2. Announcement • What: CS4390 (Cross-listed with CS5390): “Software Testing” Testing Only, Nothing Else! • When: This Summer • Prerequisite: CS4311 * Note: The V&V content covered in this course (CS4311) can be considered as an introduction to the above course

3. Basic Concepts Unit: • Function • Method • Routine • Class (OO) Unit testing: • Testing the unit • Tested in isolation from the rest of the system • Tested in a controlled environment: Use appropriately chosen input data

4. Unit Testing Procedures Driver Unit to be tested Results Test cases . . . Stub Stub

5. Two Fundamental Approaches Black box: • Based on specification • The inner logic structure of the unit is not considered White box: • Based on specification • Based on the inner logic structure of the unit to select test cases Effectiveness of the two approaches: • Black box is similar to white box, but the faults found are different. (Hetzel 1976, Myers 1978)

6. Black Box Testing • Test set is derived from specification (also known as functional testing) • Goal is to cover the input space with a minimum number of test cases • The normal approaches to describing input space: • Equivalence Classes • Boundary Analysis • Decision Table • State Transition • Use Cases

7. Equivalence Classes Method Equivalence relation: • Reflexive: x R(x,x) • Symmetric: x,y R(x,y)  R(y,x) • Transitive: x,y,z R(x,y)  R(y,z)  R(x,z) The rationale: The equivalence partitioning for software testing is based on two motivations: to have a sense of complete testing, and at the same time, to avoid test cases redundancy. Through equivalence partitioning, the domain of input can be partitioned into equivalence classes: the union of which is the entire domain, but all the equivalence classes are mutually disjoint with each other. For testing purpose, each value from the same equivalence class is equivalent to each other. In this way, we can use the minimum number of test cases, but at the same time, we can still have a sense that the testing is complete. The strategy: • Identify the equivalence relation • Partition the input into equivalence classes • Select only one input value from each equivalence class

8. Equivalence Classes Method: Tips • Identify restrictions for inputs in the specification. • If a restriction on a continuous numerical domain is specified, create one valid and two invalid classes (i.e., above, below). • If a set of values is specified where each may be treated differently, create a class for each element of the set and one more for elements outside the set. • If there is a condition, create two classes, one satisfying and one not satisfying the condition.

9. Boundary Analysis Method Boundary analysis focuses on the boundary of the input space to identify test cases. (Used alone, or, as a supplement; Black, Gray, or White) The rationale behind boundary analysis is that errors tend to occur near the extreme values of input variables: • It is often used to catch “off-by-one” errors. • Also, it is used to catch the extreme compound effects of a group of variables to see if the results have passed some boundaries. Tips: • If a domain is a restricted set, check the boundaries. e.g., D=[1,10], test 0, 1, 10, 11 • For ordered sets, check the first and last elements and their immediate neighbor outside the set. • For dynamic complex data structures, check for its extreme sizes, for example: the empty list, full list, etc. • Extremely large data sets should be tested (limits (e.g., max array size)) • The null pointers should be tested • Check for off-by-one errors • Also, check the boundaries of outputs.

10. The Exhaustive Test Examples The function fn shall take as input an integer value and return the integer part of the one number look ahead divided by 30,000. Consider testing the following code with an error (The first line should be: j = j + 1;) public static int fn (int j) { j = j - 1; j = j / 30000; return j; } Would any of your tests uncover this error? For 16-bit integers, errors at: -30,000 -29,999 29,999 30,000 4 cases out of 65,536 possible

11. Decision Table Method Decision table method is used for situations in which a number of combination of actions are taken under varying sets of conditions. This structure guarantees that every possible combination of conditions is considered. To identify test cases with decision tables, we interpret conditions as inputs and actions as outputs. A simplified version of decision table method works like this: • Identify each rule or condition in the system that depends on some input • For each input to one of these rules, list the combinations of inputs and the expected results • Construct a table using the above information.

12. State Transition Method Build STD for the unit or system, then select test cases that will cover the STD: • Visit each state at least once • Trigger each event at least once • Exercise each transition at least once • Exercise each path (might not be possible)

13. Use Case Method Use the use cases/scenarios to specify test cases: • Normal • Alternate • Exceptional

14. White Box Testing • Test set is derived from the inner logic structure of the unit (also known as glass box testing or structural testing) • Goal is to cover all the inner logic with a minimum number of test cases • Unit inner logic structure is normally represented as Control Flow Graph (CFG) • The normal categories of logic coverages: • Statement • Branch • Condition • Path • Data Define-Use

15. Control Flow Graph • Structure Programming: Each program and each program construct (called a block) in it should only have a single-entry and single-exit control structure. A structured program progresses in an orderly, disciplined way, rather than jumping around unpredictably. You can read it from top to bottom, and it executes in much the same way. Structured programming uses only the following three kinds of blocks: • Sequence • Selection • Iteration • CFG: a visual representation of the flow of control of a unit: • Node represents a sequence of statements with single entry and single exit • Edge represents transfer of control from one node to another

16. Control Flow Graph: Notations n1 n1 n1 n3 Join Join Sequence If-then-else Iterative If-then When drawing CFG, ensure that there is only one exit: use a join node if needed.