Input Domain Testing

1. Input Domain Testing • Input domain testing is based on applying the “partitioning” technique to categorize the input domain into sub-domains and test those sub-domains. • We may set the testing goals associated with (a) logic path coverage, (b) output domain coverage, etc. • In order to satisfy those testing goals, we design (or partition) the input sets that will provide the desired coverage • These designed input sub-sets (via assigning values to the input variables) are the input sub-domains that will provide the test coverage goals • In all these, we hope to evade exhaustive testing of all the possible input values ---- thus we use the partitioning technique of developing equivalence input sub-domains. • However, a major concern of testing input sub-domains is at the boundariesof the sub-domains.

2. Simple Example: “Drinking Age” Test • Requirement Statement: • The legal alcohol consumption age is 21 years old, for all the state. Thus before selling any alcohol to a person, the person’s birth-date is used to check if that person is of “legal 21 years of age.” • A programming solution may be developed in many ways to satisfy this requirement. One “pseudo-code“may look as follows (may be defective): Input variables birthDate, todayDdate; legalAge = 21; sumDate = legalAge + birthDate; if todayDdate > sumDate then output “Yes” ; else output “No” ;

3. Simple Example (cont.) • How do we design the test cases? One way is to consider the outputs (yes, no). What inputs will cover 100% of these outputs? (Taking a peek at the solution in pseudo-code) • Person’s birthday + legalAge > today’s date ----> “No” • Person’s birthday + legalAge < today’s date ----> “Yes” • Person’s birthday + legalAge = today’s date ----> “Yes” (birthdays that will result in “Yes” output) birthday = (today - legalAge) (birthdays that will result in “YES” output) birthday < (today – legalAge) (birthdays that will result in “NO” output) birthday > (today - legalAge) Legal-age Today’s date Birthday date

4. Simple Example (cont.) • The “values” for input of birthday may be partitioned into 3 sub-domains: • sub-domain A : person is older than 21 or Birthday < (today – legalAge) • sub–domain B : person is less than 21 or Birthday > (today – legalAge) • sub- domain C : person is exactly 21 or Birthday = (today – legalAge) • Note that • (A U B U C )of birthdays cover all possible birthdays • A∩B =Ø ; A∩C = Ø and B∩C = Ø • * We need to pick one birthday from each of sub-domains A, B and C respectively to test even though sub-domains A and B would fully cover the outputs of “yes” and “no” because see note below * Note :- The “boundary” for the sub-domains A and B is sub-domain C - We could have the “boundary” be part of sub-domain A - But if we mistakenly placed the boundary in sub-domain B, then we got a problem! !

5. Simple Example: Actual Testing • Assume we input today’s date as January 1, 2012. • Then any input of birth-date before January 1, 1991 will be in sub-domain A • Any input of birth-date after January 1, 1991 will be in sub-domain B. • Input of January 1, 1991 will in dub-domain C. • So, the test results: • October 10, 1982 should result in “Yes”; • October 16, 1991 should result in “No”; • and January 1, 1991 should also give us “Yes”. Unfortunately, the test results may not be what we expected !

6. Discussion on Boundary “Problem” • Note that “legal 21 years of age” is a key statement that sets the boundary. Some potential problem: • Does “legal 21 years of age” mean older than 21 or is exactly 21 years considered “legal” ----? • We interpret “legal” to include exactly 21 years and older. • This interpretation would say the boundary of “equal” to 21 years old should produce a “Yes” answer. • We could interpret “legal” to mean only older than 21 years. • This interpretation would say the boundary of “equal” to 21 years old should produce a “No” answer. Boundary Definition Clarity is very important !

7. A Sidetrack to “Boundary-Value” Testing • Boundary-Value Testing is a “common” testing technique not quite covered by your text book

8. Boundary Value Testing • A type of “Black box” functional testing • The program takes inputs and “maps” to some out-puts • The internal of the program itself is not considered • A technique to generate test cases via considering (mostly) the inputs to the program • The rationale for this focus is that there are experiences from the past which indicate that errors tend occur at the “extreme” points ----- “boundary” points. • Input data (legal versus illegal inputs --- the boundary) • Loop iteration (beginning and ending of loops) • Output fields (“yes” versus “no”)

9. A simple example • Consider a program that reads the “age” of each person here and computes the average age of the people. input (s) → Program → output: average age • How would you test this? • How many test cases would you generate? • What type of test data would you input to test this program?

10. Input (s) to Program Average • First question would be - - - how many input data? • The answer is some number : 1 or more --- not too sure, yet. • Second question would be --- what value should each input age be? • Try some typical age such as 23 or 45 • Try some atypical age 125 or 700 • How about trying a “wrong” age of -5 or 0 or @ • When we try the atypical age or some wrong age, we may discover that the program may not “handle” orprocess properly ---- possibly resulting in a failure. Failure in this case, may include strange answer, but not necessarily program termination.

11. Example: Program Age Average • Inputs composed of only typical ages: 23, 67, 97 • Look at the output average, and the average is computed either • Correctly • Incorrectly • Inputs composed of atypical or wrong ages • What is an atypical or wrong age? • Let us cap the age from 1 to 150. • So the lower and upperboundaries for age is 1 and 150, respectively.

12. Boundaries of the inputs 1 <= age <= 150 150 age 1 The “basic” boundary value testing would include 5 situations: 1. - at minimum boundary 2. - immediately above minimum 3. - between minimum and maximum (nominal) 4. - immediately below maximum 5. - at maximum boundary This is really not any different from partitioning inputs into 5 sub-domains, except that we use the upper and lower “boundaries”as our guideline for partitioning

13. The Test Cases for this Example • How many test cases for this example ? • answer for this one field : 5 (would be 7 if we included “outside” of the upper and lower boundaries) • What types of test input values? : 1 at the minimum 2 at one above minimum 45 at middle 149 at one below maximum 150 at maximum

14. Two “independent” inputs • For the previous problem, we talked about 1 input field. Suppose there are 2 “distinct” inputs that are assumed to be independent (single fault) of each other. • Input field 1: years of education ( say 1 to 23 ) • Input field 2: age • If they are independent of each other, then we can start with looking at 5 + 5 = 10 sets, but won’t really need all 10 of them. • coverage of input data: yrs of ed • 1. n= 1 ; age = whatever(37) • n =2; age = whatever • 3. n = 12; age = whatever • n = 22; age = whatever • n = 23; age = whatever • coverage of input data: age • 1. n= 12; age = 1 • n =12; age = 2 • 3. n = 12; age = 37 • n = 12; age = 149 • n = 12; age = 150

15. 2 – “independent” inputs for Boundary-Value test age yrs of ed. - Note that there needs to be only 9 test cases for 2 independent variables or inputs. - In general, there will be (4z + 1)test cases for z “independent” inputs.

16. “Robustness” testing • This is just an extension of the Boundary Values to include: • Less than minimum • Greater than maximum • There are 7 cases or values to worry about for each independent variable input. • Note: A lot of the testing of robustness is really a test of “error” handling. • Do we anticipate the error situations? • Do we issue informative error messages? • Do we allow some kind of recovery from the error? Levels of Problem Severity?

17. 2 – independent inputs for “Robustness” test X Y - Note that there needs to be only 13 test cases for 2 independent variables or inputs. - In general, there will be (6z+ 1)test cases for z independent inputs.

18. What if the input domains are NOT independent, but dependent? • For Boundary Value Testing: If the inputs are dependent, then for each of the 5 test cases for input 1, there will be 5 potential combinations for each of the other inputs . Thus there needs to be 5ztest cases, where z is the number of unique inputs. • For Robustness Testing of Boundary Values: If the inputs are dependent, then for each of the 7 test cases for input 1, there will 7 be potential combination for each of the other inputs. Thus there needs to be 7ztest cases, where z is the number of unique inputs.

19. Some More Discussions on Boundary Value • There may be different problems associated with boundary value definition: (Consider the requirement “if age > 3 then --- else -----” and using input domain test case values of age 5, 2, and 3) • Closure problem is not testing the “open” boundary. Testing with 5 behaves as “then---”, and testing with 2 behaves as “else ---”. The two logical paths are covered!We need to ensure that we test age = 3 and the behavior is “else ---”. • * Boundary shift problemis when the boundary shifted. Assume the boundary mistakenly shifted to “age > 4”in code. Then testing with 5 would still behave as “then---”; and testing with 2 still behaves as “else ---”. Now, testing with 3 will produce “else ---” behavior. All the results is the same as before ---- thus there is an error in implementation that can go on undetected. (Interesting “mutation” problem !!!) • Missing or unclear boundary would be problem from requirements to code. Missing boundary in implementation can be caught with input domain test, but missing or unclear boundary in requirement would not. • Extra boundary in the implementation may/may not be detected. If we put in an extra “if age = 0 then error message” in the code, depending on where this is placed in the code, the input domain test of (5,2,3) may not catch this.

20. Two Problems from the previous list of Input Domain Testing with Boundary Value • Boundary Shifting (in the implementation) can go on undetected. • Extra boundary (placed into the implementation) can go on undetected.

21. A Deeper Look at “Dependent” Inputs & Boundary Testing • Consider the following problem statement: • The bank X will give a “premier” interest for those customers whose (1)fixed-saving account and (2)money-market account deposits add up to $10k or more. Assume all accounts are in denomination of dollars (i.e. no cents). • The implementation pseudo code may look as follow: Input integer sav , mm; sum = sav + mm; If sum ≥ 10000 then “premier” interest rate; else “regular’ interest rate;

22. Testing Interest Rate Specification • We can do a boundary value testing of the variable sum, where sum> 10k, sum = 10k or sum < 10k. • But if we were to use “black-box” testing and we do not even have the pseudo code, then we have to devise the test with two inputs: fixed-savings account (sav) and money-market account (mm). That is, we need consider both inputs, “sav” and “mm”, and their dependency where (sav + mm) ≥ 10,000.

23. Testing Interest Rate Specification (cont.) Sav Accnt ($k) 14 12 10 8 Sav + MM ≥ 10K gives “premier” interest 6 4 “regular” interest 2 MM Accnt ($k) 0 2 4 6 8 10 12 14

24. Testing Interest Rate Specification (cont.)with “ON-OFF-ON” (Weak N x 1) strategy We would pick 2 points, A and B, ON the boundary and 1 OFF the boundary point, C, in between A and B as test cases (as close to the boundary as possible) A : sav = 8 and MM = 2 => “premier” interest B : sav = 2 and MM = 8 => “premier” interest C : sav = 5 and MM = 4 => “regular” inerest Sav accnt 14 12 10 8 A 6 We will illustrate with C that is “close” 4 C B 2 MM accnt 2 4 6 8 10 12 14

25. Testing Interest Rate Specification (cont.)with “ON-OFF-ON” (Weak N x 1) strategy We would pick 2 points, A and B, ON the boundary and 1 OFF the boundary point, C, in between A and B as test cases A : sav = 8 and MM = 2 => “premier” interest B : sav = 2 and MM = 8 => “premier” interest C : sav = 5 and MM = 4 => “regular” inerest Sav accnt 14 12 10 If the boundary shifted “inward” due to mistaken (mutated) implementation, Sav + MM ≥ 9, we will catch it because C : sav = 5 and MM = 4 would be “premier” interest 8 A 6 4 C B 2 MM accnt 2 4 6 8 10 12 14

26. Testing Interest Rate Specification (cont.)with “ON-OFF-ON” (Weak N x 1) strategy We would pick 2 points, A and B, ON the boundary and 1 OFF the boundary point, C, in between A and B as test cases A : sav = 8 and MM = 2 => “premier” interest B : sav = 2 and MM = 8 => “premier” interest C : sav = 5 and MM = 4 => “regular” inerest Sav accnt 14 12 10 If the boundary shifted “outward” due to mistaken (mutation) implementation, Sav + MM ≥ 12, we will catch it because A : sav = 8 and MM = 2 => “regular” interest B: sav = 2 and MM = 8 => “regular” interest 8 A 6 4 C B 2 MM accnt 2 4 6 8 10 12 14

27. Testing Interest Rate Specification (cont.)with “ON-OFF-ON” (Weak N x 1) strategy We would pick 2 points, A and B, ON the boundary and 1 OFF the boundary point, C, in between A and B as test cases A : sav = 8 and MM = 2 => “premier” interest B : sav = 2 and MM = 8 => “premier” interest C : sav = 5 and MM = 4 => “regular” inerest Sav accnt 14 12 10 If the boundary “tilted” due to some Strange mistaken implementation, we will catch it because Either A or B will turn to “regular” 8 A 6 4 C B 2 MM accnt 2 4 6 8 10 12 14