Black Box Testing

1. Black Box Testing Csci 565 Chapters 5-7

2. Objectives • Systematic Specification-based testing • Decision Table-Based Testing • Cause-Effect Graphs in Functional testing • Equivalence Partitioning and Boundary value Analysis • Input validation and Syntax-driven Testing

3. Black box testing • Black box testing • Also known as data-driven or input/output driven testing • Views the software as a black box • Aims at finding cases in which the software does not behave according to its specifications or requirements • Test data are derived from the specifications • (no need for taking advantage of knowledge of code) • The main criterion is exhaustive input testing!!!!

4. Decision table-based Testing (DTT) • Applicable to the software requirements written using “if-then” statements • Can be automatically translated into code • Conditions = inputs • Actions = outputs • Rules = test cases • Assume the independence of inputs • Example • If cond1 AND cond2 OR cond3 then Action1

5. Sample of Decision table • A decision table consists of a number of columns (rules) that comprise all test situations • Action ai will take place if c1 and c2 are true • Example: the triangle problem • C1: a, b,c form a triangle • C2: a=b • C3: a= c • C4: b= c • A1: Not a triangle • A2:scalene • A3: Isosceles • A4:equilateral • A5: impossible

7. Test cases from Decision Tables

8. Example: Simple editor • A simple text editor should provide the following features • Copy • Paste • Boldface • Underline

9. In general, for n conditions, we need 2n rules

10. Decision tables as a basis for test case design • The use of decision-table model to design test cases is applicable • The spec is given by table or is easily converted to one • The order in which the conditions are evaluated does not affect the interpretation of rules or the resulting action • The order in which the rules are evaluated does not affect the resulting action • Once a rule has been satisfied and an action is selected, no other rule need be examined • If multiple actions result from the satisfaction of a rule, the order in which the actions take place is not important • Inconsistency problem and nondeterministic table • When two rules with same conditions elevated to two different actions then the whole decisions tables is nondeterministic because it is difficult to decide which rule should be applied

11. The implications of rules • The above conditions have the following implications • Rules are complete (i.e., every combination of decision table values including default combinations are inherent in the decision table) • The rules are consistent (i.e., there is not two actions for the same combinations of conditions)

12. Cause-effect graphs in black box testing • Captures the relationships between specific combinations of inputs (causes) and outputs (effects) • Deals with specific cases • Avoids combinatorial explosion • Explore combinations of possible inputs • Causes/effects • represented as nodes of a cause effect graph • The graph also includes a number of intermediate nodes linking causes and effects

13. Drawing Cause-Effect Graphs A B If A then B (identity) A C B If (A=true and B=true)then C

14. Drawing Cause-Effect Graphs A C B If (A or B) then C A C  B If (not(A AND B)) then C

15. Drawing Cause-Effect Graphs A C  B If (not (A OR B))then C  A B If (not A) then B

16. Constraint Symbols a at most, one of a and b can be 1 (i.e., a and b cannot be 1 at the same time) E b a Exactly one of A and B can be 1 O b a I b at least one of a, or b must be 1 (i.e., a and b cannot be 0 at the same time)

17. Example: ATM • For a simple ATM banking transaction system • Causes (inputs) • C1: command is credit • C2: command is debit • C3: account number is valid • C4: transaction amount is valid • Effects (outputs) • E1: print “invalid command” • E2: print “ invalid account number” • E3: print “debit amount not valid” • E4: debit account • E5: credit account

18.  E1 or C1 and E2  C2 and E3  C3 and E4 C4 E5 and

19. Description of processing nodes used in a Cause-effect graph ()

20. ATM Cause-effect decision table Don’t Care condition

21. C/E: Print Message problem The "Print message" is a software that read two characters and, depending of their values, messages must be printed. • The first character must be an "A" or a "B". • The second character must be a digit. • If the first character is an "A" or "B" and the second character is a digit, the file must be updated. • If the first character is incorrect (not an "A" or "B"), the message X must be printed. • If the second character is incorrect (not a digit), the message Y must be printed.

22. Step 1: Identify Causes and Effect • Causes: • 1 - first character is "A" • 2 - first character is "B" • 3 - second character is a digit • Effects: • 70 - the file is updated • 71 - message X is print • 72 - message Y is print

23. Step 2: Draw the Graph

24. Step 2: Improved Graph Exclusive Or (i.e., 1 and 2 cannot be true at same time)

25. Step 3: Convert the Graph into Decision table

26. Steps to create cause-effect graph • Study the functional requirements and divide the requirements into workable pieces • Identify causes and effects in the specifications • Causes: distinct input conditions • Effects: an output condition or a system transformations. • Assign unique number to each cause and effect • Use the semantic content of the spec and transform it into a Boolean graph • Annotate the graph with constrains describing combinations of causes and/or effects • Convert the graph into a limited-entry decision table • Use each column as a test case

27. Black-box testing

28. Equivalence partitioning • Black box testing that involves identifying a small set of input values that produces as many different output conditions as possible • Input data and output results often fall into different classes where all members of a class are related • Each of these classes is an equivalence partition where the program behaves in an equivalent way for each class member • Test cases design involves • Identifying all partitions satisfying the input values for a product • Identifying all partitions satisfying the expected outputs for a product • Selecting one member value from each partition to maximize complete coverage

29. Equivalence partitioning

30. Guidelines for equivalence classes • If an input condition specifies range, • one valid and two invalid equivalence classes are needed • If an input condition requires a specific value, • then one valid and two invalid equivalence classes are needed • If an input condition specifies a member of a set • one valid and one invalid equivalence class are needed • If an input condition is Boolean, • one valid and one invalid class are needed

31. Example: ATM • Consider data maintained for ATM • User should be able to access the bank using PC and modem (dialing) • User needs to provide eight-digit password • Need to follow a set of typed commands representing various transactions

32. Data format for phone(xxx-xxx-xxxx) • Software accepts • Area code: • blank or three-digit • Prefix: • three-digit number not beginning with 0 or 1 • Suffix: • four digits number • Password: • eight digit alphanumeric value • Command: • {“check”, “deposit,” “ bill pay”, “transfer” etc.}

33. Input conditions for ATM • Input conditions • Area code: • Boolean: (the area code may or may not be present) • Range: values defined between 200-999 • Specific value: no value > 905 • Prefix: • range : specific value >200 • Suffix: • Value: four-digit length • Password: • Boolean: password may or may not be present • value : eight char-string • Command: • set containing commands noted previously

34. Boundary Value Analysis (BVA) • Complements equivalence partitioning • Focuses is on the boundaries of the input • If input condition specifies a range bounded by a certain values, say, a and b, then test cases should include • The value for “a” • The value for “b” • The values just above “a” • The values just below “a” • The values just above “b” • The values just below “b” • The nominal value • If an input condition specifies any number of values, test cases should be exercise • the minimum and maximum numbers, • the values just above and just below the minimum and maximum values

35. Example 2: Equivalence Partitioning (EP)

36. EP: Process • Identify valid partitions • Identify invalid partitions • Identify output partitions

37. Input: Valid partitions • The valid partitions can be • 0<=exam mark <=75 • 0<=course work <=25

38. Input: Invalid partitions • The most obvious partitions are • Exam mark > 75 • Exam mark < 0 • Coursework mark > 25 • Coursework nark <0

39. Exam mark and course work (c/w) mark

40. Less obvious invalid input EP • Invalid INPUT EP should include

41. Partitions for the OUTPUTS • EP for valid OUTPUTS should include FM

42. The EP and boundaries • The EP and boundaries for total mark

43. Unspecified Outputs • Three unspecified Outputs errors can be identified • Output = “E” (errors below zero) • Output = “A+” (errors exceeding 100) • Output = “null” (errors corresponding to blanks)

44. Total PEs Input partitions for exams Input partitions for course works Input partitions for invalid data Output partitions for valid data Output partitions for special cases

45. Test Cases corresponding to PE exam mark (INPUT)

46. Test Case 4-6 (coursework)

47. test case for Invalid inputs

48. Test cases for outputs:1

49. Test cases for outputs:2

50. Test cases for Special output classes:3