SOFTWARE TESTING TECHNIQUE

1. SOFTWARE TESTING TECHNIQUE RACHAPONG PORNWIRIYANGKURA 1 FEBRUARY 2010

2. AGENDA • OVERVIEW • BLACK BOX TESTING • WHITE BOX TESTING

3. OVERVIEWWHAT IS SOFTWARE TESTING? • What is Software Testing? “Testing is the process of demonstrating that errors are not present” “The purpose of testing is to show that a program performs its intended functions correctly.” “Testing is the process of establishing confidence that a program does what it is supposed to do.” “Testing is the process of executing a program with the intent of finding errors”

4. OVERVIEWWHY DO WE NEED SOFTWARE TESTING? • Testing can reduce overall software cost. It has been shown in the industry that finding defects in the early state can reduce huge amount of support and maintenance cost. • Black, president and principal consultant of RBCS, emphasized the cost-effectiveness of creating quality software. "The money you spend to build it right the first time is always less than the money it costs to fix it," he said. The amount of money saved can be two times, four times or even 32 times the cost of investment, he continued.

5. OVERVIEWBENEFIT OF SOFTWARE TESTING Cost Cost of fixing defects Increase over time Requirement Design Code Test Operation Development Phase

6. OVERVIEWBENEFIT OF SOFTWARE TESTING • Protect reputation of organization

7. OVERVIEWWHY SOFTWARE TESTING TECHNIQUE • It is almost impossible to find all defects even with a trivial program • Testing technique is used to catch defects effectively. It will help determining the subset of test cases that has the highest priority of finding defects. • “Testing never end but it just stops”

8. TIPS & TRICKS • Error: A human action that cause incorrect result • Defect: A flaw in the system that can cause undesired result • Failure: Unexpected results of a system • Not all defects will cause failure as some of them might stay in the code that never reach. • Exhaustive Testing is a way of testing with tries to find all errors by using every possible inputs.

9. CATEGORIES OF SOFTWARE TESTING • Several ways of categorize software testing • Based on knowledge of system • e.g. black box vs white box • Based on time in development cycle • e.g. system test vs unit test • Based on method to verify software • e.g. dynamic vs static

10. CATEGORIES OF SOFTWARE TESTING

11. AGENDA • OVERVIEW • BLACK BOX TESTING • WHITE BOX TESTING

12. BLACK BOX TESTING “A test design technique based on the view of system by looking at only inputs and output without the knowledge of how system is structured.” • Concentrate on what the system does not how it does • Black Box Test Technique • Equivalence Partitioning • Boundary Value Analysis • Decision Table Testing • State Transition testing

13. AGENDA • OVERVIEW • BLACK BOX TESTING • EQUIVALENCE PARTITIONING • BOUNDARY VALUE ANALYSIS • DECISION TABLE TESTING • STATE TRANSITION TESTING • WHITE BOX TESTING

14. BLACK BOX TESTING EQUIVALENCE PARTITIONING (EP) “A black box test technique in which test cases are designed to execute representatives from equivalence partitions” • Dividing input or output of the software into groups that has similar behavior • Partition can cover both valid and invalid input/output • Select value from each partition as a representative with the assumption that if one value works all values in that partition will work. • No need to test all values in a partition but need to test all partitions as it has more possibility to find a defect.

15. BLACK BOX TESTING EQUIVALENCE PARTITIONING (EP) • Example: In a banking system, a software to calculate the interest of a saving account is designed based on the balance in the account. If a balance ranges from 0 to 10k has a 1% interest rate, a balance over 10k and up to 50k has a 3% interest rate, balances over 50k has a 5% interest rate. What should be our test cases based on equivalence partitions?

16. BLACK BOX TESTING EQUIVALENCE PARTITIONING (EP) • 1st: define what the input is. • E.g. Balance in an account • 2nd: determine what should be characteristic of each group • E.g. Interest rate • 3rd: determine group of valid input • E.g. Balance for 1%, 3%, and 5% • 4th: determine group of invalid input • E.g. Negative Balance • 5th: Select representative of each partition • E.g. 1, 500, 25k, 100k

17. BLACK BOX TESTING EQUIVALENCE PARTITIONING (EP) Invalid x <= 0k Valid 0k < x <= 10k Valid 10k < x <= 50k Valid 50k < x N/A 1% 3% 5% -1 500 25k 100k

18. BLACK BOX TESTING EQUIVALENCE PARTITIONING (EP) • Using EP, we can ensure that all % interest rate has been calculated. • Error message might also throw out from the invalid case • Comparing to naive tester, EP can achieve more efficiency • Select input for every 10k (e.g. 10k, 20k, 30k, 40k, 50k) • Need to run 5 test cases and still missing negative value

19. BLACK BOX TESTING EQUIVALENCE PARTITIONING (EP) • Other considerations • Non numeric input is also possible input e.g. ‘a’, ‘ • Decimal place is still not yet taken into account e.g. 500.50

20. AGENDA • OVERVIEW • BLACK BOX TESTING • EQUIVALENCE PARTITIONING • BOUNDARY VALUE ANALYSIS • DECISION TABLE TESTING • STATE TRANSITION TESTING • WHITE BOX TESTING

21. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) “A black-box test design technique in which test cases are designed based on value on the edge of equivalent partitioning.” • Boundary Value is an value on the edge of equivalent partitioning or the smallest incremental distance from the edge. • One edge has two boundary values • Boundary value can be both valid and invalid input depending on partition that boundary resided. • Defects are tended to be around the boundary

22. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) • Example: An input for a copying machine can be from 1 to 99 for the number of copies. What should be our test cases based on Boundary Value Analysis?

23. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) • Why defects are tended to be around boundary? • Human error • A developer should write the code as • If (numOfCopies >= 1) …. • However, he/she might unintentionally forget to put equal sign and end up like this • If (numOfCopies > 1) …. • What will happen? • Of course, user will not be able to do one copy

24. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) • 1st: Determine Equivalence Partitioning • E.g. Valid ranges 1 to 99, Invalid < 1, and Invalid > 99 • 2nd: Identify the minimum and maximum value of each valid partition • E.g. Boundary value are 1 and 99 for the valid partitions. • 3rd: Identify the first or last value of each invalid partition • E.g. Boundary value are 0 and 100 for the invalid partitions • 4th: Finalize test cases • 0, 1, 99, 100

25. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) • Open boundary exists in the partition that does not define minimum or maximum. • Interest Rate 5% has open boundary for the maximum • Look at data type of input (e.g. signed integer is 65,535 max)

26. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) • Two boundary values vs Three boundary values • Three boundary values contain one more additional point to test in the valid partition

27. BLACK BOX TESTING BOUNDARY VALUE ANALYSIS (BVA) 0 1 2 2-Value 3-Value If the correct code should be IF (x>=1) print OK

28. AGENDA • OVERVIEW • BLACK BOX TESTING • EQUIVALENCE PARTITIONING • BOUNDARY VALUE ANALYSIS • DECISION TABLE TESTING • STATE TRANSITION TESTING • WHITE BOX TESTING

29. BLACK BOX TESTING DECISION TABLE TESTING “Test design technique based on a table showing the relationship between inputs and outputs” • Also known as cause-effect graphing • Very helpful for the system with combinations of inputs must be considered • Exploring complex business requirements

30. BLACK BOX TESTING DECISION TABLE TESTING • Example: Username and Password checking. • Actions of the program is based on information provided in Username and Password fields. • If Username and Password are filled in, the program will authenticate the login. If Username or Password field is missing, it will ask for the input.

31. BLACK BOX TESTING DECISION TABLE TESTING

32. BLACK BOX TESTING DECISION TABLE TESTING • Example: Consider a function that has two input variables, Customer and Order, and one output variable, Discount. Customers may be of type A or B. Order has a range of 1 to 1,000. The function computes Discount, which is based on the Customer type and the Order. • Customers of type A receive a 0 percent discount for less than 10 items, 5 percent discount for 10 to 99 items, 10 percent discount for 100 or more items. • Customers of type B receive a 5 percent discount for less than 10 items, 15 percent discount for 10 to 99 items, 25 percent discount for 100 or more items.

33. BLACK BOX TESTING DECISION TABLE TESTING

34. AGENDA • OVERVIEW • BLACK BOX TESTING • EQUIVALENCE PARTITIONING • BOUNDARY VALUE ANALYSIS • DECISION TABLE TESTING • STATE TRANSITION TESTING • WHITE BOX TESTING

35. BLACK BOX TESTING STATE TRANSITION TESTING • State Transition Diagram shows states of system with the events that drive the system to change from one to another state. • State Transition Table depicts the change of state in a system based on event driven. • Suitable for event-driven system e.g. internet applications, embedded software • Any system that can get different output from the same input depending on what happen before.

36. BLACK BOX TESTING STATE TRANSITION TESTING • State is the status occupying by software • Transition occurs between each state • Event is the stimulus that drive the system to change from one state to another (or might be on the same state) • Action is the result of system after receiving Event. Event 1 Action 1 S1 S2 Event 2 Action 2

37. BLACK BOX TESTING STATE TRANSITION TESTING • Example: A cassette tape player which has only stop, play, and record buttons. There are two led – green to display play and red to display record.

38. BLACK BOX TESTING STATE TRANSITION TESTING Press STOP All LED Off Stop Press PLAY Green LED On Press REC Red LED On Press STOP Red LED Off Press STOP Green LED Off Press REC Red LED On Play Press PLAY Green LED On Record Press PLAY Green LED On RED LED Off Press REC Green LED Off RED LED On

39. BLACK BOX TESTING STATE TRANSITION TESTING

40. BLACK BOX TESTING STATE TRANSITION TESTING • Chow’s 0–Switch Coverage • Touch on one state transition • Chow’s 1–Switch Coverage • Transition pair • Chow’s 2–Switch Coverage • Transition tripples

41. AGENDA • OVERVIEW • BLACK BOX TESTING • WHITE BOX TESTING

42. WHITE BOX TESTING “A procedure to derive and/or select test cases based on an analysis of the internal structure of a component or system” [2] • Also known as glass box • Look at the programming code • Require the knowledge of how a system does. • Normally executed by developers • Relating to Test Coverage • Require tools to support

43. WHITE BOX TESTING • Statement Testing • Decision Testing • Branch Testing • Condition Testing • Decision/Condition Testing

44. AGENDA • OVERVIEW • BLACK BOX TESTING • WHITE BOX TESTING • TEST COVERAGE • CONTROL FLOW GRAPH • STATEMENT TESTING • DECISION TESTING • BRANCH TESTING • CONDITION TESTING • DECISION/CONDITION TESTING

45. WHITE BOX TESTINGTEST COVERAGE • Test Coverage = Number of items exercised Total number of items • Normally represented in percentage • Code Coverage requires to instrument code • Test result cannot be used as reference • Only measure on what has been written but do not cover missing information.

46. WHITE BOX TESTINGTEST COVERAGE • Test Coverage in black box • EP: Number of partitions exercised • BVA: Percentage of boundary exercised • STT: Percentage of states visited

47. WHITE BOX TESTINGTEST COVERAGE • 1st: Identify the items of measurement • 2nd: Look at the code and count for total items • 3rd: Instrument code • 4th: Run a test suite and get the exercised items • 5th: Calculate coverage from exercised and total items

48. AGENDA • OVERVIEW • BLACK BOX TESTING • WHITE BOX TESTING • TEST COVERAGE • CONTROL FLOW GRAPH • STATEMENT TESTING • DECISION TESTING • BRANCH TESTING • CONDITION TESTING • DECISION/CONDITION TESTING

49. WHITE BOX TESTINGCONTROL FLOW GRAPH • Control Flow is a graphical represent the program structure • Do not provide details information • Node: a single statement or a sequential statements • Branch: A logical link between two nodes indicate that next statement can be executed • Decision Statement

50. WHITE BOX TESTINGCONTROL FLOW GRAPH • IF condition THEN do this ENDIF • IF condition THEN do this ELSE do that ENDIF