Chapter 5 Retrospective on Functional Testing 322 235 Software Testing

1. Chapter 5 Retrospective on Functional Testing322 235 Software Testing By WararatSongpan(Rungworawut),PH.D. Department of Computer Science, Faculty of Science, KhonKaen University 322 235 การทดสอบซอฟต์แวร์

2. Functional testing or Black Box • We have seen 3 main types of functional testing, BVT, EC, and DT. • Boundary Value Testing (BVT): • Boundary Value Analysis (BVA) – 5extreme values – single assumption • Robustness Testing (RT) – 7extreme values – single assumption • Worst-case Testing (WT) – 5extreme values – Multiple assumption • Robust Worst-case Testing (RWT)- 7extreme values – Multiple assumption • Equivalence Class Testing (EC): • Weak Normal (WN) - valid EC(try no repeat) ->single assumption • Strong Normal (SN) - valid EC (Every)-> multiple assumption • Weak Robust(WR) - valid & Invalid EC ->single assumption • Strong Robust (SR) -valid & Invalid EC-> multiple assumption • Decision Table-based Testing (DT): • It finds out of some test cases that do not make sense (Impossible test cases) 322 235 การทดสอบซอฟต์แวร์

3. Test effort (1) • Look at questions related to testing effort, testing efficiency, and testing effectiveness between 3 techniques. Number of Test Cases high low Sophistication Boundary value Decision table Equivalence class 322 235 การทดสอบซอฟต์แวร์

4. Test effort (1) Effort to Identify Test Cases high low Sophistication Boundary value Equivalence class Decision table 322 235 การทดสอบซอฟต์แวร์

5. Test Identification Effort VS Test Execution Effort (BVT) Effort to Identify Test Cases Number of Test Cases high high low Sophistication low Boundary value Equivalence class Decision table Sophistication BVA: They are mechanical on generating test cases and therefore are easy to automate. Boundary value Decision table Equivalence class 322 235 การทดสอบซอฟต์แวร์

6. Test Identification Effort VS Test Execution Effort (EC) Effort to Identify Test Cases Number of Test Cases high high low Sophistication low Boundary value Equivalence class Decision table Sophistication EC: The equivalence class techniques take into account data dependencies and the program logic however, more thought and care is required to define the equivalence relation, partition the domain, and identify the test cases. Boundary value Decision table Equivalence class 322 235 การทดสอบซอฟต์แวร์

7. Test Identification Effort VS Test Execution Effort (EC) Effort to Identify Test Cases Number of Test Cases high high low Sophistication low Boundary value Equivalence class Decision table Sophistication DT: The decision table technique is the most sophisticated, because it requires that we consider both data and logical dependencies. Boundary value Decision table Equivalence class 322 235 การทดสอบซอฟต์แวร์

8. Functional Test Cases: BVA (15 TCs)

9. Functional Test Cases: WN.EC (11 TCs)

10. Functional Test Cases: DT (8 TCs.)

11. The number of test cases 322 235 การทดสอบซอฟต์แวร์

12. The number of test cases 322 235 การทดสอบซอฟต์แวร์

13. The number of test cases 322 235 การทดสอบซอฟต์แวร์

14. Test cases Trend Line: All Program Test case per method-all problem 322 235 การทดสอบซอฟต์แวร์

15. Testing Efficiency (1) • The data above, reveal the fundamental limitation of functional testing: the twin possibilities of • gaps of untested functionality • redundant tests • For example: • The decision table technique for the NextDateprogram generated 22 test cases (fairly complete) • The worst case boundary analysis generated 125 cases. These are fairly redundant (check January 1 for five different years, only a few February cases but none on February 28, and February 29, and no major testing for leap years). • The strong equivalence class test cases generated 36 test cases 11 of which are impossible. • There are gaps and redundancy in functional test cases, and these are reduced by using more sophisticated techniques (i.e. Decision Tables).

16. Testing Efficiency (2) • The question is how can we quantify what we mean by the term testing efficiency. • The intuitive notion of an efficient testing technique is that it produces a set of test cases that are “just right” that is, with no gaps and no redundancy. • We can even develop various ratios of useful test cases (i.e. not redundant, and with no gaps) with respect to the total number of test cases generated by method A and method B. • One way to identify redundancy by annotating test cases with a brief purpose statement. Test cases with the same purpose provide a sense of redundancy measure. • Detecting gaps is harder and this can be done only by comparing two different methods, even though there are no guarantees for complete detection of gaps. • Overall, the structural methods (we will see later in the course), support interesting and useful metrics and these can provide a better quantification of testing efficiency.

17. Guidelines • Kinds of faults may reveal some pointers as to which testing method to use. • If we do not know the kinds of faults that are likely to occur in the program then the attributes most helpful in choosing functional testing methods are: • Whether the variables represent physical or logical quantities • Whether or not there are dependencies among variables • Whether single or multiple faults are assumed • Whether exception handling is prominent

18. Guidelines for Functional Testing Technique Selection (1) • The following selection guidelines can be considered: • If the variables refer to physical quantities and/or are independent domain testing and equivalence testing can be considered. • If the variables are dependent, decision table testing can be considered • If the single-fault assumption is plausible to assume, boundary value analysis and robustness testing can be considered • If the multiple-fault assumption is plausible to assume, worst case testing, robust worst case testing, and decision table testing can be considered • If the program contains significant exception handling, robustness testing and decision table testing can be considered • If the variables refer to logical quantities, equivalence class testing and decision table testing can be considered

19. Guidelines for Functional Testing Technique Selection (2)

20. Test Summary Report 322 235 การทดสอบซอฟต์แวร์

21. Test Summary Report 322 235 การทดสอบซอฟต์แวร์

22. Test Summary Report 322 235 การทดสอบซอฟต์แวร์

23. Bug life Cycle BTT= Bug Tracking Tool 322 235 การทดสอบซอฟต์แวร์

24. Basic of Testing effectiveness Metrics • Test Coverage • Productivity • Defect Detection Effectiveness • Defect Acceptance Ratio • Estimation Accuracy 322 235 การทดสอบซอฟต์แวร์

25. Test Coverage 322 235 การทดสอบซอฟต์แวร์

26. Productivity 322 235 การทดสอบซอฟต์แวร์

27. Defect Detection Effectiveness 322 235 การทดสอบซอฟต์แวร์

28. Defect Acceptance Ratio 322 235 การทดสอบซอฟต์แวร์

29. Estimation Accuracy 322 235 การทดสอบซอฟต์แวร์