Software Testing and Quality Assurance

1. Software Testing and Quality Assurance Tutorial 1 - Introduction to Software Testing

2. Lecture Outline • Testing activities. • Introduction to test cases, test oracles and their execution. • Black-box & White-box Testing

3. Unit Testing module to be tested results software engineer test cases

4. Integration Testing • The Big-bang Approach. • Incremental Approach.

5. Top-Down Integration A top module is tested with stubs B F G stubs are replaced one at a time, "depth first" C as new modules are integrated, some subset of tests is re-run D E

6. Bottom-Up Integration A B F G drivers are replaced one at a time, "depth first" C worker modules are grouped into builds and integrated D E cluster

7. System Testing • System Functional Test • Test entire system against the functional requirements. • System Performance Test • Test the non-functional requirements of the system. For example, • Response times, load testing etc. • System Acceptance Test • Set of tests that the software must pass before it is accepted by the client.

8. Trivial Example • You have been asked to write a term paper on ‘Integration Testing in Component Based System’. To do this, you need to find references from a range of library databases. • You logs on to the KFUPM library system and uses the search facility to find relevant papers from IEEE, ACM and Elsevier databases. • One paper of special interest requires authentication and you have to fill an online form to receive the paper. • If you are allowed, the paper will be downloaded and ready for collection. • An email will be send to you once the paper is ready. Scenario-based Testing

9. Student Activity • Identify the possible interactions for the system testing of library system.

10. Trivial Example - System Testing • Test the login mechanism using correct and incorrect login. • Test the search facility using queries against known source to check that the search mechanism is actually documents. • Test system presentation facility to check that information about documents is displayed properly. • Test the mechanism to request permission for downloading. • Test the e-mail response indicating that the download document is available.

11. Regression Testing • Change do not always effect the entire program. • Change in one part of system can effect other part. • After each change • Entire test suite of a system must be run again. Need for an automatic test suite execution.

12. Test Activities • Boils down to selecting and executing test cases. Test case consists of…… • Set of test inputs, of if the program is non-terminating, a sequence of test inputs. • Expected results when the inputs are executed; and • Execution conditions or execution environment in which the inputs are to be executed. These steps generally remain same from unit testing to system testing.

13. Test Case Selection • Coverage criterion; • Equivalence Partitioning • Boundary-Value Analysis • Coverage-Based Testing • Control-flow • Data-flow • Expected behavior of every test input to be generated. (Test Oracles) • Testing environment.

14. Test Oracles • Determines whether or not the program has passed or failed the test case. • A test oracle is • A program • A process • A body of data • In many cases - directly form the requirements. • For example, a test case assessing performance - performance threshold. Difficult to automate or to assess their quality

15. Test Execution • Test inputs on the ‘program-under-test’ • Record the actual behavior. Generally can be automated to an extend !!!!

16. Test Evaluation • Compare the actual behavior with the expected behavior. Generally can be automated to an extend !!!!

17. Test Reporting • Report the outcome of the testing. • Developers • Project Mangers etc. Generally can be automated to an extend !!!!

18. requirements output input events Black-Box Testing

19. Black-box Testing • Test cases are derived from formal specification of the system. • Test case selection can be done without any reference to the program design or code. • Only tests the functionality and features of the program. • Not the internal operation.

20. Black-box Testing • Advantages • Test case selection is done before the implementation of a program. • Help in getting the design and coding correct with respect to the specification.

21. White-Box Testing

22. White-Box Testing • Test cases are derived from the internal design specification or actual code for the program. • Advantages • Tests the internal details of the code; • Checks all paths that a program can execute. • Limitations • Wait until after designing and coding the program under test in order to select test cases.