CPIS 357 Software Quality & Testing

1. CPIS 357 Software Quality & Testing I.Rehab Bahaaddin Ashary Faculty of Computing and Information Technology Information Systems Department Fall 2010

2. Chapter 1 :Basic Concepts and Preliminaries • Lecture Objectives • Central Issue in Testing • Testing Activities • Test Levels • Sources of Information for Test Case Selection • White- Box and Black – Box Testing • Test Planning and Design • Monitoring and Measuring Test Execution • Test Tools and Automation • Test Team Organization and Management

3. Central Issue in Testing Figure 1: A subset of the input domain exercising a subset of the program behavior • Divide the input domain D into D1 and D2 • Select a subset D1 of D to test program P • It is possible that D1 exercise only a part P1 of P • Selection of the subset of the input domain must be done in a systematic and careful manner so that the deduction is as accurate and complete as possible. For example. The idea of coverage is considered while selecting the test case.

4. Testing Activities Figure 2: Different activities in process testing • Identify the objective to be tested. (Purpose designing one or more test case) • Select inputs.(Requirements specification, source code) • Compute the expected outcome.(High Level understanding both objective and specification) • Set up the execution environment of the program. (Local/Remote) • Execute the program.(Different location at different time) • Analyze the test results.(Pass/Fail/Inconclusive)

5. Test Levels • Unit testing • Individual program units, such as procedure, methods, Function and Classes in isolation • Integration testing • Modules are assembled to construct larger subsystem and tested by software developer and test engineer • System testing • Includes wide spectrum of testing such as functionality, Security, Stability, Performance, reliability and load • Acceptance testing • Customer’s expectations from the system • Two types of acceptance testing • UAT: System satisfies the contractual acceptance criteria • BAT: System will eventually • Measure the quality of the product rather than searching for the defect. Figure 3: Development and testing phases in the V model

6. Test Levels Figure 4: Regression testing at different software testing levels • New test cases are not designed • Test are selected, prioritized and executed • To ensure that nothing is broken in the new version of the software

7. Source of Information for Test Case Selection • Requirement and Functional Specifications • Capturing user needs. • Informal or formal manner • Examples: plaintext, flowchart, figures, equations, use case, entity relationship diagram or class diagram • Source Code • Internal behavior of the system. • By software engineer (for the design details) and programmer add more details Example: Sort Function • Input and Output Domain • Example: factorial (0) =1; factorial (1) =1 ; • factorial (n) =n* factorial (n-1) • Operational Profile • Is a quantitative characterization of how the system will be used and guide test engineers in selecting test case input. • Fault Model • Error Guessing • Fault Seeding (Fault Injection) • Mutation Analysis (Fault Simulation)

8. White-box and Black-box Testing

9. Test Planning and Design • The purpose is to get ready and organized for test execution. • A test plan provides a: • Framework • A set of ideas, facts or circumstances within which the tests • will be conducted. • Scope • The test activities and the outline of the domain. • Details of resource needed • Effort required • Schedule of activities • Budget • During the test design phase the system requirements are critically studied, system features to be tested and the objective and the detailed behavior of test cases are defined. • Test objectives are identified from different sources. • Each test case is designed as a combination of modular test components called test steps. • Test steps are combined together to create more complex tests.

10. Monitoring and Measuring Test Execution • Metrics for monitoring test execution (concern on test). • Metrics for monitoring defects (concern on the result from the test). • Needed to be tracked and analyzed in a periodic basic (daily or weekly). • Metrics is meaningful if enable management to make decision making which reduce delay, improve quality and lower the cost. • The evaluation done through the following: • Evaluate the effectiveness, Evaluate productivity, Evaluate the quality, • Evaluate the product. • Test case effectiveness metrics • Measure the “defect revealing ability” of the test suite • Use the metric to improve the test design process • Test-effort effectiveness metrics • Number of defects found by the customers that were not found by the test engineers • After product Developed.

11. Test Tools and Automation

12. Test Team Organization and Management Figure 5: Structure of test groups • Unit Testing  Programmer • System Integration  Test engineer • System test group  must be separated from development group and they put the plan for the UAT, BAT • Acceptance  Client special group (SW quality assurance engineer, business associate, customer support engineer) • Hiring and retaining test engineers is a challenging task. • Interview is the primary mechanism for evaluating applicants.