Software Inspections and Testing

1. Software Inspections and Testing Mohammed El-Affendi

2. Inspection-Based versus Test-Based Quality Strategies: The traditional approach in software development is to rely on tests to discover software defects. This is known as the test-based quality strategy and has proved to be not very effective. In one classical example, the test-based strategy: a 22 KLOC product produced 186 defects during test, 42 of them critical. Only one critical defect was found in the first year. In another example, testing took six years. The last 10 critical defects were found after 288 weeks

3. Inspection-Based Quality Strategy: The current trend in software development is to put quality products into test not to test poor quality products. Inspections are used to remove as much as possible of the defects before the test phase. This proved to reduce the test time from 50% to only 10%. In some real cases this strategy reduced test time form 22% to 2.7% only.

4. Some Terminology: A defect is any requirements, design, or implementation element that if not changed could cause improper design, implementation, test, use or maintenance of the product. A major defect is any problem that, when fixed would change the executable program. Fixing a minor defect will not change the executable.

5. Some Terminology: Defects/KLOC is a measure of quality in the test phase. Defect ratios give insight into the quality of reviews. Development time ratios: e.g. detailed design versus coding time. A/FR (appraisal/Failure ratios): e.g Time spent in reviews versus time spent in test.

6. What are Inspections: An inspection is a form of review in which one or more engineers review a product produced by another engineer to find defects and problems. This may be done for all types of products during the development process: use cases, SRS, SDS, code, documentation …etc.

7. How Inspections are Performed? Briefing phase. Review phase. Meeting Phase. Repair phase.

8. A Suggested Inspection Process: Producer contacts moderator (quality manager) telling him that the product is ready for inspection. Moderator reviews product to ensure that it is ready for inspection. Moderator invites the inspection team for a briefing. Moderator describes the inspection process. Producer familiarizes team with product. Team members decide inspection approach and method (viewpoints, product concentration …etc.).

9. Inspection Process: Moderator fixes time and date for inspection meeting. Producers separately review the product. Conduct inspection meeting: moderator opens meeting and if some reviewers are not ready reschedule meeting. Conduct a walkthrough: moderator walks through the product section by section, recording every defect on the INSPECTION Form, notes every engineer who discovered a major defect. Estimate the remaining defects.

10. Estimating the Remaining Defects: A Typical Inspection Form.doc Count unique engineer defects. Identify the engineer with the most unique defects, W. Check each major defect found by W in column A. In Column B check all defects found by other engineers. Count C, the common defects between A and B. Number found is A+B-C Total estimated AB/C Remaining defects (AB/C) – (A+B-C).

11. Inspection Rates: Inspection rates are measured either in LOCs per hour or pages per hour. 2 pages/ hour  req. 5 pages/hour  HLD 200 LOC/hour.

12. Development Ratios: Spend 50% of design time in design inspection. You may try to collect your own statistics to deduce a guideline for how much time to spend in inspection based on development time.

13. Inspection Yields: Measures the per cent of defects found during inspection. This helps the development team to evaluate their review processes and compare them.

14. Testing Unit testing. Build and integration testing. System testing. Alpha testing. Beta Testing.

15. Build and Integration Strategies: Bing-Bang. One-at-time (bottom-up). Cluster strategy. Flat-system strategy (top-down).

16. System Testing Strategies: Does the system properly perform the functions it is supposed to perform? Does the system meet its quality goals? Will the system operate properly under normal conditions? Will the system operate properly under adverse conditions?

17. Test Process: Build: -- Make sure that all needed parts are available. --Build the product and make it ready for integration testing. -- Record all defects in defect LOG.

18. Integration Test: Run the product to make sure that it can be started and run. Make sure that all interfaces are properly matched and functional. Remember that this is not a system test and should be simple.

19. System Test: Test the system for normal and stress conditions. Test the product for installation, conversion and recovery. Record all test activities in the test log form.

20. Test Log Form: The date the test was run. The name of the person running the test. The tests run (name or number). The product and configuration being tested. The time each test was started. The time each test was completed. The number of defects found (LOGD references). Test results.

21. Preparing the Test Cases and Material: Preparing test cases may not be a trivial exercise. It depends greatly on the type of application. The test engineers should start early collecting the required data sets and the expected results. The data should cover boundary conditions and all the expected domains. You may need to read full chapters on how test cases are prepared (see for example Pressman).