Integration and System Testing

1. Integration and System Testing CSSE 376, Software Quality Assurance Rose-Hulman Institute of Technology March 29, 2007

2. Requirements High-level Design Low-level Design Code Unit Testing Integration Testing System Testing Acceptance Testing Lifecycle Testing Relationships

3. Testing in the Large • Integration testing: putting the pieces together • System testing: function and performance

4. 1. Integration Testing

5. Integration Testing Objectives • Gain confidence in the integrity of overall system design • Ensure proper interaction of components • Run simple system-level tests

6. Integration Strategies • Top-down • Bottom-up • Critical-first • Function-at-a-time • As-delivered • Big bang

7. Top-Down Integration • Start with top-level modules • Use stubs for lower-level modules • As each level is completed, replace stubs with next level of modules

8. Top-Down Issues • Pros: • Always have a top-level system • Stubs can be written from interface specifications • Cons: • May delay performance problems until too late • Stubs can be expensive

9. Bottom-Up Integration • Start with bottom-level modules • Use drivers for upper-level modules • As each level is completed, replace drivers with next level of modules

10. Bottom-Up Issues • Pros: • Primitive functions get most testing • Drivers are usually cheap • Cons: • Only have a complete system at the end

11. Critical-First Integration • Integrate the most critical components first • Add remaining pieces later • Issues: • guarantees that most important components work first • may be difficult to integrate

12. Function-at-a-Time Integration • Integrate all modules needed to perform one function at the same time • For each function, add another group of modules • Issues: • makes for easier test generation • may postpone function interaction too long

13. As-delivered Integration • Integrate all of the modules that can be integrated as they become available • Issues • efficient: just-in-time integration • lazy: may lead to missed schedules

14. Big Bang Integration • Wait until all the components are ready, then put everything together at once • Issues: • avoids cost of scaffolding (stubs or drivers) • does not provide any locality for finding faults

15. Summary of Integration Strategies • Top-down • Bottom-up • Critical-first • Function-at-a-time • As-delivered • Big bang

16. Question:What should be used as integration test cases?

17. 2. System Testing

18. System Testing Objectives • Gain confidence in the integrity of the system as a whole • Ensure compliance with functional requirements • Ensure compliance with performance requirements

19. Testing Functional Requirements • Prepare a test plan from the functional specification of the system • Prepare tests for all areas of functionality • Review test plan and tests • Execute tests • Monitor fault rate

20. 120 100 80 60 40 20 0 Fault Rate Total Faults 1/1/00 1/2/00 1/3/00 1/4/00 1/5/00 1/6/00 1/7/00 1/8/00 1/9/00 1/10/00 1/11/00 1/12/00 1/13/00 1/14/00 1/15/00 1/16/00 1/17/00 1/18/00 1/19/00 1/20/00

21. Testing Performance Requirements • Identify stress points of system • Create or obtain load generators • might use existing system • might buy/make special purpose tools • Run stress tests • Monitor system performance • usually need instrumentation