Integration and system Testing Chapter 12 & 13

1. Integration and system Testing Chapter 12 & 13 H.Reza Csci565 Spring 2009

2. Objectives • Integration Testing • Simple ATM (SATM) • discuss integration testing strategies • Top-down • Bottom-up • Decomposition Based Integration Testing (DBIT) • Call Graph Based Integration Testing (CGBIT) • Path-Based Integration Testing (PBIT) H.Reza

3. A Software Testing Strategy: Spiral Model System Testing validation Testing Integration Testing Unit Testing H.Reza Code Design Requirements System eng.

4. Integration Testing:1 • If all units/components work individually, do they work as a whole when we put them together? • Not necessarily • The problem is “putting them together” or interfacing them H.Reza

5. Problems with interfacing • Integration faults often traceable to incomplete or misunderstood interface specifications • mismatched assumptions about other components • Individually acceptable imprecision may be magnified to unacceptable levels • Global data structures can present problems • Inconsistent interpretation of parameters or values • Mixed units (meters/yards) in Martian Lander • Violations of value domains, capacity, or size limits • … H.Reza

6. Integration testing • Tests complete systems or subsystems composed of integrated components • Integration testing should be black-box testing when tests derived from the specification • Main difficulty is localising errors • Incremental integration testing reduces this problem H.Reza

7. Approaches to integration testing • Two major approaches • Incremental approaches • The decomposition-based techniques or tree • Stubs/Drivers • Call Graph-based techniques • No stubs or drivers • Non-incremental approaches H.Reza

8. Incremental integration testing H.Reza

9. Incremental approaches: Top-down • Top-down testing • Start with high-level system • integrate from the top-down replacing individual components by stubs where appropriate • Depth-first • Breadth-first • No-best order • Critical sections • Early skeletal version using I/O modules H.Reza

10. Stubs • Stubs • Special module to simulate some functionality • Its production is nontrivial task because the code may simulate a very complicated tasks • E.g. • Writing a stub performing a database table search routine • Creating multiple version of same stub for various reasons H.Reza

11. Top-down testing H.Reza

12. Top-down: Complications • The most common complication occurs when processing at low level hierarchy demands adequate testing of upper level • To overcome: • Either, delay many tests until stubs are replaced with actual modules (BAD) • Or, develop stubs that perform limited functions that simulate the actual module (GOOD) • Or, Integrate the software using bottom up approach • Confusion about overlapping with design H.Reza

13. Incremental testing: Bottom up • Bottom-up testing • Integrate individual components in levels until the complete system is created H.Reza

14. Bottom-up approach • Starts with construction and testing with atomic modules • No need for stub • Low level components are combined into cluster (or builds) to perform a specific sub-function • A driver (a control program for testing) is written • Contain hardcoded test input, calls the module being tested, and display the results • Cluster is tested • Drivers are removed and clusters are combined moving upward in the program structure H.Reza

15. Bottom-up testing M H.Reza

16. A C D B I H H.Reza E F G J L K

17. second state in the top-down A Stub C Stub D B Stub E H.Reza Stub F

18. Intermediate state in the top-down A Stub C D B I Stub E Stub H H.Reza F J

19. Top-down • Possible Sequences of modules • A, B, C, D, E, F, G, H, I, J, K, L • A, B, E, F, J, C, G, K, D, H, L, I • A, D, H, I, K, L, C, G, B, F, J, E • A, B, F, J, D, I, E, C, G,K, H, L • If parallel testing allowed, then other alternatives are possible • After A has been tested, one programmer could take A and test the combination A-B, • Another programmer could test A-C H.Reza

20. Guideline for Integration testing • Integrate the components that implement the most frequently used functionality • Perform regression testing for existing features • Perform progression testing for new features H.Reza

21. Non-incremental • Big-bang • Imposes no order (GOOD) • Test all the units (Modules) at once • Very easy, but difficult to localize the source of errors H.Reza

22. Integration Test Document • Overall plan for integration of the system under construction must be documented in a Test Specification • The test plan should describe the overall strategy for integration • Example of phases • User interaction (menu, button, forms, display presentation) • Data Manipulation and analysis • Display processing and generation (RT-2D, RT-3D, etc) • Database Mgt • Logic?? H.Reza

23. Integration: Criteria • The following criteria • Interface integrity • (internal and external interfaces are tested as each module (or cluster) is incorporated • Functional validity • Testes to uncover functional error • Information validity • Tests to uncover error related to local/ global data • Performance (Quality) • Tests designed to verify performance bounds during software design H.Reza

24. Top-down Vs. Bottom-up • Architectural validation • Top-down integration testing is better at discovering errors in the system architecture • System demonstration • Top-down integration testing allows a limited demonstration at an early stage in the development • Test implementation • Often easier with bottom-up integration testing H.Reza

25. The waterfall life cycle Requirement specifications System testing Integration testing Preliminary design H.Reza Unit testing details design coding

26. Integration testing: Software Architecture • Integration testing • How software architecture can be used to conduct tests to uncover errors related to the interface? H.Reza

27. Fig. 12.2: Primary Design (or informal Software Architecture) of the ATM using Tree-based decomposition Requirement specifications Terminal I/O Mange Session Conduct Transactions H.Reza Select Transaction Card Entry PIN Entry

28. More on Primary design • How do perform Integration testing for non-tree based functional decomposition? • E.g • integration testing for OO • Integration testing for Client/server systems • Integration testing for Layered systems • …. H.Reza

29. Simple ATM (SATM) • An ATM simple • Provides 15 screens for interactions • includes 3 function buttons • Modeled in structural analysis • Data Model (ERD) • Functional Model (DFD) • Behavioral model (STD) H.Reza

30. Figure 12.7 H.Reza

31. Figure 12.8 H.Reza

32. Figure 12.9 H.Reza

33. Figure 12.10 H.Reza

34. Figure 12.11 H.Reza

35. Figure 12.12 H.Reza

36. Figure 12.13 H.Reza

37. Decomposition based strategies • Decomposition based • Top/down • Bottom up • Sandwich • Big bang H.Reza

38. Figure 12.14 H.Reza

39. Figure 13.1 H.Reza

40. Decomposition based testing:1 • Discussion revolves around the tree-based decomposition and the order by which units are tested and combined • Top-to-bottom • Bottom-to-top • Sandwich • Big bang • The focus is on the structural compatibility among interfaces H.Reza

41. Test Sessions • A test session refers to one set of tests for a specific configuration of actual code and stubs • The number of integration test sessions using a decomposition tree can be computed • Sessions=nodes – leaves + edges H.Reza

42. decomposition based testing: 2 • For SATM system • 42 integration testing session (i.e., 42 separate sets of integration test cases) • top/down • (Nodes-1) stubs are needed • 32 stub in SATM • bottom/up • (Nodes-leaves) of drivers are needed • 10 drivers in SATM H.Reza

43. Decomposition based strategies: Pros and con • Intuitively clear and understandable • In case of faults, most recently added units are suspected ones • Can be tracked against decomposition tree • Suggests breadth-first or depth-first traversals • Units are merged using the decomposition tree • Correct behavior follows from individually correct units and interfaces • Stubs/Drives are major development Overhead H.Reza

44. Call graph based integration testing • Call graph • A directed graph • Nodes corresponds to unit • Edges corresponds to the call • E.g. • AB (i.e., A is calling B) • Attempts to overcome the decomposition problem (structural) • Moves toward behavioral testing H.Reza

45. Call graph (CG): approaches • Two main approaches based on Call Graph • Pair-wise integration • Neighborhood integration H.Reza

46. Figure 13.2: SATM Call Graph H.Reza

47. Table 2: AM H.Reza

48. Pair-wise integration • The main idea is to eliminate the overhead (i.e., stub/drive) • Uses actual code by restricting a session testing to a pair of units in the Call Graph • One integration test for each edge in CG • 40 edges means 40 integration tests for the SATM H.Reza

49. Pair-wise integration H.Reza -Uses actual code -one integration test session for each edge -40 edges for SATM

50. Neighborhood Integration • The neighborhood of a node refers to the nodes that are one edge away from the given nodes • SATM Neighborhoods • Number of neighborhoods can be computed • Neighborhoods = nodes – sink nodes • Or the number of interior-nodes + X ( x=1 if there exists leaf nodes connected directly to the root node otherwise X= 0) • Results a drastic reduction in the number of integration test session • In case of SATM (11 vs. 40) H.Reza