Integration and Higher Level Testing

1. Integration and Higher Level Testing Software Testing and Verification Lecture 11 Prepared by Stephen M. Thebaut, Ph.D. University of Florida

2. Context • Higher-level testing begins with the integration of (already unit-tested) modules to form higher-level program entities (e.g., components). • The primary objective of integration testingis to discover interface and blatant higher-level design errors associated with the elements being integrated. (Somewhat in keeping with the spirit of “smoke testing”.)

3. Context (cont’d) • Once the elements have been successfully integrated (i.e., once they are able to function together), the functional and non-functional characteristics of the higher-level element can be tested thoroughly (via component, product, or system testing).

4. Integration Testing • Integration testing is carried out when integrating (i.e., combining): • Units or modules to form a component • Components to form a product • Products to form a system • The strategy employed can significantly affect the time and effort required to yield a working, higher-level element.

5. Integration Testing (cont’d) • Note that ‘‘integration testing’’ is sometimes defined as thelevel of testing between unit and system.We use a more general model of the testing process.

6. Levels of Testing A More General View SystemTest A Popular View Integration Test System Test ProductTest IntegrationTest Integration Test Unit Test Component Test IntegrationTest UnitTest

7. Integration Testing Strategies • The first (and usually the easiest...) issue to address is the choice between instantaneous and incrementalintegration testing. • The former is sometimes referred to as the big bang approach. (Guess why!) Locating subtle errors can be very difficult after the “bang”. • “Integration is a process, not an event.”

8. Integration Testing Strategies (cont’d) • Incremental integration testing results in some additional overhead, but can significantly reduce error localization and correction time. (What is the overhead?) • The optimum incremental approach is inherently dependent on the individual project and the pros and cons of the various alternatives.

9. As an aside… Here’s a slide from an on-line lecture† that I came across recently… Principles of Integration • Proper policy and plans • Advocacy • Manpower training • Realistic tasks • Coordination with other sectors • Proper support • Access to drugs † “Integration of Substance Abuse Disorders in National Rural Health Mission (NRHM),” Centre for Community Medicine, All India Institute of Medical Sciences, New Delhi

10. Incremental Strategies • Suppose we are integrating a group of modules to form a component,the control structure of which will form a ‘‘calling hierarchy’’ as shown.

11. Incremental Strategies (cont’d) • In what order should the modules be integrated? • From the top (“root”) module toward the bottom? • From bottom (“leaf”) modules toward the top? • By function? • Critical or high-risk modules first? • By availability?

12. Incremental Strategies (cont’d) • How many should be combined at a time? • What scaffolding(i.e., drivers and stubs to exercise the modules and oracles to interpret/inspect test results) will be required? • Is scaffolding ever required outside the context of integration testing?

13. Top-Down Strategy • Start with the‘‘root’’ and one or more called modules. • Test this group using stubs to take the place of missing called modules, and one driver (if necessary) to call the root module. • Add one or more othercalled modules, replacing and providing new stubs as necessary. (cont’d)

14. Top-Down Strategy (cont’d) • Continue the process until all elements have been integrated and tested.

15. Top-Down Strategy (cont’d) A B C C D D E F G G H H I I J K L

16. Top-Down Strategy (cont’d) driver A B stub stub stub stub

17. Top-Down Strategy (cont’d) driver A B C C stub stub stub stub stub

18. Top-Down Strategy (cont’d) driver A B C C D D stub stub stub stub stub stub

19. Top-Down Strategy (cont’d) driver A B C C D D E stub stub stub stub stub

20. Top-Down Strategy (cont’d) driver A B C C D D E F stub stub stub stub stub

21. Top-Down Strategy (cont’d) driver A B C C D D E F G G stub stub stub stub

22. Top-Down Strategy (cont’d) driver A B C C D D E F G G H H stub stub stub stub

23. Top-Down Strategy (cont’d) driver A B C C D D E F G G H H I I stub stub stub

24. Top-Down Strategy (cont’d) driver A B C C D D E F G G H H I I J stub stub

25. Top-Down Strategy (cont’d) driver A B C C D D E F G G H H I I J K stub

26. Top-Down Strategy (cont’d) driver A B C C D D E F G G H H I I J K L

27. Top-Down Strategy (cont’d) • Potential Advantages: • Allows early verification of high-level behavior. • One driver (at most) is required. • Modules can be added one at a time with each step if desired. • Supports both ‘‘breadth first’’ and ‘‘depth first’’ approaches.

28. Top-Down Strategy (cont’d) • Potential Disadvantages: • Delays verification of low-level behavior. • Stubs are required for missing elements. • Test case inputs may be difficult to formulate. • Test case outputs may be difficult to interpret. (Oracles may be needed to interpret/inspect test results.)

29. Bottom-Up Strategy • Start at the bottom of the hierarchy with two or more sibling leaf modules, or an only-child leaf module with its parent. • Test this group using a driver. (No stubs are required.) • Add one or more additional siblings, replacing drivers with modules only when all modules they call have been integrated. (cont’d)

30. Bottom-Up Strategy (cont’d) • Continue the process until all elements of the subtree have been integrated and tested. • Repeat the steps above for the remaining subtrees in the hierarchy (or handle in parallel). • Incrementally combine the sub-trees and then add the root module.

31. Bottom-Up Strategy (cont’d) A B C C D D E F G G H H I I J K L

32. Bottom-Up Strategy (cont’d) driver F J

33. Bottom-Up Strategy (cont’d) driver E F J

34. Bottom-Up Strategy (cont’d) driver B E F J

35. Bottom-Up Strategy (cont’d) driver B E F driver J K L

36. Bottom-Up Strategy (cont’d) driver B driver E F H H J K L

37. Bottom-Up Strategy (cont’d) driver B driver E F H H I I J K L

38. Bottom-Up Strategy (cont’d) driver driver B D D E F H H I I J K L

39. Bottom-Up Strategy (cont’d) driver driver B D D driver E F G G H H I I J K L

40. Bottom-Up Strategy (cont’d) driver driver driver B C C D D E F G G H H I I J K L

41. Bottom-Up Strategy (cont’d) driver driver B C C D D E F G G H H I I J K L

42. Bottom-Up Strategy (cont’d) driver driver B C C D D E F G G H H I I J K K L

43. Bottom-Up Strategy (cont’d) driver B C C D D E F G G H H I I J K L

44. Bottom-Up Strategy (cont’d) driver B C C D D E F G G H H I I J K L

45. Bottom-Up Strategy (cont’d) driver A B C C D D E F G G H H I I J K L

46. Bottom-Up Strategy (cont’d) • Potential Advantages: • Allows early verification of low-level behavior. • No stubs are required. • Easier to formulate input data for some subtrees. • Easier to interpret output data for others.

47. Bottom-Up Strategy (cont’d) • Potential Disadvantages: • Delays verification of high-level behavior. • Drivers are required for missing elements. • As subtrees are combined, a large number of elements may be integrated at one time.

48. Hybrid Incremental Integration Approaches • Risk Driven Start by integrating the most critical or complex modules together with modules they call or are called by. • Schedule Driven To the extent possible, integrate modules as they become available. (cont’d)

49. Hybrid Incremental Integration Approaches (cont’d) • Function or Thread Driven Integrate the modules associated with a key function (thread); continue the process by selecting another function, etc.

50. How about Object-Oriented Systems? • Just as a calling hierarchy allows design of an integration strategy for imperative software, use/include relations serve this purpose for object-oriented software. • Since there may be no single “root” class, testing usually proceeds cluster by cluster in a “bottom-up”fashion, starting with “leaf” classes that depend on no others. • We will come back to this in Lecture 12.