Integration: Process, Testing, Issues

1. Integration: Process, Testing, Issues Slides from: - Ch8 – Software Testing, Sommerville, Software Engineering 9 - Unit Testing SWENET Module Chapter 8 Software testing

2. Testing: Implementation • Once one has determined the testing strategy, and the units to tested, and completed the unit test plans, the next concern is how to carry on the tests. • If you are testing a single, simple unit that does not interact with other units , then one can write a program that runs the test cases in the test plan. • However, if you are testing a unit that must interact with other units, then it can be difficult to test it in isolation.

3. Test Implementation Terms • Test Driver • a class or utility program that applies test cases to a component being tested. • Test Stub • a temporary, minimal implementation of a component to increase controllability and observability in testing. • When testing a unit that references another unit, the unit must either be complete (and tested) or stubs must be created that can be used when executing a test case referencing the other unit. • Test Harness • A system of test drivers, stubs and other tools to support test execution

4. Testing sequence • Once the design for the unit is complete, carry out a static test of the unit. This might be a single desk check of the unit or it may involve a more extensive symbolic execution or mathematic analysis. • Complete a test plan for a unit. • If the unit references other units, not yet complete, create stubs for these units. • Create a driver (or set of drivers) for the unit, which includes the following; • construction of test case data (from the test plan) • execution of the unit, using the test case data • provision for the results of the test case execution to be printed or logged as appropriate

5. Integration: Beyond the unit • Classes • Components/Modules • System Chapter 8 Software testing

6. What is integration? • The term "integration" refers to a software development activity in which separate software components are combined into a whole. Integration is done at several levels and stages of the implementation: • Integrating the work of a team working in the same implementation subsystem before releasing the subsystem to system integrators. • Integrating subsystems into a complete system.

7. Big Bang Integration • Often the choice of rushed/inexperienced programmers • Create modules in parallel • Assemble them in one operation • Benefits: • No need for test drivers and stubs! • Issues: • Complexity and cost of fault isolation Chapter 8 Software testing

8. Incremental Integration • Software modules are developed and assembled into progressively larger parts of the system • Complexity increases incrementally • Easier to isolate integration problems • Continues until complete system (at which point system testing commences) • Software builds created – integration tests executed against the build Chapter 8 Software testing

9. Bottom up integration • Need to understand the dependencies between modules • Modules that are most depended on are developed and integrated first • Then the modules that depend on them are integrated next, etc. • Reduces the needs for stubs Chapter 8 Software testing

10. Top down integration • Modules at the top of the dependency tree are developed first, with integration proceeding down the tree • Requires a lot of stubs (dependent modules not yet ready) • Advantage: top level modules are typically higher level functionality (e.g., user interfaces) and are tested early in cycle – gives early feeling for application, allows time for important modifications Chapter 8 Software testing

11. Sandwich Integration • Employ both top down and bottom up • Create stubs for the intervening classes Chapter 8 Software testing

12. Continuous Integration • Small amounts of code added to the baseline on a very frequent basis (perhaps daily) • Newly integrated code may not even be at level of a completed method/class • As long as code is unit-tested (introduces no errors), it can be integrated into the baseline • One of the 12 practices of Extreme Programming Chapter 8 Software testing

13. Integration in practice Chapter 8 Software testing

14. The role of the Integrator

15. Subsystem integration planning • Define the builds • What use cases and / or scenarios • Identify the classes and components that will be integrated • Identify subsystem imports • The subsystems (and versions) needed to make this integration work

16. Use-case testing • System level example • The use-cases developed to identify system interactions can be used as a basis for system testing. • Each use case usually involves several system components so testing the use case forces these interactions to occur. • The sequence diagrams associated with the use case documents the components and interactions that are being tested. Chapter 8 Software testing

17. Collectweather data sequence chart Chapter 8 Software testing

18. A usage scenario for MHC-PMS Kate is a nurse who specializes in mental health care. One of her responsibilities is to visit patients at home to check that their treatment is effective and that they are not suffering from medication sid-effects. On a day for home visits, Kate logs into the MHC-PMS and uses it to print her schedule of home visits for that day, along with summary information about the patients to be visited. She requests that the records for these patients be downloaded to her laptop. She is prompted for her key phrase to encrypt the records on the laptop. One of the patients that she visits is Jim, who is being treated with medication for depression. Jim feels that the medication is helping him but believes that it has the side -effect of keeping him awake at night. Kate looks up Jim’s record and is prompted for her key phrase to decrypt the record. She checks the drug prescribed and queries its side effects. Sleeplessness is a known side effect so she notes the problem in Jim’s record and suggests that he visits the clinic to have his medication changed. He agrees so Kate enters a prompt to call him when she gets back to the clinic to make an appointment with a physician. She ends the consultation and the system re-encrypts Jim’s record. After, finishing her consultations, Kate returns to the clinic and uploads the records of patients visited to the database. The system generates a call list for Kate of those patients who she has to contact for follow-up information and make clinic appointments. Chapter 8 Software testing

19. Features tested by scenario • Authentication by logging on to the system. • Downloading and uploading of specified patient records to a laptop. • Home visit scheduling. • Encryption and decryption of patient records on a mobile device. • Record retrieval and modification. • Links with the drugs database that maintains side-effect information. • The system for call prompting. Chapter 8 Software testing

20. Subsystem integration planning • Define the builds • What use cases and / or scenarios • Identify the classes and components that will be integrated • Identify subsystem imports • The subsystems (and versions) needed to make this integration work

21. Defining the build

22. System integration • Identify subsystems • Define build sets • Things that are typically integrated together • Define a series of builds • To incrementally integrate the work • Evaluate the integration plan

23. Identify subsystems

24. Define build sets

25. Define a series of builds

26. Regular builds important • Build the software and regression test it at regular intervals • Make sure the new code doe not compromise pre-existing functionality • Build frequency depends on phase of project • Weekly builds early on • Daily builds at tail end to deal with last-minute changes/additions (bux fixes!) Chapter 8 Software testing

27. Code freezes • Deadlines set after which no new code is accepted for the day • Can be late (6pm) • Can be early (noon) to make sure that if there are build issues, developers are around to help resolve them • Code then built • If issues, assumed the defect lies in new checked in since last build • Can confirm new baseline or revert to previous baseline Chapter 8 Software testing

28. Beyond unit testing Testing TIPS & techniques Chapter 8 Software testing

29. Component testing • Software components are often composite components that are made up of several interacting objects. • For example, in a weather station system, a reconfiguration component includes objects that deal with each aspect of the reconfiguration. • You access the functionality of these objects through the defined component interface. • Testing composite components should therefore focus on showing that the component interface behaves according to its specification. • You can assume that unit tests on the individual objects within the component have been completed. Chapter 8 Software testing

30. Interface testing Chapter 8 Software testing

31. Interface testing • Objective is to detect faults due to interface errors or invalid assumptions about interfaces. • Interface types • Parameter interfaces Data passed from one method or procedure to another. • Shared memory interfaces Block of memory is shared between procedures or functions. • Procedural interfaces Sub-system encapsulates a set of procedures to be called by other sub-systems. • Message passing interfaces Sub-systems request services from other sub-systems Chapter 8 Software testing

32. Interface errors • Interface misuse • A calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong order. • Interface misunderstanding • A calling component embeds assumptions about the behaviour of the called component which are incorrect. • Timing errors • The called and the calling component operate at different speeds and out-of-date information is accessed. Chapter 8 Software testing

33. Interface testing guidelines • Design tests so that parameters to a called procedure are at the extreme ends of their ranges. • Always test pointer parameters with null pointers. • Design tests which cause the component to fail. • Use stress testing in message passing systems. • In shared memory systems, vary the order in which components are activated. Chapter 8 Software testing

34. Module integration • Once the interface tests are complete, can feel confident about the interface between modules • Then ready to test the functionality more completely with integration tests • Integration tests focus on the additional functionality gained by assembling modules Chapter 8 Software testing

35. Integration testing • Ensure that the components in the implementation model operate properly when combined to execute a use case • Common point of software failure • Usually black-box

36. Integration Test Plan • An Integration Test checks and verifies that the various components in a class cluster communicate properly with each other. • With this plan, the emphasis is not on whether system functionality is correctly implemented, but rather do all the cluster classes “fit” together properly. • A single test plan, for each cluster, which tests the communication between all cluster components is typically sufficient.

37. Incremental Testing • Incremental testing indicates that we test each unit in isolation, and then integrate each unit, one at a time, into the system, testing the overall system as we go. • Classes that are dependent on each other called class clusters, are good candidates for an increment integration. • Candidate class clusters: • Classes in a package • Classes in a class hierarchy. • Classes associated with the interaction diagram for a use case.

38. Increment Test Planning • Determine the classes to be tested in an increment. • Determine the appropriate “class clusters”. • Develop a unit test plan and test driver for each class. • Develop an integration test plan and test driver for each class cluster. • Develop a test script that details the components to be tested and the order in which the plans will be executed.

39. Object-Oriented Testing Issues • The object class is the basic testing unit for system developed with OO techniques. • This is especially appropriate if strong cohesion and loose coupling is applied effectively in the class design. • An incremental testing approach is dictated by • The package/class design decomposition. • The incremental nature of the development process. • Information hiding restricts/constrains testing to using a white-box approach. • Encapsulation motivates testing based on the class interface.

40. Class Testing (1) • Class test cases are created by examining the specification of the class. • This is more difficult if the class is a subclass and inherits data and behavior from a super class. A complicated class hierarchy can be pose significant testing problems. • If you have a state model for the class, test each transition - devise a driver that sets an object of the class in the source state of the transition and generates the transition event. • Class Constraints/Invariants should be incorporated into the class test. • All class methods should be tested, but testing a method in isolation from the rest of the class is usually meaningless • In a given increment, you may not implement all methods; if so, create stubs for such methods.

41. Class Testing (2) • To test a class, you may need instances of other classes. • Interaction diagrams provide useful guidance in constructing class test cases: • They provide more specific knowledge about how objects of one class interact with instances of other classes. • Messages sent to a class object provide a guidance on which test cases are most critical for that class.

42. Method Testing (1) • A public method in a class is typically tested using a black-box approach. • Start from the specification, no need to look at the method body. • Consider each parameter in the method signature, and identify its equivalence classes. • Incorporate pre-conditions and post-conditions in your test of a method. • Test exceptions. • For complex logic, also use white-box testing or static testing.

43. Method Testing (2) • For private methods, either • modify the class (temporarily) so that it can be tested externally: • change the access to public • or incorporate a test driver within the class • or use static test methods, such as program tracing or symbolic execution

44. Testing Tools • There are a number of tools that have been developed to support the testing of a unit or system. • googling “Software Testing Tools” will yield thousands of results. • JUnit testing (http://www.junit.org/index.htm) is a popular tool/technique that can be integrated into the development process for a unit coded in Java. • Test infected: Programmers just love writing tests http://members.pingnet.ch/gamma/junit.htm

45. System testing • System testing during development involves integrating components to create a version of the system and then testing the integrated system. • The focus in system testing is testing the interactions between components. • System testing checks that components are compatible, interact correctly and transfer the right data at the right time across their interfaces. • System testing tests the emergent behaviour of a system. Chapter 8 Software testing

46. System and component testing • During system testing, reusable components that have been separately developed and off-the-shelf systems may be integrated with newly developed components. The complete system is then tested. • Components developed by different team members or sub-teams may be integrated at this stage. System testing is a collective rather than an individual process. • In some companies, system testing may involve a separate testing team with no involvement from designers and programmers. Chapter 8 Software testing

47. Testing policies • Exhaustive system testing is impossible so testing policies which define the required system test coverage may be developed. • Examples of testing policies: • All system functions that are accessed through menus should be tested. • Combinations of functions (e.g. text formatting) that are accessed through the same menu must be tested. • Where user input is provided, all functions must be tested with both correct and incorrect input. Chapter 8 Software testing

48. Release testing • Release testing is the process of testing a particular release of a system that is intended for use outside of the development team. • The primary goal of the release testing process is to convince the supplier of the system that it is good enough for use. • Release testing, therefore, has to show that the system delivers its specified functionality, performance and dependability, and that it does not fail during normal use. • Release testing is usually a black-box testing process where tests are only derived from the system specification. Chapter 8 Software testing

49. Release testing and system testing • Release testing is a form of system testing. • Important differences: • A separate team that has not been involved in the system development, should be responsible for release testing. • System testing by the development team should focus on discovering bugs in the system (defect testing). The objective of release testing is to check that the system meets its requirements and is good enough for external use (validation testing). Chapter 8 Software testing

50. Requirements based testing • Requirements-based testing involves examining each requirement and developing a test or tests for it. • Example: mental health care prescription management system (MHC-PMS) requirements: • If a patient is known to be allergic to any particular medication, then prescription of that medication shall result in a warning message being issued to the system user. • If a prescriber chooses to ignore an allergy warning, they shall provide a reason why this has been ignored. Chapter 8 Software testing