The Software Development Life Cycle: An Overview

1. Presented by Maxwell Drew and Dan Kaiser Southwest State University Computer Science Program The Software DevelopmentLife Cycle: An Overview

2. Last Time • Introduction to the Principles of Object Technology • Object Oriented Design • Object Technology and MSF • Object Technology and RUP

3. Session 6: Testing • Introduction to the Principles of Testing • The Testing Process • Schwan’s Development Standards • MSF Testing • RUP Implementation and Testing

4. Purpose of Testing • In most engineering disciplines the purpose of testing is to verify that the product meets its specifications. • Most physical products have only a finite number of failure modes. • If your toaster doesn’t get hot then either: • the cord is defective • the heating element is broken • the connections are bad

5. Purpose of Testing • In software engineering, the purpose of testing is to find errors. • When a piece of software does not work there can be an almost unlimited number of causes.

6. Terminology • Two views of error. • Failure - The manifestation of the error. The software does something that is contrary to its required behavior. • Fault - The cause of the error. A human mistake made during some software activity that produces a failure.

7. Formal Definition of Testing • According to the IEEE The purpose of software testing is to find failures so that once the software has failed under test, faults responsible for the failures can be found and fixed.

8. Table 8.1. IBM orthogonal defect classification. Fault type Meaning Function Fault that affects capability, end-user interfaces, product interfaces, interface with hardware architecture, or global data structure Interface Fault in interacting with other components or drivers via calls, macros, control blocks or parameter lists Checking Fault in program logic that fails to validate data and values properly before they are used Assignment Fault in data structure or code block initialization. Timing/serialization Fault that involves timing of shared and real-time resources Build/package/merge Fault that occurs because of problems in repositories, management changes, or version control Documentation Fault that affects publications and maintenance notes Algorithm Fault involving efficiency or correctness of algorithm or data structure but not design

9. Hewlett Packard Fault Classification Scheme

11. Overall Goal • Our overall goal is still to validate and verify the software.

12. Validation "Are we building the right product?" • The software should do what the user really requires

13. Verification "Are we building the product right?” • The software should conform to its specification

14. The V & V process • Is a whole life-cycle process - V & V must be applied at each stage in the software process. • Has two principal objectives • The discovery of defects in a system • The assessment of whether or not the system is usable in an operational situation.

15. Dynamic vs Static Verification • Dynamic verification • Concerned with exercising and observing product behavior (testing) • Static verification • Concerned with analysis of the static system representation to discover problems

16. Static and Dynamic V&V

17. Types of Testing • Statistical testing • tests designed to reflect the frequency of user inputs. Used for reliability estimation. • Defect testing • Tests designed to discover system defects. • A successful defect test is one which reveals the presence of defects in a system.

18. Testing and Debugging • Defect testing and debugging are distinct processes • Defect testing is concerned with confirming the presence of errors • Debugging is concerned with locating and repairing these errors • Debugging involves formulating a hypothesis about program behavior then testing these hypotheses to find the system error

19. The Debugging Process

20. Testing Stages • Unit testing • testing of individual components • Module testing • testing of collections of dependent components • Sub-system testing • testing collections of modules integrated into sub-systems

21. Testing Stages • System testing • testing the complete system prior to delivery • Acceptance testing • testing by users to check that the system satisfies requirements. Sometimes called alpha testing.

22. The Testing Process

23. Object-oriented System Testing • Less closely coupled systems. Objects are not necessarily integrated into sub-systems • Cluster testing. Test a group of cooperating objects • Thread testing. Test a processing thread as it weaves from object to object.

24. Test Planning and Scheduling • Describe major phases of the testing process • Describe trace-ability of tests to requirements • Estimate overall schedule and resource allocation • Describe relationship with other project plans • Describe recording method for test results

25. The Test Plan • The testing process • Requirements trace-ability • Tested items • Testing schedule • Test recording procedures • Hardware and software requirements • Constraints

26. The V-model of Development

27. Testing Strategies • Testing strategies are ways of approaching the testing process • Different strategies may be applied at different stages of the testing process • Strategies covered • Top-down testing • Bottom-up testing • Thread testing • Stress testing • Back-to-back testing

28. Top-down testing

29. Top-down testing • Start with the high-levels of a system and work your way downwards • Testing strategy which is used in conjunction with top-down development • Finds architectural errors • May need system infrastructure before any testing is possible • May be difficult to develop program stubs

30. Bottom-up testing

31. Bottom-up testing • Necessary for critical infrastructure components • Start with the lower levels of the system and work upward • Needs test drivers to be implemented • Does not find major design problems until late in the process • Appropriate for object-oriented systems

32. Thread testing • Suitable for real-time and object-oriented systems • Based on testing an operation which involves a sequence of processing steps which thread their way through the system • Start with single event threads then go on to multiple event threads • Complete thread testing is impossible because of the large number of event combinations

33. Process interactions

34. Thread testing

35. Multiple-thread testing

36. Stress testing • Exercises the system beyond its maximum design load. Stressing the system often causes defects to come to light • Stressing the system test failure behavior. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or data • Particularly relevant to distributed systems which can exhibit severe degradation as a network becomes overloaded

37. Back-to-back testing • Present the same tests to different versions of the system and compare outputs. Differing outputs imply potential problems • Reduces the costs of examining test results. Automatic comparison of outputs. • Possible when a prototype is available or with regression testing of a new system version

38. Back-to-back testing

39. Incremental testing

40. Key Points • Verification and validation are not the same thing • Testing is used to establish the presence of faults and to show fitness for purpose • Testing activities include unit testing, module testing, sub-system testing, integration testing and acceptance testing • Object classes should be tested in O-O systems

41. Key Points • Testing should be scheduled as part of the planning process. Adequate resources must be made available • Test plans should be drawn up to guide the testing process • Testing strategies include top-down testing, bottom-up testing, stress testing, thread testing and back-to-back testing

42. Questions?

43. Schwan’s Development Standards

45. ObjectiveThe objective of the Business Test Cases is to have a written plan to confirm after creation that the system meets the business needs of the customer. This plan should include a list of functionality needed at the business level. Customers should help create the Business Test Cases. Required: ProjectsSupportOptional: <None>Deliverable: Business Test CasesDeliverable to: Systems DevelopmentCustomer (Optional)Responsibility: Business Systems PlanningSAP Tie: 2.4 Business Test Cases (260)

46. ObjectiveThe objective of the Test Plan is to take the Business Test Cases created by the Business Systems Planning group during project analysis and create a system test plan to be used during the test phase of the project development cycle. Required: Projects SupportOptional: <None>Deliverable: Test PlanDeliverable to: Systems DevelopmentResponsibility: Systems DevelopmentSAP Tie: 3.1 Create Test Plan (440)

47. ObjectiveThe objective of the Unit Testing is to run the application through a rigorous test using the test cases to verify all functionality works as expected. A unit test is required on all programs and should be repeatable. Required: ProjectsOptional: SupportDeliverable: Documented Repeatable Unit TestDeliverable to: Systems DevelopmentResponsibility: Systems DevelopmentSAP Tie: 3.5 Unit Test (530)

48. ObjectiveThe objective of the System Testing is to test the application(s) with the system to verify it has the desired effects and no undesired affects. Required: ProjectsOptional: SupportDeliverable: Documented Repeatable System TestDeliverable to: Systems DevelopmentResponsibility: Joint ResponsibilitySAP Tie: 4.3 System Test (540)

49. ObjectiveThe objective of the Acceptance Testing is to get approval from customers and IS on final product to be moved to production. Support Services may be involved with this step in preparation for support of the final system. Required: ProjectsOptional: SupportDeliverable: Customer ApprovalDeliverable to: Systems DevelopmentResponsibility: Joint ResponsibilitySAP Tie: 4.3, 4.6 Acceptance Test

50. Questions?