1 / 23

Software Testing and Reliability Software Test Process

Graduate Assistants :. Ramkumar Natarajan Baskar Sridharan. Software Testing and Reliability Software Test Process. Aditya P. Mathur Purdue University August 12-16 @ Guidant Corporation Minneapolis/St Paul, MN. Last update: August 9, 2002. Learning Objectives.

gyula
Download Presentation

Software Testing and Reliability Software Test Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Graduate Assistants: Ramkumar Natarajan Baskar Sridharan Software Testing and ReliabilitySoftware Test Process Aditya P. Mathur Purdue University August 12-16 @ Guidant Corporation Minneapolis/St Paul, MN Last update: August 9, 2002

  2. Learning Objectives • What is software life cycle? • How to mesh various types of testing into the development cycle? • What documents are generated for and as a result of testing? • What test-metrics needs to be collected and why? Software Testing and Reliability Aditya P. Mathur 2002

  3. References • Software Testing in the Real World, Edward Kit, Addison Wesley, 1995. Software Testing and Reliability Aditya P. Mathur 2002

  4. Life Cycle • Software Development Cycle is a sequence of steps in which a software system is developed. We shall refer to this sequence as, simply, Life Cycle. • There are many models of Life Cycle. • The waterfall model • Iterative refinement model • Incremental development model • Spiral model • Unified Process model Software Testing and Reliability Aditya P. Mathur 2002

  5. Testing and the Life Cycle • It is widely believed that testing is necessary in each phase of the life cycle regardless of which model is adopted. • The key questions with which most process managers struggle are: • How should the test process be “meshed” with the development cycle? • What tools and techniques should be used in each phase? • What data should be collected to evaluate the effectiveness of the test process? Software Testing and Reliability Aditya P. Mathur 2002

  6. Requirements specification Design specification System Integration System Deployment System Modification Code Integration Meshing the Test Process with the Life Cycle Run regression Tests Test the requirements Test the design Test for acceptance Test the modules Test the subsystems Test the system Software Testing and Reliability Aditya P. Mathur 2002

  7. Test Tools and Techniques: Requirements Phase • Inspection and walkthroughs: Ensure completeness, unambiguity, precision, consistency, testability, and traceability. [Rational: Rose and Analyst Studio, I-Logix: Rhapsody] • Simulation: Use this to improve the understanding of the system to be built; where possible, work with the customer. • Generate test procedures, plans, and test cases: Develop procedures, plans and test cases to be used for testing each requirement. Identify what tools will be used in each phase, who will use these, and what data will be collected. Determine product release criteria. Software Testing and Reliability Aditya P. Mathur 2002

  8. Test Tools and Techniques: Design Phase • Inspection and walkthroughs [Rational: Rose] • Simulation: Check for the correctness of scenarios. [I-Logix: Rhapsody] • Refine test procedures and test cases: Use the artifacts created during design to generate additional test cases and refine or correct the existing ones. • Risk analysis: Use the application architecture to perform risk analysis. [Rational: ROOM] Software Testing and Reliability Aditya P. Mathur 2002

  9. Test Tools and Techniques: Coding Phase • Inspection and walkthroughs: Ensure that the code matches the design. If not then at leant one of the two needs to be modified to achieve consistency.[Rational: Rose] • Perform unit testing: Test each module thoroughly. [CleanScape: TestWise; Rational: Purify and Visual Test; AMC: CodeTest] Software Testing and Reliability Aditya P. Mathur 2002

  10. Test Tools and Techniques: Integration Phase • Perform integration testing: Test each subsystem thoroughly. [CleanScape: TestWise; Rational: Purify and Visual Test; Applied Microsystems: CodeTest; Windriver: Sniff+] Software Testing and Reliability Aditya P. Mathur 2002

  11. Test Tools and Techniques: System Integration • Perform system testing: Test the system to ensure correct functionality with respect to use cases, do performance, stress, and robustness tests. [CMU: Ballista; Telcordia: AETG; Applied Microsystems: CodeTest; Windriver: Diab RTA suite and Sniff+] Software Testing and Reliability Aditya P. Mathur 2002

  12. Test Tools and Techniques: System Changes • Perform regression testing: Test the modified system to ensure that existing functions perform as intended. Note that system testing for the added features ought to be performed. [CleanScape: Testwise for test minimization] Software Testing and Reliability Aditya P. Mathur 2002

  13. Test Tools and Techniques: In-target system • Perform system testing: Test the application for all functionality, its performance, and robustness while the application runs embedded in the target platform. [Advanced Microsystems: CodeTest; Home-grown test generators; Rational: Purify] Software Testing and Reliability Aditya P. Mathur 2002

  14. Test Tools and Techniques: Usability testing • Perform usability testing: Have a sample of the potential users test the application and the documents that accompany it. The goal is to ensure that (a) the documents are understandable, consistent, and correct and (b) the GUI satisfies the requirements of the potential users. Usability testing can begin when the GUI is ready for use, i.e. developed and tested for correctness, but the remainder of the application code might not be. Software Testing and Reliability Aditya P. Mathur 2002

  15. Test Tools and Techniques: Security testing • Perform security and safety testing: Security and safety testing must be done when an application is accessible from the “outside” and might compromise the user’s privacy and safety, The objective of security and safety testing is to demonstrate that privacy and safety cannot be compromised when the assumptions regarding the use of the application are met. • Security and safety testing is done with respect to the requirements for security and safety. Traditional testing techniques are useful but might have to be augmented with techniques specific to the application environment. Software Testing and Reliability Aditya P. Mathur 2002

  16. Responsibilities • Planning: (High level) Test team • Unit tests: Developer. (Who checks the developers work?) • Integration tests: Developer and/or test team depending on the complexity of the sub-system to be tested. • System level tests: Test team. These tests will generally require communication with the developers. This must be a part of the detail plan built at the beginning of the development cycle. • Acceptance and usability tests: Test team in collaboration with the end user (or a sample of the end user set). Software Testing and Reliability Aditya P. Mathur 2002

  17. Release Criteria • Units: Code coverage based. Example: all statements, decisions, and conditions must be covered. • Integration tests: Code coverage and function based. Low level code coverage might not be usable if the integration produces a “large” component. • System level tests: System reliability measured in terms of the number and type of failures detected. • Acceptance and usability tests: Determined by the user primarily based on correct functionality, robustness, performance, and usability. Software Testing and Reliability Aditya P. Mathur 2002

  18. Standards for Test Documents • SQAP: : Software Quality Assurance Plan, IEEE/ANSI, 1989 [Std 730-1989]. • SVVP: Software Verification and Validation Plan, IEEE/ANSI, 1986 [Std 1012-1986]; one per SQAP. • VTP: Verification Test Plan, one per verification activity. • MVP: Master Validation Test Plan, IEEE/ANSI, 1983 [Std 829-1983]; one per SVVP. • DTP: Detailed Validation Test Plan, IEEE/ANSI, 1983 [Std 829-1983]; one or more per activity. Software Testing and Reliability Aditya P. Mathur 2002

  19. Standards for Test Documents (contd.) • TDS: : Test Design Specification, IEEE/ANSI, 1983 [Std 829-1983], one per DTP. • TCS: Test Case Specification IEEE/ANSI, 1983 [Std 829-1983]; one or more per TDS/TPS. • TPS: Test Procedure Specification, IEEE/ANSI, 1983 [Std 829-1983]; one or more per TDS. • TC: Test Case, one per TCS. Software Testing and Reliability Aditya P. Mathur 2002

  20. SQAP SVVP VTP VTP VTP MTP DTP DTP DTP DTP TDS TDS TDS TDS TCS TCS TPS TPS TC TC Structure of Standards for Test Documents Software Testing and Reliability Aditya P. Mathur 2002

  21. Defect Tracking and Classification • Defect tracking and classification [Applied Innovation management: Bug/Defect Tracking Expert;defectx: defectX] • Defect estimation: Use inspection data for defect estimation; design metrics; past data on similar projects. • Defects open vs. closed • Cross-project and cross version comparison of the number of defects. • Cost of repairing a defect: time and manpower resources needed. Software Testing and Reliability Aditya P. Mathur 2002

  22. Why collect data? • Assists with improving the test process; correlations between other process related metrics, such as code coverage, and defects can be established. • Helps improve product quality. • Useful in setting up a reward system (no penalty please!). Software Testing and Reliability Aditya P. Mathur 2002

  23. Summary • Software development cycle • Test process and the software development cycle • Assignment of responsibilities • Standards • Metrics Software Testing and Reliability Aditya P. Mathur 2002

More Related