Test & QA Process

1. Test & QA Process Presentation by Imran Ali Shamim

2. In this presentation….. • Software Quality. • What is Verification & Validation? • Verification Strategies. • Validation Strategies. • Establishing a Software Testing Methodology. • When Testing Should Occur? • Types of Testing. • Test Phases. • Configuration Management. • Defect Tracking Process. • Deliverables. Test & QA Process

3. Software Quality • What is quality? • Software Quality • Software functional quality • Software structural quality • Software Quality Assurance (SQA) • Software Quality Control (SQC) Test Process.

4. What is Verification & Validation? Verification and Validation are the basic ingredients of Software Quality Assurance (SQA) activities. “Verification” checks whether we are building the right system, and “Validation” checks whether we are building the system right. Test & QA Process

5. Verification Strategies Verification Strategies comprise of the following: Requirements Review. Design Review. Code Walkthrough. Code Inspections. Test & QA Process

6. Validation Strategies Validation Strategies comprise of the following: Unit Testing. Integration Testing. System Testing. Performance Testing. Alpha Testing. User Acceptance Testing (UAT). Installation Testing. Beta Testing. Test & QA Process

7. Verification Strategies…in detail Test & QA Process

8. Validation Strategies…in detail Test & QA Process

9. Validation Strategies…in detail Test & QA Process

10. Establishing a Software Testing Methodology. • Most software development companies have quiet a vast experience in software testing field and uses following Five-Step testing methodology which can help us to minimize project risk, streamlining test releases. • We can start the testing course by developing a comprehensive test plan. The test plan includes the testing criteria for general functionalities along with specific requirements, on a number of platform combinations. • Test Planning • Testing Environment and Harness preparation • Testing Execution • Testing Reporting • Final Reporting and Analysis Test & QA Process

11. Establishing a Software Testing Methodology. • 1. Test Planning • It includes the following steps: • Collecting of requirements, their analysis and specification; • Test plan, test scenario, test cases preparation and approval; • Automated testing tools appropriateness assessment, tools selection and confirmation. • 2. Testing Environment and Harness preparation • Test automation framework design and development, test scripts creation. There is a large set of testing equipment, system software and Web middleware. This ensures creation of proper testing environment and essentially decreases the time for testing configurations deployment. Test & QA Process

12. Establishing a Software Testing Methodology. • 3. Testing Execution • Manual and automated software testing; • Functional • Bug and functional regression • Load / Performance / Stress • Compatibility • Usability (GUI) • Configuration • Localization • Documentation • For each of these types of testing we have established approaches, proven testing tools, and adjusted reporting documents templates. Testing techniques applied: • Black Box / Gray Box / White Box testing • Ad Hoc / Exploratory testing • Scripted testing Test & QA Process

13. Establishing a Software Testing Methodology. 4. Testing Reporting Testing reporting, analysis and bug fixing management.We can use the industry-recognized bug tracking tools like Jira, BugZilla etc. It allows both parts of team (Development & Test) to work as a single team with better communication among them for reporting, analyzing and fixing bugs. It also allows better and timely management of bugs and bugs tracking. 5. Final Reporting and Analysis Defects analysis and recommendations regarding improvement of the tested software and the process of its development. Test & QA Process

14. When Testing should occur..? • Testing can and should occur throughout the phases of a project. • Requirements Phase • Determine the test strategy. • Determine adequacy of requirements. • Generate functional test conditions. • Design Phase • Determine consistency of design with requirements. • Determine adequacy of design. • Generate structural and functional test conditions. • Program (Build) Phase • Determine consistency with design. • Determine adequacy of implementation. • Generate structural and functional test conditions for programs/units. Test & QA Process

15. When Testing should occur..? • Test Phase • Determine adequacy of the test plan. • Test application system. • Installation Phase • Place tested system into production. • Maintenance Phase • Modify and retest. Test & QA Process

16. Types of Testing. • Two types of testing can be taken into consideration. • Functional or Black Box Testing. • Structural or White Box Testing. • Functional testing ensures that the requirements are properly satisfied by the application system. The functions are those tasks that the system is designed to accomplish. • Structural testing ensures sufficient testing of the implementation of a function. Test & QA Process

17. Structural Testing. Test & QA Process

18. Structural Testing. Test & QA Process

19. Functional Testing. Test & QA Process

20. Functional Testing. Test & QA Process

21. Test Phases. Design Review Requirements Review Code Inspection Code Walkthrough Unit Testing Integration Testing Performance Testing System Testing Alpha Testing User Acceptance Testing Beta Testing Installation Testing Test & QA Process

22. Test Phases and Definitions Formal Technical Review’s (FTR) The focus of FTR is on a work product (e.g. Requirements document, Code etc.). After the work product is developed, the Project Leader calls for a Review. The work product is distributed to the personnel who involves in the review. The main audience for the review should be the Project Manager, Project Leader and the Producer of the work product. Major reviews include the following: 1. Requirements Review. 2. Design Review. 3. Code Review. Test & QA Process

23. Test Phases and Definitions Unit Testing Goal of Unit testing is to uncover defects using formal techniques like Boundary Value Analysis (BVA), Equivalence Partitioning, and Error Guessing. Defects and deviations in Date formats, Special requirements in input conditions (for example Text box where only numeric or alphabets should be entered), selection based on Combo Box’s, List Box’s, Option buttons, Check Box’s would be identified during the Unit Testing phase. Integration Testing Integration testing is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing. The objective is to take unit tested components and build a program structure that has been dictated by design. Usually, the following methods of Integration testing are followed: 1. Top-down Integration approach. 2. Bottom-up Integration approach. Test & QA Process

24. Test Phases and Definitions • Top-down Integration • Top-down integration testing is an incremental approach to construction of program structure. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module. Modules subordinate to the main control module are incorporated into the structure in either a depth-first or breadth-first manner. • The Integration process is performed in a series of five steps: • The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module. • Depending on the integration approach selected subordinate stubs are replaced one at a time with actual components. • Tests are conducted as each component is integrated. • On completion of each set of tests, another stub is replaced with the real component. • Regression testing may be conducted to ensure that new errors have not been introduced. Test & QA Process

25. Test Phases and Definitions • Bottom-up Integration • Button-up integration testing begins construction and testing with atomic modules (i.e. components at the lowest levels in the program structure). Because components are integrated from the button up, processing required for components subordinate to a given level is always available and the need for stubs is eliminated. • A Bottom-up integration strategy may be implemented with the following steps: • Low level components are combined into clusters that perform a specific software sub function. • A driver is written to coordinate test case input and output. • The cluster is tested. • Drivers are removed and clusters are combined moving upward in the program structure. Test & QA Process

26. Test Phases and Definitions • System Testing • System testing is a series of different tests whose primary purpose is to fully exercise the computer based system. Although each test has a different purpose, all work to verify that system elements have been properly integrated and perform allocated functions. • The following tests can be categorized under System testing: • Recovery Testing. • Security Testing. • Stress Testing. • Performance Testing. • Recovery Testing • Recovery testing is a system test that focuses the software to fall in a variety of ways and verifies that recovery is properly performed. If recovery is automatic, reinitialization, checkpointing mechanisms, data recovery and restart are evaluated for correctness. If recovery requires human intervention, the mean-time-to-repair (MTTR) is evaluated to determine whether it is within acceptable limits. Test & QA Process

27. Test Phases and Definitions • Security Testing • Security testing attempts to verify that protection mechanisms built into a system will, in fact, protect it from improper penetration. During Security testing, password cracking, unauthorized entry into the software, network security are all taken into consideration. • Stress Testing • Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume. The following types of tests may be conducted during stress testing: • Special tests may be designed that generate ten interrupts per second, when one or two is the average rate. • Input data rates may be increases by an order of magnitude to determine how input functions will respond. • Test Cases that require maximum memory or other resources. • Test Cases that may cause excessive hunting for disk-resident data. • Test Cases that my cause thrashing in a virtual operating system. Test & QA Process

28. Test Phases and Definitions • Performance Testing • Performance tests are coupled with stress testing and usually require both hardware and software instrumentation. • Regression Testing • Regression testing is the re-execution of some subset of tests that have already been conducted to ensure that changes have not propagated unintended side affects. • Regression may be conducted manually, by re-executing a subset of al test cases or using automated capture/playback tools. • The Regression test suit contains three different classes of test cases: • A representative sample of tests that will exercise all software functions. • Additional tests that focus on software functions that are likely to be affected by the change. • Tests that focus on the software components that have been changed. Test & QA Process

29. Test Phases and Definitions Alpha Testing The Alpha testing is conducted at the developer sites and in a controlled environment by the end-user of the software. User Acceptance Testing User Acceptance testing occurs just before the software is released to the customer. The end-users along with the developers perform the User Acceptance Testing with a certain set of test cases and typical scenarios. Beta Testing The Beta testing is conducted at one or more customer sites by the end-user of the software. The beta test is a live application of the software in an environment that cannot be controlled by the developer. Test & QA Process

30. Configuration Management • Software Configuration management is an umbrella activity that is applied throughout the software process. SCM identifies controls, audits and reports modifications that invariably occur while software is being developed and after it has been released to a customer. All information produced as part of software engineering becomes of software configuration. The configuration is organized in a manner that enables orderly control of change. • The following is a sample list of Software Configuration Items: • Management plans (Project Plan, Test Plan, etc.) • Specifications (Requirements, Design, Test Case, etc.) • Customer Documentation (Implementation Manuals, User Manuals, Operations Manuals, On-line help Files) • Source Code (PL/1 Fortran, COBOL, Visual Basic, Visual C, etc.) • Executable Code (Machine readable object code, exe's, etc.) • Libraries (Runtime Libraries, Procedures, %include Files, API's, DLL's, etc.) • Databases (Data being Processed, Data a program requires, test data, Regression test data, etc.) • Production Documentation Test & QA Process

31. The Tester/Developer finds the Bug. Reports the Defect in the Defect Tracking Tool. Status “Open” The concerned Developer is informed The Developer fixes the Defect If the Defect re-occurs, the status changes to “Re-Open” The Developer changes the Status to “Resolved” The Tester Re-Tests and changes Status to “Closed” Defect Tracking Process. Test & QA Process

32. Defect Classification. This section defines a defect Severity Scale framework for determining defect criticality and the associated defect Priority Levels to be assigned to errors found software. The defects can be classified as follows: Critical: There is s functionality block. The application is not able to proceed any further. Major: The application is not working as desired. There are variations in the functionality. Minor: There is no failure reported due to the defect, but certainly needs to be rectified. Cosmetic: Defects in the User Interface or Navigation. Suggestion: Feature which can be added for betterment. Test & QA Process

33. Defect Priority. The priority level describes the time for resolution of the defect. The priority level would be classified as follows: Immediate: Resolve the defect with immediate effect. At the Earliest: Resolve the defect at the earliest, on priority at the second level. Normal: Resolve the defect. Later: Could be resolved at the later stages. Test & QA Process

34. Deliverables. The Deliverables from the Test team would include the following: Test Plan. Test Case Documents. Defect Reports. Status Reports (Daily/weekly/Monthly). Test Scripts (if any). Metric Reports. Product Sign off Document. Test & QA Process

35. Thank You. Test & QA Process