Software Testing

Software Testing Software testing is an integral part of Software Development Life Cycle

Software Development Life Cycle (SDLC) • is a procedural process, in the development of a software product. The process is carried in a set of steps, which explains the whole idea about how to go through each product. The classification of Software Development Life Cycle process is as follows • Planning • Analysis • Design • Software Development • Implementation • Software Testing • Deployment • Maintenance Software Testing is an important factor in a product's life cycle, as the product will have greater life, only when it works correctly and efficiently according to the customer's requirements.

Introduction to Software Testing • Before moving further towards introduction to software testing, we need to know a few concepts that will simplify the definition of software testing. • Error: Error or mistake is a human action that produces wrong or incorrect result. • Defect (Bug, Fault): A flaw in the system or a product that can cause the component to fail or misfunction. • Failure: It is the variance between the actual and expected result. • Risk: Risk is a factor that could result in negativity or a chance of loss or damage. Thus Software testing is the process of finding defects/bugs in the system, that occurs due to an error in the application, which could lead to failure of the resultant product and increase in probability of high risk.

software testing have different goals and objectives • finding defects; • gaining confidence in and providing information about the level of quality; • preventing defects;

Scope of Software Testing The primary function of software testing is to detect bugs in order to correct and uncover it. The scope of software testing includes • execution of that code in various environment and • also to examine the aspects of code – • does the software do what it is supposed to do • function according to the specifications?

When to start testing?" and "When to stop testing? • It is recommended to start testing from the initial stages of the software development. This not only helps in rectifying tremendous errors before the last stage, but also reduces the rework of finding the bugs in the initial stages every now and then. It also saves the cost of the defect required to find it. Software testing is an ongoing process, which is potentially endless but has to be stopped somewhere, due to the lack of time and budget. It is required to achieve maximum profit with good quality product, within the limitations of time and money. The tester has to follow some procedural way through which he can judge if he covered all the points required for testing or missed out any

Software Testing Key Concepts • Defects and Failures: As we discussed earlier, defects are not caused only due to the coding errors, but most commonly due to the requirement gaps in the non-functional requirement, such as usability, testability, scalability, maintainability, performance and security. A failure is caused due to the deviation between an actual and an expected result. But not all defects result to failures. A defect can turn into a failure due to the change in the environment and or the change in the configuration of the system requirements. • Input Combination and Preconditions: Testing all combination of inputs and initial state (preconditions), is not feasible. This means finding large number of infrequent defects is difficult. • Static and Dynamic Analysis: Static testing does not require execution of the code for finding defects, whereas in dynamic testing, software code is executed to demonstrate the results of running tests. • Verification and Validation: Software testing is done considering these two factors.

Verification and Validation • Verification: This verifies whether the product is done according to the specification? • Validation: This checks whether the product meets the customer requirement?

Software Testing Artifacts • Test Plan: A test specification is called a test plan. A test plan is documented so that it can be used to verify and ensure that a product or system meets its design specification. • Traceability matrix: This is a table that correlates or design documents to test documents. This verifies that the test results are correct and is also used to change tests when the source documents are changed. • Test Case: Test cases and software testing strategies are used to check the functionality of individual component that is integrated to give the resultant product. These test cases are developed with the objective of judging the application for its capabilities or features. • Test Data: When multiple sets of values or data are used to test the same functionality of a particular feature in the test case, the test values and changeable environmental components are collected in separate files and stored as test data. • Test Scripts: The test script is the combination of a test case, test procedure and test data. • Test Suite: Test suite is a collection of test cases.

Software Testing Process • Software testing process is carried out in the following sequence, in order to find faults in the software system: • Create Test Plan • Design Test Case • Write Test Case • Review Test Case • Execute Test Case • Examine Test Results • Perform Post-mortem Reviews

Types of tests

Smoke Test • smoke test is a series of test cases that are run prior to commencing with full-scale testing of an application. The idea is to test the high-level features of the application to ensure that the essential features work. If they do not work, there is no need to continue testing details of the application, so the testing team will refuse to do any additional testing until all smoke test cases pass. This puts pressure on the development team to ensure that the testing team does not waste valuable testing time with code that is not ready to be tested.

smoke tests • Once smoke tests are implemented as part of your software development life cycle, you will see the overall quality of the product improve and the sensitivity to producing high quality software increased.

Smoke Tests - How to Implement • To implement smoke tests, the quality assurance (QA) lead will develop a set of test cases that are run any time a new release is provided from the development team. These set of test cases are used to test the major functional areas of the system. They are not nit-picky test cases that test low-level detail of the application, they are higher level test cases that test overall functionality. Normally, a smoke test will consist of 20 to 30 high level test cases. • To ensure quality awareness, the smoke test cases are publicized to the development team ahead of time, so that they are aware of the quality expectations. If a build fails a smoke test, it should not be taken lightly, as this may illustrate a discipline problem with the development team. Each developer should be held responsible for ensuring that their code does not break the smoke test. Under no circumstance should the QA team agree to continue testing the release until all the smoke test cases have passed.

Functional Testing • Functional testing covers how well the system executes the functions it is supposed to execute—including user commands, data manipulation, searches and business processes, user screens, and integrations. Functional testing covers the obvious surface type of functions, as well as the back-end operations (such as security and how upgrades affect the system).

System Test • System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic.

System Testing ( Cont…) • As a rule, system testing takes, as its input, all of the "integrated" software components that have successfully passed integration testing and also the software system itself integrated with any applicable hardware system(s). The purpose of integration testing is to detect any inconsistencies between the software units that are integrated together (called assemblages) or between any of the assemblages and the hardware. System testing is a more limiting type of testing; it seeks to detect defects both within the "inter-assemblages" and also within the system as a whole.

System Testing ( Cont…) • System testing is performed on the entire system in the context of a Functional Requirement Specification(s) (FRS) and/or a System Requirement Specification (SRS).).

GUI software testing • GUI software testing is the process of testing a product that uses a graphical user interface, to ensure it meets its written specifications. This is normally done through the use of a variety of test cases

Usability testing • is a technique used to evaluate a product by testing it on users. • it gives direct input on how real users use the system • Usability testing is a black-box testing technique. The aim is to observe people using the product to discover errors and areas of improvement.

Goals of usability testing • Usability testing generally involves measuring how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response. The results of the first test can be treated as a baseline or control measurement; all subsequent tests can then be compared to the baseline to indicate improvement. • Performance -- How much time, and how many steps, are required for people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.) • Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?) • Recall -- How much does the person remember afterwards or after periods of non-use? • Emotional response -- How does the person feel about the tasks completed? Is the person confident, stressed? Would the user recommend this system to a friend?

Compatibility testing • Compatibility testing, part of software non-functional tests, is testing conducted on the application to evaluate the application's compatibility with the computing environment. Computing environment may contain some or all of the below mentioned elements: • Computing capacity of Hardware Platform (IBM 360, HP 9000, etc.).. • Bandwidth handling capacity of networking hardware • Compatibility of peripherals (Printer, DVD drive, etc.) • Operating systems (MVS, UNIX, Windows, etc.) • Database (Oracle, Sybase, DB2, etc.) • Other System Software (Web server, networking/ messaging tool, etc.) • Browser compatibility (Firefox, Netscape, Internet Explorer, Safari, etc.)

sanity test • A sanity test or sanity check is a basic test to quickly evaluate whether a claim or the result of a calculation can possibly be true

Exploratory testing • Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and test execution

Ad hoc testing • Ad hoc testing is a commonly used term for software testing performed without planning and documentation • The tests are intended to be run only once, unless a defect is discovered. Ad hoc testing is a part of exploratory testing, being the least formal of test methods. In this view, ad hoc testing has been criticized because it isn't structured, but this can also be a strength: important things can be found quickly. It is performed with improvisation, the tester seeks to find bugs with any means that seem appropriate. It contrasts to regression testing that looks for a specific issue with detailed reproduction steps, and a clear expected result. Ad hoc testing is most often used as a complement to other types of testing.

Performance Testing • Performance Testing covers a broad range of engineering or functional evaluations where a material, product, system, or person is not specified by detailed material or component specifications: • software performance testing is used to determine the speed or effectiveness of a computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring the response time or the number of MIPS (millions of instructions per second) at which a system functions.

Load testing • Load testing is the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any bottlenecks and determine which element is causing degradation. When the load placed on the system is raised beyond normal usage patterns, in order to test the system's response at unusually high or peak loads, it is known as stress testing. The load is usually so great that error conditions are the expected result, although no clear boundary exists when an activity ceases to be a load test and becomes a stress test.

Stress testing • Stress testing is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results

Regression testing • Regression testing is any type of software testing that seeks to uncover software errors by partially retesting a modified program. The intent of regression testing is to provide a general assurance that no additional errors were introduced in the process of fixing other problems. Regression testing is commonly used to test the system efficiently by systematically selecting the appropriate minimum suite of tests needed to adequately cover the affected change. Common methods of regression testing include rerunning previously run tests and checking whether previously fixed faults have re-emerged. "One of the main reasons for regression testing is that it's often extremely difficult for a programmer to figure out how a change in one part of the software will echo in other parts of the software

sample Test Case • Here is a sample Test Case for you:# Software Test Case for Login Page: • Purpose: The user should be able to go to the Home page. • Pre-requisite: • S/w should be compatible with the Operating system. • Login page should appear. • User Id and Password textboxes should be available with appropriate labels. • Submit and Cancel buttons with appropriate captions should be available. • Test Data: Required list of variables and their values should be available.eg: User Id:{Valid UserId, Invalid UserId, empty}, Password:{Valid, Invalid, empty}.

Software Testing