Some Simple Definitions for Testing

1. Some Simple Definitions for Testing • Error: • a mistake made by some person (no input y = 0 check) • Fault or Defect: • the result of an error (several divide by y places in code with no input y check) • there may be many faults or defects caused by a single error • Failure (or problems): • the result of a defect that manifests itself during execution • there may be several failures due to a single defect • there may never be any failure even when there are defects or faults simply because the specific combination of conditions never occurred during the execution. (y never assigned 0 value during execution)

2. Testing ( at Program Level) • Testing is finding faults (in contrast to the notion of showing that the system works) once the code is complete and executable. We want to find defects caused by: • errors made in the actual programming task • errors made from wrong requirement specification and propagated through to programming phase • -errors made from faulty design and propagated into the program • errors made in translating from good requirements/design to code • Testing has two major components: • preparing for and identifying faults (test case dev. & testexecution • correcting and removing the faults(fault debugging and correction --------- followed by retest

3. Some Types of Faults(Defects) • As part of the fault identification, we need to classify the types of faults: (why worry about this classification stuff?) • algorithmic fault : logic error such as wrong initialization, wrong comparison, etc. • computation and precision fault: implementing wrong formula, lack of degree of accuracy, etc. • stress fault : limited buffer size, undersized queue size, etc. • performance & capacity fault : not enough throughput, limited number of simultaneous users, etc. that are lesser than what requirements asked for • timing fault: real-time processing miss where the timing of multiple, synchronous processing is key • fault recovery fault: lack of or erroneous fault processing • documentation fault: erroneous documentation of design or of requirements specification • interface fault : system to system or hardware ----- to software UI interfaces not working as specified or documented

4. Classification of Defects • There are several ways to classify defects: (why do we care about these ?) • by activity source of defect : activity of the development phase( such as requirements, high level design, prototype, etc.) where error was made • by type of defects : as shown earlier or some other organizational choices (such as UI interface, component invocation, database, etc.) • by people source (such as analyst, architect, designer, programmer, etc.) • By severity of defects (such as stops the processing, produces erroneous results, looks ugly, etc.) • An important aspect of classification is to have the defect classification be “orthogonal” as in orthogonal vector from linear algebra.

5. Types of Testing • There are several levels of testing • Unit testing : e.g. module z • often performed by the programmer himself/herself • testing only that independent chunk of code • Functional testing : e.g. printing checks • testing a group of units for a particular required function • often performed by someone else than the code author • Component testing : “monthly” employee payments • testing a group of related functions for a piece of major requirements • should be performed by an independent test group • System testing : payroll system • testing the complete system • may be further divided into : performance, regression, acceptance, installation, etc. kind of testing

6. One View of Levels of Testing Unit test Functional test Unit test . . . . Component test System test . . Unit test . . . . . Acceptance; installation; performance; regression; etc. Component test Functional test Unit test

7. Views on Testing • Programmers are usually builders and want to construct and show what they have completed works. • Testersmust show that the software works under all conditions prescribed by the requirements statements andmore ---- through finding defects. • Putting together conditions that are both “ordinary” and “exceptional” • looking for faults • Development must embrace both the positive and the negativeattitudes and work together • up to unit tests may be conducted by programmers • all other tests should be performed by an independent test group for both productivity and neutrality reasons

8. Black Box and White Box Tests • Black Box : • testing the functions from requirement statements • do not look inside the module and the code • can not catch any extraneous code that may have faults • White Box : • testing the internal logic and all the logic via structural paths • look at the inside of the module and the code • can not easily detect missing function • We actually use both testing method, based on : • test objectives and goal • complexity of the system • possible number of logical paths

9. Ideal Testing Situation Requirement. Coding testing Requirements grow and not all are implemented Implementation grows more than the requirements Requirements, implementation and testing almost (may never be perfect) matches

10. Unit Testing • This is the testing of a module or a unit of code within a module • usually by the programmer • natural “flow” would include: • checking to see if all the functions are included via reviewing the code • compiling to make sure that there is no obvious syntax or compile error • generate test cases to demonstrate that the test inputs produce the expected results (both valid and invalid inputs) • run the test cases • as failures occur, defects are analyzed and fixed • the fixed code and the test case is rerun • repeat the previous two steps until all test cases are ran and no failure occurs.

11. Unit Test (cont.) • Prior to the days when PC and machine time is widely available, code inspections and reviewswere often times, both an economically and defect discovery rate wise, better choice. • Today, with the PC and reduced CPU cost, the developers are often running the various unit test cases and fixing his/her own code • Many development organizations do not force unit tests to be conducted by a different person or group • some unit test guideline may be provided • exit criteria for unit test is also provided (e.g clean compile)