MUTATION TESTING

1. MUTATION TESTING

2. MUTATION TESTING • Mutation testing is a technique that focuses on measuring the adequacy (quality) of test data (or test cases). • Modify a program by introducing a single small change to the code • A modified program is called mutant • A mutant is said to be killed when the execution of test case cause it to fail. The mutant is considered to be dead • A mutant is an equivalent to the given program if it always produce the same output as the original program • A mutant is called killable or stubborn, if the existing set of test cases is insufficient to kill it • A mutation score for a set of test cases is the percentage of non-equivalent mutants killed by the test suite • The test suite is said to be mutation-adequate if its mutation score is 100%

3. the following program P that finds rank corresponding to the first time the maximum value appears in the array 1. main(argc,argv) 2. intargc, r, i; 3. char *argv[]; 4. { r = 1; 5. for i = 2 to 3 do 6. if (atoi(argv[i]) > atoi(argv[r])) r = i; 7. printf("Value of the rank is %d \n", r); 8. exit(0); }

4. TestCases • Test case 1: Input: 1 2 3 • Output: Value of the rank is 3 • Test case 2: Input: 1 2 1 • Output: Values of the rank is 2 • Test case 3: Input: 3 1 2 • Output: Value of the rank is 1

5. let us mutate the program P. We can start with the following changes: 1. Mutant 1: Change line 5 to for i = 1 to 3 do 2. Mutant 2: Change line 6 to if (i > atoi(argv[r])) r = i; 3. Mutant 3: Change line 6 to if (atoi(argv[i]) >= atoi(argv[r])) r = i; 4. Mutant 4: Change line 6 to if (atoi(argv[r]) > atoi(argv[r])) r = i;

6. Mutants 1 and 3: The programs will completely pass the test suite. In other words, mutants 1 and 3 are not killed. • Mutant 2: The program will fail test case 2. • Mutant 4: The program will fail test case 1 and test case 2.

7. calculate the mutation score • Mutation score = 100×D/(N −E), where • D: is the dead mutants, • N: the total number of mutants, • E: the number of equivalent mutants. • Assuming that mutants 1 and 3 are nonequivalent. • mutation score is 50%. It is low because we assumed that mutants 1 and 3 are nonequivalent to the original program • we need to add new test cases to kill these two mutants

8. 1 and 3 are equivalent mutants or those are killable??!! • analyze mutant 1 --- is an equivalent mutant. • add a fourth test case as follows: • Test case 4: Input: 2 2 1 • P : the output “Value of the rank is 1” • mutant 3 : the output “Value of the rank is 2.” • Thus, this test data kills mutant 3, • which give us a mutation score of 100%.

9. Steps to build a robust test suite • Step 1: Begin with a program P and a set of test cases T known to be correct. • Step 2: Run each test case in T against the program P. • If it fails (o/p incorrect) P must be modified and restarted. Else, go to step 3 • Step 3: Create a set of mutants {Pi }, each differing from P by a simple, syntactically correct modification of P.

10. Step 4: Execute each test case in T against each mutant Pi . • If the o/p is differthe mutant Pi is considered incorrect and is said to be killed by the test case • If Pi produces exactly the same results: • P and Pi are equivalent • Pi is killable (new test cases must be created to kill it)

11. Step 5: Calculate the mutation score for the set of test cases T. • Mutation score = 100×D/(N −E), • Step 6: If the estimated mutation adequacy of T in step 5 is not sufficiently high, then design a new test case that distinguishes Pi from P, add the new test case to T, and go to step 2.