1 / 11

MUTATION TESTING

MUTATION TESTING. MUTATION TESTING. Mutation testing is a technique that focuses on measuring the adequacy (quality) of test data (or test cases). Modify a program by introducing a single small change to the code A modified program is called mutant

tahlia
Download Presentation

MUTATION TESTING

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. MUTATION TESTING

  2. MUTATION TESTING • Mutation testing is a technique that focuses on measuring the adequacy (quality) of test data (or test cases). • Modify a program by introducing a single small change to the code • A modified program is called mutant • A mutant is said to be killed when the execution of test case cause it to fail. The mutant is considered to be dead • A mutant is an equivalent to the given program if it always produce the same output as the original program • A mutant is called killable or stubborn, if the existing set of test cases is insufficient to kill it • A mutation score for a set of test cases is the percentage of non-equivalent mutants killed by the test suite • The test suite is said to be mutation-adequate if its mutation score is 100%

  3. the following program P that finds rank corresponding to the first time the maximum value appears in the array 1. main(argc,argv) 2. intargc, r, i; 3. char *argv[]; 4. { r = 1; 5. for i = 2 to 3 do 6. if (atoi(argv[i]) > atoi(argv[r])) r = i; 7. printf("Value of the rank is %d \n", r); 8. exit(0); }

  4. TestCases • Test case 1: Input: 1 2 3 • Output: Value of the rank is 3 • Test case 2: Input: 1 2 1 • Output: Values of the rank is 2 • Test case 3: Input: 3 1 2 • Output: Value of the rank is 1

  5. let us mutate the program P. We can start with the following changes: 1. Mutant 1: Change line 5 to for i = 1 to 3 do 2. Mutant 2: Change line 6 to if (i > atoi(argv[r])) r = i; 3. Mutant 3: Change line 6 to if (atoi(argv[i]) >= atoi(argv[r])) r = i; 4. Mutant 4: Change line 6 to if (atoi(argv[r]) > atoi(argv[r])) r = i;

  6. Mutants 1 and 3: The programs will completely pass the test suite. In other words, mutants 1 and 3 are not killed. • Mutant 2: The program will fail test case 2. • Mutant 4: The program will fail test case 1 and test case 2.

  7. calculate the mutation score • Mutation score = 100×D/(N −E), where • D: is the dead mutants, • N: the total number of mutants, • E: the number of equivalent mutants. • Assuming that mutants 1 and 3 are nonequivalent. • mutation score is 50%. It is low because we assumed that mutants 1 and 3 are nonequivalent to the original program • we need to add new test cases to kill these two mutants

  8. 1 and 3 are equivalent mutants or those are killable??!! • analyze mutant 1 --- is an equivalent mutant. • add a fourth test case as follows: • Test case 4: Input: 2 2 1 • P : the output “Value of the rank is 1” • mutant 3 : the output “Value of the rank is 2.” • Thus, this test data kills mutant 3, • which give us a mutation score of 100%.

  9. Steps to build a robust test suite • Step 1: Begin with a program P and a set of test cases T known to be correct. • Step 2: Run each test case in T against the program P. • If it fails (o/p incorrect) P must be modified and restarted. Else, go to step 3 • Step 3: Create a set of mutants {Pi }, each differing from P by a simple, syntactically correct modification of P.

  10. Step 4: Execute each test case in T against each mutant Pi . • If the o/p is differthe mutant Pi is considered incorrect and is said to be killed by the test case • If Pi produces exactly the same results: • P and Pi are equivalent • Pi is killable (new test cases must be created to kill it)

  11. Step 5: Calculate the mutation score for the set of test cases T. • Mutation score = 100×D/(N −E), • Step 6: If the estimated mutation adequacy of T in step 5 is not sufficiently high, then design a new test case that distinguishes Pi from P, add the new test case to T, and go to step 2.

More Related