1 / 65

Data Dependence Based Testability Transformation in Automated Test Generation

Data Dependence Based Testability Transformation in Automated Test Generation. Presented by: Qi Zhang. Outline. Introduction to test data generation Test data generation methods Data dependence oriented test generation Testability transformation in test data generation Conclusions.

tsargent
Download Presentation

Data Dependence Based Testability Transformation in Automated Test Generation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Dependence Based Testability Transformation in Automated Test Generation Presented by: Qi Zhang

  2. Outline • Introduction to test data generation • Test data generation methods • Data dependence oriented test generation • Testability transformation in test data generation • Conclusions

  3. Test Data Generation Problem Given: a target Goal: find a program input on which the target is executed

  4. Example F(int a[10], int b[10], int target) { int i; bool fa, fb; i=1; fa=false; fb=false; while (i < 10 { if (a[i] == target) fa=true; i=i + 1; } if (fa == true) { i=1; fb=true; while (i < 10) { if (b[i] != target) fb=false; i=i+1; } } if (fb==true) printf(“message1”); else printf(“message2”); } target statement

  5. Target • A statement • A branch • A path • A data flow • A multiple condition • An assertion • A specific output value • …

  6. Application of Test Data Generation • Code-based (white-box) testing • Identification of program properties • Specification-based testing • Testing specification conformance • …

  7. Test Data Generation Methods • Random test generation • Path-oriented test generation • Symbolic execution oriented test generation • Execution-oriented test generation • Goal-oriented test generation • Chaining approach of test generation • Simulated annealing • Evolutionary algorithms • …

  8. Path-Oriented Test Generation Target statement S Select path P to target statement S Find input to execute path P no An input to execute path P and target statement S yes Input found?

  9. Example for path oriented test generation en 1 2 1 input (a,n); 2 max=a[1]; 3 min=a[1]; 4 i=2; 5 while (i<=n) { 6,7 if (max<a[i]) max=a[i]; 8,9 if (min>a[i]) min=a[i]; 10 i=i+1; } 11 output(min,max); 3 4 5 6 7 8 9 10 11 ex

  10. Path-oriented test generation • Finding input to execute the selected path • Symbolic execution oriented test generation • Execution-oriented test generation

  11. Path-Oriented Test Generation Problems: • Selected paths are frequently non-executable • A lot of search effort is “wasted” on non-executable paths • It is considered a restrictive in the presence of loops

  12. Goal-Oriented Test Generation • Paths are not selected • Based on actual program execution • A control graph of the program is used • It solves problems (sub-goals) as they occur to reach the target statement • Fitness functions are used to guide the search

  13. Goal-Oriented Test Generation x execute program on any input problem node this execution may lead to the target this execution does not lead to the target target statement

  14. Goal-Oriented Test Generation en 1 target statement 2 1 input (a,n); 2 max=a[1]; 3 min=a[1]; 4 i=2; 5 while (i<=n) { 6,7 if (max<a[i]) max=a[i]; 8,9 if (min>a[i]) min=a[i]; 10 i=i+1; } 11 output(min,max); 3 4 5 6 7 8 9 10 11 ex

  15. Goal-Oriented Test Generation en Initial input: 1 a={2, 7}, n=-5 2 1 input (a,n); 2 max=a[1]; 3 min=a[1]; 4 i=2; 5 while (i<=n) { 6,7 if (max<a[i]) max=a[i]; 8,9 if (min>a[i]) min=a[i]; 10 i=i+1; } 11 output(min,max); 3 4 5 6 7 8 9 10 11 ex

  16. Goal-Oriented Test Generation en Initial input: 1 a={2, 7}, n=-5 2 1 input (a,n); 2 max=a[1]; 3 min=a[1]; 4 i=2; 5 while (i<=n) { 6,7 if (max<a[i]) max=a[i]; 8,9 if (min>a[i]) min=a[i]; 10 i=i+1; } 11 output(min,max); F=i-n=7 3 4 5 6 7 8 9 10 11 find new value of a and n such that F<=0 ex

  17. Goal-Oriented Test Generation • There are many searching algorithms that can be used to find a new program input based on the fitness function • Hill-climbing algorithm • Simulated annealing • Evolutionary algorithm • …

  18. Chaining Approach • The chaining approach is an extension of the goal-oriented approach • The chaining approach uses: • Control flow graph • Data flow (data dependence) information

  19. 1 void F(int A[], int C[]) { int i, j, top, f_exit; 2 i=1; 3 j = 1 ; 4 top = 0 ; 5 f_exit=0; 6 while (C[j]<5) { 7 j = j + 1 ; 8 if (C[j] == 1) { 9 i = i + 1 ; 10 if (A[i] > 0) { 11,12 top = top + 1; AR[top] = A[i] ; }; }; 13 if (C[j] == 2) { 14 if (top>0) { 15,16 write(AR[top]); top = top - 1 ; }; }; 17 if (C[j]==3) { 18,19 if (top>100) {write(1);} //target statement 20 else write(0); }; }; //endwhile }

  20. data dependence concepts There exists a data dependence between statement S1 and S2 if: • S1 is a definition of variable v (assigns value to v) • S2 is an use of variable v (references v) • There exists a path in the program from S1 to S2 along which v is not modified

  21. 1 void F(int A[], int C[]) { int i, j, top, f_exit; 2 i=1; 3 j = 1 ; 4 top = 0 ; 5 f_exit=0; 6 while (C[j]<5) { 7 j = j + 1 ; 8 if (C[j] == 1) { 9 i = i + 1 ; 10 if (A[i] > 0) { 11,12 top = top + 1; AR[top] = A[i] ; }; }; 13 if (C[j] == 2) { 14 if (top>0) { 15,16 write(AR[top]); top = top - 1 ; }; }; 17 if (C[j]==3) { 18,19 if (top>100) {write(1);} 20 else write(0); }; }; //endwhile }

  22. Chaining Approach • It may significantly increase chances of finding inputs over the goal-oriented approach • It relies on direct data dependences related to problem statements • The chaining approach does not have a “global view” of dependences in the program

  23. Data Dependence Based Test Generation • We present data dependence based test generation • This approach uses a data dependence graph rather than individual data dependences during the search

  24. 1 void F(int A[], int C[]) { int i, j, top, f_exit; 2 i=1; 3 j = 1 ; 4 top = 0 ; 5 f_exit=0; 6 while (C[j]<5) { 7 j = j + 1 ; 8 if (C[j] == 1) { 9 i = i + 1 ; 10 if (A[i] > 0) { 11,12 top = top + 1; AR[top] = A[i] ; }; }; 13 if (C[j] == 2) { 14 if (top>0) { 15,16 write(AR[top]); top = top - 1 ; }; }; 17 if (C[j]==3) { 18,19 if (top>100) {write(1);} 20 else write(0); }; }; //endwhile }

  25. Data Dependence Based Test Generation • Data dependence based test generation is used when the existing methods fail to find the solution • Suppose the existing methods fail at some conditional statement (predicate) which is referred to as a problem node • The data dependence based test generation constructs a data dependence graph which contains the statements that influence the problem node

  26. Data Dependence Based Test Generation • The data dependence graph is used by the search engine to guide the search • The data dependence based test generation identifies different sequences for exploration in the data-dependence graph leading to the problem statement • The identified sequences are used in the program to guide the search

  27. Data Dependence Based Test Generation Data dependences with respect to variable top 4 11 16 18 Data-dependence graph

  28. Data Dependence Based Test Generation • The data dependence graph is used by the search engine to guide the search • The data dependence based test generation identifies different sequences for exploration in the data-dependence graph leading to the problem statement • The identified sequences are used in the program to guide the search

  29. Data Dependence Based Test Generation 4 11 16 18

  30. Data Dependence Based Test Generation 4 11 16 18 en, 4, 11, 16, 11, 18

  31. Data Dependence Based Test Generation Sample sequences generated from the data dependence graph: P1: en, 4, 18 P2: en, 4, 11, 18 P3: en, 4, 16, 18 P4: en, 4, 11, 16, 18 P5: en, 4, 16, 11, 18 …

  32. Data Dependence Based Test Generation • The data dependence graph is used by the search engine to guide the search • The data dependence based test generation identifies different sequences for exploration in the data-dependence graph leading to the problem statement • The identified sequences are used by the search engine to “execute” (explore) them in the program

  33. Data Dependence Based Test Generation • For some programs, a large number of different sequences can be generated from the data dependence graph for exploration before the solution is found • Many sequences may not lead to the solution • It may be expensive to explore sequences in the original program • The search engine may require a lot of effort to move from one node to another one as specified by the sequences

  34. Testability transformation • The idea is to explore these sequences not in the original program but • in a transformed program in which it should be much easier (faster) to determine whether the fitness function associated with the problem node may evaluate to the target value for a given sequence

  35. Testability transformation input x input x Original program Transformed program Sequence S fitness function F

  36. Testability transformation • The transformed program is used to identify promising sequences • A promising sequence is a sequence for which it is possible to find a program input on which the fitness function at the problem node evaluates to the target value

  37. Testability transformation input x Find input x on which Fitness function F evaluates to the target value during execution of sequence S Transformed program Sequence S fitness function F

  38. Testability transformation • It is inexpensive to identify promising/unpromising sequences in the transformed program • Identified promising sequences are then explored in the original program to find the solution

  39. Testability transformation • A data dependence graph is used to construct a “corresponding (transformed) program”

  40. Testability transformation 4 11 16 18

  41. float TransFunc(int A[], int C[],int PathSize, int S[], int R[]) { int i, j, top; 2 i=1; 3 while (i<=PathSize) { 4 switch (S[i]) { 5 case 4: {top = 0; // 4 6 break; } 7 case 11: {top = top + 1; // 11 8 for (j=1;j<R[i];j++) top = top + 1; 9 break; } 10 case 16: {top = top - 1; // 16 11 for (j=1;j<R[i];j++) top = top - 1; 12 break; } 13 } 14 i++; 15 }; 16 return 100-top; //computation of the fitness function at node 18 }

  42. float TransFunc(int A[], int C[],int PathSize, int S[], int R[]) { int i, j, top; 2 i=1; 3 while (i<=PathSize) { 4 switch (S[i]) { 5 case 4: {top = 0; // 4 6 break; } 7 case 11: {top = top + 1; // 11 8 for (j=1;j<R[i];j++) top = top + 1; 9 break; } 10 case 16: {top = top - 1; // 16 11 for (j=1;j<R[i];j++) top = top - 1; 12 break; } 13 } 14 i++; 15 }; 16 return 100-top; //computation of the fitness function at node 18 }

  43. float TransFunc(int A[], int C[],int PathSize, int S[], int R[]) { int i, j, top; 2 i=1; 3 while (i<=PathSize) { 4 switch (S[i]) { 5 case 4: {top = 0; // 4 6 break; } 7 case 11: {top = top + 1; // 11 8 for (j=1;j<R[i];j++) top = top + 1; 9 break; } 10 case 16: {top = top - 1; // 16 11 for (j=1;j<R[i];j++) top = top - 1; 12 break; } 13 } 14 i++; 15 }; 16 return 100-top; //computation of the fitness function at node 18 }

  44. Testability transformation 4 11 16 18

  45. Testability transformation How many times? 4 11 16 18

  46. float TransFunc(int A[], int C[],int PathSize, int S[], int R[]) { int i, j, top; 2 i=1; 3 while (i<=PathSize) { 4 switch (S[i]) { 5 case 4: {top = 0; // 4 6 break; } 7 case 11: {top = top + 1; // 11 8 for (j=1;j<R[i];j++) top = top + 1; 9 break; } 10 case 16: {top = top - 1; // 16 11 for (j=1;j<R[i];j++) top = top - 1; 12 break; } 13 } 14 i++; 15 }; 16 return 100-top; //computation of the fitness function at node 18 }

  47. Testability transformation Find input A[], C[], and R[] on which F < 0 during execution of sequence S A[] C[] R[] Transformed program PathSize Sequence S F

  48. Testability transformation 4 11 16 18 en, 4, 11*, 18

  49. S = R = A = C = 4 ? ? ? ? ? 11 ? Testability transformation Given: PathSize = 2 Find: Such that F < 0

  50. R = A = C = 1 - - 101 - - Testability transformation Solution:

More Related