Character String Predicate Based Automatic Software Test Data Generation

1. Character String Predicate Based Automatic Software Test Data Generation Michael R. Lyu Computer Science Dept. Chinese University of Hong Kong Lyu@cse.cuhk.edu.hk Ruilian Zhao Computer Science Dept. Beijing University of Chemical Technology Rlzhao@mail.buct.edu.cn

2. Outline Introduction An overview of related work Test data generation based on character string predicate Experimental results Conclusion

3. Introduction Software testing is usually difficult, expensive and time consuming. If test data could be automatically generated, the cost of software testing would be significantly reduced.

4. Introduction There are many automatic test data generation approaches. But, little attention has been paid to the problem of test data generation for programs whose predicates can contain character string variables.

5. Introduction Character string is an important element in programming. So,how to generate test data of character string is a problem that needs further research. Here, we present an approach to automatically generate test data for program paths that include character string predicates, and a corresponding test data generator is developed.

6. Introduction The effectiveness of test data generator is examined on a number of programs. The experimental results illustrate that the test data generator is effective.

7. An overview of related work 1. Predicate-based testing Predicate testing is a common approach to software testing, which requests each predicate in the program under test to be checked. There are a lot of predicate testing strategies. However, they demand that predicates in tested programs must be numerical predicates.

8. Random test data generation Symbolic execution-based test data generation Dynamic test data generation An overview of related work 2. Test data generation There are many automatic test data generation approaches. For example,

9. An overview of related work Some systems are developed by using testing techniques to generate test data of integer, real or float types. However, they do not generate test data of character string.

10. Test data generation based on character string predicate The goal of test data generation is to find a program input on which a chosen program path will be traversed. This problem can be reduced to a sequence of subgoals where each subgoal is solved by performing function minimization using gradient descent.

11. Test data generation based on character string predicate We focus on how to automatically generate test data for program paths that include character string predicates. A character string predicate is the predicate that consists of at least one character string variable and one character string comparison function.

12. Test data generation based on character string predicate Similarly to the numerical predicate, we can construct a branch function with regard to a character string predicate, which is not take the requirement branch, so that its value is positive for initial input x0. For example, strcmp(str1,str2) > 0 Let (x)=str1-str2 , if str1 - str2 is positive for initial input x0, otherwise (x)=str2-str1 . The current values of str1 and str2in this predicatecan be calculated by using program instrumentation technique.

13. Test data generation based on character string predicate The program input is adjusted gradually until (x) becomes negative. The required inputs have been found, namely, the predicate takes the requirement branch. A problem that we must resolve is how to compare two character strings as well as how to evaluate the branch function (x) .

14. where str is a character string, L is its length, is a positive weighting factor representing a weighted value imposed upon each character element of the string, and w is equal to 128. Test data generation based on character string predicate So, we first define a function ع

15. Theorem: Suppose S is a set of character strings, is a set of nonnegative integers. Let is defined as above. Then is a one-to-one function from S to . Test data generation based on character string predicate By the theorem, a character string can be transformed into a unique nonnegative integer.

16. Where L1 and L2 are the length of string str1, str2, L=max(L1,L2). Test data generation based on character string predicate The distance between two strings can be defined as below: The distance dis(str1,str2) determines a nonnegative integer, and can be used to evaluate the branch function (x) with regard to a character string predicate.

17. It is easy to see that by the verification of above theorem > Test data generation based on character string predicate We search an appropriate adjustment direction for the 0th character of an input variable, and adjust the character by gradient descent until 0<0. As a result, we can find an input that makes the string predicate to take the requirement branch.

18. Test data generation based on character string predicate For an equality (=) or non-equality (≠) predicate , we need to construct branch functions for every unequal character such that i >0, where i[0,L], L=max(L1,L2) Then, we search an adjustment direction to improve the branch function until i 0.

19. Experimental results Intmax(int argc,char ** argv) { argc--; argv++; if((argc>0)&&('-'==**argv)) {if (!strcmp(argv[0],"-ceiling")) {strncpy(ceiling,argv[1],BUFSIZE); argv++; argv++; argc--;argc--; } else { fprintf(stderr,"Illegal option %s.\n",argv[0]); return(2); } } if(argc==0) { fprintf(stderr,"Max requires at least one argument.\n"); return(2); } for(;argc>0;argc--,argv++) { if(strcmp(argv[0],result)>0); strncpy(result,argv[0],BUFSIZE); } if (strcmp(ceiling,result)<=0) printf("\n max:%s",ceiling); else printf("\n max:%s",result); return(0); } The specification: Which prints the lexicographic maximum of command-line arguments. There is one option:-ceiling This provides a ceiling: If the maximum would be larger than this specified ceiling, it is the maximum.

20. Experimental results Intmax(int argc,char ** argv) { argc--; argv++; if((argc>0)&&('-'==**argv)) { if (!strcmp(argv[0],"-ceiling")) {strncpy(ceiling,argv[1],BUFSIZE); argv++; argv++; argc--;argc--; } else { fprintf(stderr,"Illegal option %s.\n",argv[0]); return(2); } } if(argc==0) { fprintf(stderr,"Max requires at least one argument.\n"); return(2); } for(;argc>0;argc--,argv++) { if(strcmp(argv[0],result)>0); strncpy(result,argv[0],BUFSIZE); } if (strcmp(ceiling,result)<=0) printf("\n max:%s",ceiling); else printf("\n max:%s",result); return(0); } record (argc,0,'>',"&&"); record('-',**argv, '='); if ((argc>0)&&('-'==**argv)) { record(argv[0],"-ceiling", '!'); if (!strcmp(argv[0],"-ceiling")) …; } record(argc,0,'=',""); if(argc==0) …; record(argc,0,'>',""); for(;argc>0;argc--,argv++) { record(argv[0],result, '>', ""); if (strcmp(argv[0],result)>0) …; record(argc,0,'>',""); } record(ceiling,result, '-', ""); if (strcmp(ceiling,result)<=0) …;

21. Experimental results Considering that the FOR loop is executed 0 time, 1 time and 2 times, there are 31 pathsin Max program. We design 50 program inputs at random, which are used as the original input to the test data generator. As a result, 16 test inputs are generated by the test data generator.

22. Experimental results We measure the coverage of generated test data using the ATAC coverage testing tool.

23. Experimental results Compare the evaluation number of branch function in the gradient descent, the gradual descent and the random-number test data generator under the same coverage. The gradient descent test data generator is more economical than the gradual descent and the random generator.

24. To our knowledge, this is the first automatic test data generation approach based on character string predicates. The preliminary experimental results show that the methodology is effective. Conclusion