Automated Test-Input Generation

1. Automated Test-Input Generation Frank Xu, Ph.D. Gannon University • Xu, W., Ding, T., Wang, H., Xu. D., Mining Test Oracles for Test Inputs Generated from Java Bytecode, Proc. of the 37th Annual International Computer Software & Applications Conference, pp. 27-32, Kyoto, Japan, July 2013 • Mining Decision Trees as Test Oracles for Java Bytecode (Extended version of conference paper), Accepted by Journal of Systems and Software

2. About Me – Frank Xu • Education • Ph.D. in Software Engineering, North Dakota State University • M.S. in Computer Science, Towson University • B.S. in Computer Science, Minor in Math, Southeast Missouri State University • Working Experience • GE Transportation, 2008- present, Consultant • Gannon University, 2008- present, Assistant Professor of Software Engineering, Director of Keystone Software Development Institute • University VA –Wise, 2007- 2008, Assistant Professor of Software Engineering • Swanson Health Products, 2005 ~ 2007, Sr. Web Programmer Analyst • Volt Information Science Inc., 2004 ~ 2005, Software Engineer (Web)

3. Teaching • Source: Student Evaluation Report

4. Research • Source: Google scholar: http://scholar.google.com/citations?user=9_I4ZUgAAAAJ&hl=en

5. Automated Test-Input Generation • Introduction • Software testing • Test automation • Test Inputs • How to Generate Test Inputs • Simplifying Java Code • Applying rules • Empirical Study/Demo • Conclusions

6. Introduction

7. Exercise • Implementing a method to solve Triangle problem

8. What is Triangle Problem?

9. What is Method? • Is a function or a service to complete a task • A method that determines the maximum of two numbers. • A method that sorts a list of names • A method that opens a file from the file system • Method • Invoked by a method call • Returns a result to calling method (caller) • Similar to a boss (caller) asking a worker (called method) to complete a task

10. y is the parameter of method square Method square returns the square of y Method square returns int that result stores 1 // Fig. 6.3: SquareIntegers.java 2 // Creating and using a programmer-defined method. 3 public class SquareIntegers { 4 public static void main (String args[]) 5 { 6 int result; // store result of call to method square 7 // loop 10 times 8 for ( int counter = 1; counter <= 10; counter++ ) { 9 result = square( counter ); // method call 10 // print the result of one call to the method 11 System.out.println ("The square of " + counter + " is " + 12 result ); 13 } // end for 14 } // end method main() 15 // square method declaration 16 public static int square( int y ) 17 { 18 return y * y; // return square of y 19 } // end method square 20 } // end class SquareIntegers  2003 Prentice Hall, Inc.

11. How to test Triangle? • String getTriangleType (int a, int b, intc){ • if((a<b+c) && (b<a+c) && (c<a+b)){ • if (a==b && b==c) • return“Equilateral ”; • elseif (a!=b && a!=c &&b!=c) • return“Scalene ”; • else • return “Isosceles” ; • } • else • return“NotATriangle “; • }

12. if ... elseControl Flow • if (a==b && b==c) • return“Equilateral ”; • elseif (a!=b && a!=c &&b!=c) • return “Scalene ”; • else • return “Isosceles” ;

13. Control Flow Diagram

14. Summary: Test Triangle Steps assertEquals(“Isosceles ”, getTriangleType(7,7,9)) Step 3 Step 1 Step 2 assertEquals(“Isosceles ”, getTriangleType(6,6,8)) ….. Source Code Control Flow Diagram Paths (based on coverage) Junit Test cases

15. How to Auto-Generate Testing Inputs? assertEquals(“Isosceles ”, getTriangleType(7,7,9) ? assertEquals(“Isosceles ”, getTriangleType(6,6,8) …..

16. Solution • Randomly generate inputs • (5,7,6) for Isosceles • Adjust input values • a=5, b=7, a==b

17. Revisit: Triangle Problem

18. Java is Complex • Statement • contains comparison and expression • a <b+c(Java) • Condition • (a<b+c) && (b<a+c) && (c<a+b)

19. Java Simpler Version • Simplify Statement • a <b+c (Java) • [1]$i3=i1+i2 and[2]i0>=$i3(Jimple) • Simplify condition • (a<b+c) && (b<a+c) && (c<a+b) (Java) • Jimple if (a<b+c) { if (b<a+c) { if(c<a+b) … }}} • www.sable.mcgill.ca/soot/

20. Path generation Generate CFG Generate inputs

21. How to Generate Test Inputs

22. Path: [1]→[2]→[3]→[4][5]>[18] • Search an input that make predicate [5]:i0>=$i3 to true • a>=b+c (NotATriangle) • Challenge: backtracking $i3 to input variables • Recall $i3=i1+i2 • Solution:

23. Apply Rules for Generating Test Inputs 10 7 4

25. Empirical Study

26. Three Study Subjects

27. Goal of Empirical Studies • What is the performance of the proposed approach?

29. Demo • http://perceval.gannon.edu/xu001/research/GannonJVM/ • Path 1: Equilateral • Path 5, 7, 8: Isosceles • Path 6: Scalene • Path 10, 11, 12: not a triangle

30. Key Points • Understand requirements before implementation • Test your code • Auto test your code if possible • Auto generate test inputs

31. Thanks

32. Future Research Directions • Requirements Engineering & Natural language Process • Generating UML diagrams, e.g., Use case, Class diagram • Validating SRS • Deriving test cases from SRS • Software Design & Social Networks Analysis • Utilizing SSA for analyzing communication diagram, class diagram, and sequence diagram for improving the quality of the software • Software Implementation & Big Data • Mining repository for software quality assurance using Hadoop • Software Testing & Mobile/Cloud Application • Testing mobile applications and distributed applications