300 likes | 473 Views
Event Driven Automated Testing for GUI-based Java Programs. Presented by Yanhong Sun Thesis Advisor: Dr. Edward L. Jones July 17, 2003. Outline. Problem Statement Background Motivation of Research Jemmy Module AGUIJ Design Constructions of AGUIJ Design AGUIJ Prototype
E N D
Event Driven Automated Testing for GUI-based Java Programs Presented by Yanhong Sun Thesis Advisor: Dr. Edward L. Jones July 17, 2003
Outline • Problem Statement • Background • Motivation of Research • Jemmy Module • AGUIJ Design • Constructions of AGUIJ Design • AGUIJ Prototype • Test Specification language • Jemmy Test Engine Generator (JTEG) • Demonstration • Limitations of AGUIJ • The future works • References
Event Driven Automated Testing for GUI-based Java Programs • Problem Statement • The amount of testing Java program increases • Testing GUI-based Java programs by hand is a redundant, boring and time intensive work. • AGUIJ --- Automated grader for GUI-based Java programs
Event Driven Automated Testing for GUI-based Java Programs • Background • All programs have interfaces • Text interface • Graphical user interface • The applications with text interfaces are driven by data • Automated grader uses I/O redirection • Julia and I have been working on this for one year
Event Driven Automated Testing for GUI-based Java Programs • GUIs are event-driven • The type of input is more varied button clicks, mouse movements, etc. • It is not so easy to reassign the input stream with the help of I/O redirection • Special tool is needed to simulate the input and capture the output
Event Driven Automated Testing for GUI-based Java Programs • Motivation of Research • Testing life cycle
Event Driven Automated Testing for GUI-based Java Programs • Testing life cycle Figure 1 Testing life cycle Specification Analysis Test Strategy/ Plan Design Test Cases Implementation Test Script, Data, Driver Execution Test Results Problem Report Evaluation
Event Driven Automated Testing for GUI-based Java Programs • Motivation of Research • Testing life cycle? • What activities the automated GUI testing process tends to apply to?
Event Driven Automated Testing for GUI-based Java Programs • What activities the automated GUI testing tends to apply to? • Test case generation • Test execution • Test verification (Test oracle)
Event Driven Automated Testing for GUI-based Java Programs • Motivation of Research • Testing life cycle? • What activities the automated testing process tends to apply to? • What tools are available for automated GUI testing?
Event Driven Automated Testing for GUI-based Java Programs • What tools are available for automated GUI testing? • Almost all of the testing tools for GUI-based applications are based on capture/replay technique (see Figure 2) • There exist various commercial capture/replay tools XRunner [10] JavaStar [11] Marathon [12]
Software under test Capture/Replay tool replay interactions record script editing Tester Test script Event Driven Automated Testing for GUI-based Java Programs Figure 2 Standard commercial capture/replay tool
Event Driven Automated Testing for GUI-based Java Programs • The shortcomings of using capture/replay tool • Needs external oracle support. • Manual testing is necessary for capturing events sequence. • May not be able to replay until the tester has manually tested a perfect program
Event Driven Automated Testing for GUI-based Java Programs • Jemmy module • Jemmy is a JavaTM library • Can perform test execution & test verification in one program --- no external oracle support • No need for recorder --- no any manual testing, start automated test right away
class file Event Driven Automated Testing for GUI-based Java Programs • AGUIJ Design Specification Specification test specification test specification generate generate Jemmy test Jemmy Jemmy Test Engine Test Engine engine Generator (JTEG) Generator (JTEG) (Java) class file(s) Compile Compile Jemmy Jemmy test engine test engine Student Programs Student Programs (class) (class) compile Not cmpile Not compile run run Grading Log files
Event Driven Automated Testing for GUI-based Java Programs • Constructions of AGUIJ design • The initial AGUIJ implementation • Test Specification language design. • Jemmy Test Engine Generator (JTEG) design
Event Driven Automated Testing for GUI-based Java Programs • The initial AGUIJ Figure 3 The initial AGUIJ Jemmy test engine source file (Java) Grading plan Specification Specification Jemmy test engine (Java) class class file(s file(s ) ) compile compile Compile Compile Jemmy test engine Jemmy test engine Student Programs Student Programs (class) (class) run Not compile Grading Log files
Event Driven Automated Testing for GUI-based Java Programs • Case Study Table 1 Time consumed in grading (total 66 students) • The ratio of consistency: 92.4%
Event Driven Automated Testing for GUI-based Java Programs • Limitations of the initial AGUIJ --- To implement Jemmy test engine source file is not a fully automated step • The tester needs to be familiar with Jemmy API --- Lower productivity • To implement Jemmy test engine source file start-to-finish for each program assignment • Have similar structure
Event Driven Automated Testing for GUI-based Java Programs • Test Specification language design • The criteria must be met • Easy to write • Easy to understand • Amenable to automated generation of the Jemmy test engine (JTE) source file
<class> climate </class> <component> type = JTextField; order = 1 type= JTextField; order = 2 </component> <testcases> number = 1 <action> type = JTextField; order = 1; name = "state"; action = type “Florida\n" </action> <expect> type = JTextField; order = 2; name = "temperature"; content = equals "Warm"; deduction = 25 </expect> number = 2 <action> type = JTextField; order = 1; name = "state"; action = type “Alaska\n" </action> <expect> type = JTextField; order = 2; name = "temperature"; content = equals “Cold"; deduction = 25 </expect> </testcases> <done>
Event Driven Automated Testing for GUI-based Java Programs • JTEG Design Figure 4 The function of JTEG read generate Test Specification Jemmy Test Engine(JTE) source file JTEG
Event Driven Automated Testing for GUI-based Java Programs • Demonstration
Event Driven Automated Testing for GUI-based Java Programs • Limitations of AGUIJ • Only support Java GUI with one single frame • Access most Swing components, not all of them • The syntax check of the test specification is not powerful enough
Event Driven Automated Testing for GUI-based Java Programs • Future Works • Extending AGUIJ to support multiple frames • Naming problem • Application state --- hierarchical relationship --- parallel relationship • Using XML to describe test cases in test specification • Generating test specification automatically
Event Driven Automated Testing for GUI-based Java Programs • References [1] Jemmy Module. See http://jemmy.netbeans.org [2] Alan Walworth. Java GUI Testing. Dr. Dobb’s Journal February 1997 [3] Tessella Support Services PLC. Automated GUI Testing. Tessella Scientific software solutions January 1999, issue V1.R1.M1 [4] J. Skrivanek and A. Sotona. Tesing ForteTM for JAVATM [5] Cay S. Horstmann, Gary Cornell. The core of JAVA, second edition. [6] A. M. Memon, M. L. Soffa, and M. E. Pollack. Coverage Criteria for GUI Testing. [7] J. Chen, and S. Subramaniam. A GUI Environment to Manipulate FSMs for Testing GUI-Based Applications in Java. In proceedings of the 34th Hawaii international Conference on System Sciences, 2001
Event Driven Automated Testing for GUI-based Java Programs [8] A. M. Memon, M. E. Pollack, and M. L. Soffa. Hierarchical GUI test case generation using automated planning. IEEE Transactions on Software Engineering, 27(2): 144-155, Feb.2001 [9] T. Ostrand, A. Aanodide, H. Foster, and T. Goradia. A Visual Test Development environment for GUI Systems. In proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis(ISSTA-98), pages 82-92, New York, Mar.2-5 1998. ACM Press. [10] XRunner. See http://www.merc-int.com/products /xrunner6 [11] JavaStar. See http://www.sun.com/suntest/products/index.html [12] Marathon. See http://marathonman.sourceforge.net [13] J. Takahashi. An Automated Oracle for Verifying GUI Objects. Software Engineering Notes, vol. 26 no 4: 83-88, July 2001 [14] Fewster, Mark. Software Test Automation, Addison Wesley, New York, 1999 [15] XMLSPY 5. See http://www.altova.com/download_spy_enterprise.html
Event Driven Automated Testing for GUI-based Java Programs • Acknowledgement • CIS department • CREST • Dr. Edward Jones • Dr. Clement Allen • Dr. YiLi Zeng • Dr. Roselyn Williams • Mr. Aubrey Rembert • Mr. Hongtao Chen