A Novel Approach to Unit Test: The Aspect-Oriented Way

1. A Novel Approach to Unit Test: The Aspect-Oriented Way Guoqing Xu and Zongyuan Yang Software Engineering Lab (SEL) East China Normal University http://www.cs.ecnu.edu.cn/sel/~harryxu Oct.21 2004 at ISFST

2. Background • Test Oracle problem -- how to identify oracles in unit test? • Automating the test oracle generation -- Manually writing test codes is a labor-intensive job. -- Cheon and Leavens automatically generate JUnit test codes and use JML runtime assertions as test oracle to test the Java programs annotated with JML specifications [CL02]. • Automating the test case generation -- Korat generates test cases based on java predicts. [BKM02] -- JMLAutoTest generates test cases based on JML class invariants specified in the program. [XY03]

3. Problems • Current test automating process relies on formal assertions and predicts. • Traditional specifications only focus on programs’ functional behaviors, with little support for specifying their non-functional behaviors, e.g. temporal logic, performance... • How to find oracles to test non-functional aspects? How to automate this process?

4. Our approach • In Aspect-Oriented Programming (AOP), crosscutting properties are monitored to model the program from different aspects, a lot of tasks which have been difficult to be handled in traditional ways are easily done. • Using a crosscutting property of the program as the criterion to check the correctness of the application in the corresponding aspect is well suited to the unit testing problems.

5. Application-specific Aspect • A new notion: application-specific aspects (ASS) -- top-level application related aspects. -- established at the design level. -- may be picked up from low language level aspects. -- all the ASS for the same use share some common features, e.g. testing ASS, tracing ASS... -- may be translated into language level aspects.

6. Aspect-Oriented Test Description Language • How to describe ASS? -- a formal way is needed. -- can not be too complicated. • Aspect-Oriented Test Description Language(AOTDL) -- uses the first order predict logic as part of its syntax -- used by the designer at design level -- can be translated into AspectJ aspects. • Basic units in AOTDL -- Utility unit -- MeaninglessCase Advice unit -- Error Advice unit advicetype (arguments): pointcuts:conditions: message

7. AOTDL (cond.) TestingAspect tempLogic{ Utility{ protected boolean isInitialized = false; //push is reached pointcut pushReached(Stack st): target(st)&&call(void Stack.push(Integer)); //init is reached pointcut initReached(Stack st): target(st)&&call(void Stack.init(void)); // after advice after(Stack st):initReached (st){ isInitialized = true; } } Class Stack{ public void init(){...} public void push ( Node n){...} ... }

8. AOTDL (cond.) Error Advice{ /*advices for specifying criteria of test errors */ before(Stack s): pushReached(s): ! isInitialized : ”Not Initialized” ; ... } } MeaninglessCase Advice{ /*advices for specifying criteria of meaningless test cases */ before(Stack s) : pushReached(s) : s.getSize() >=MAX : ”Overflow”; ... }

9. AOTDL (cond.) //error advices before(Stack s) throws TestErrorException: pushReached(s){ if (!isInitialized){ TestErrorException ex =new TestErrorException(“Not Initialized”); ex.setSource(“TempLogic”); throw ex; } }

10. JAOUT: Automated Generation of AO Testing Framework

11. JAOUT: Automated Generation of AO Testing Framework • JAOUT takes Java class M.java as the input, and automatically generate JUnit test framework. -- Aspect_M_Test.java, the JUnit unit test class. -- Aspect_M_TestCase.java, the test case provider. -- Aspect_M_TestClient.java, JMLAutoTest test case generation class.

12. Test Suite Definition • For class C and its method M(A1 a1, A2 a2…An an), the generated test suite is defined as --- C[ ] receivers -- A1[ ] vA1; ... ; An[ ] vAn; • There is a corresponding init_Ai method for each type Ai and a method init_receiver in test case provider to initialize test cases. • Testers use APIs provided by JMLAutoTest to generate test case space in test client, and pass it to the test case provider.

13. Generated Test Method catch (TestErrorException e) { String msg = e.getSource(); fail(msg + NEW LINE + e.getMessage()); } catch (java.lang.Throwable e) { continue; } finally { setUp(); // restore test cases } } } public void testM(){ for (int i0 = 0; io < receiver.length; i0++){ for (int i1 = 0; i1 < a1.length; i1++){ … try { receiver[i0].M(a1[i1], : : :, an[in]); } catch (MeaninglessCaseException e) { /* ... tell framework test case was meaningless ... */ continue; }

14. Test Result • ...in push • false • F • Time: 0.06 • There was 1 failure: • 1) testPush (sample.Stack_Aspect_TestCase)junit.framework.AssertionFailedError: In Testing Aspect TempLogic: Not Initilized! • at ample.Stack_Aspect_Test.testPush • (Stack_Aspect_Test.java:155) • at sun.reflect.NativeMethodAccessorImpl.invoke0 • (Native Method) • at sun.reflect.NativeMethodAccessorImpl.invoke • (NativeMethodAccessorImpl.java:39) • at sun.reflect.DelegatingMethodAccessorImpl.invoke • (DelegatingMethodAccessorImpl.java:25) • at sample.Stack_Aspect_Test.run(Stack_Aspect_Test.java:26) • at sample.Stack_Aspect_TestCase.main • (Stack_Aspect_TestCase.java:24) • FAILURES!!! • Tests run: 3, Failures: 1, Errors: 0, Meaningless:0

15. Related Work (Spec-based test) • TestEra – Automating OO test generation. [MK01] MIT • JMLUnit – Generating test oracles from JML runtime assertions. [CL02] Iowa State Univ. • Korat – Generating test case based on Java predicts. [BKM02] MIT. • JMLAutoTest – Generating test framework from JML runtime assertions and test cases based on class invariants. [XY03] ECNU. • Jov -- java automated unit test based on inferred program properties. [XN03] Univ. of Washington.

16. Conclusions • Traditional formal predicts tend not to deal with non-functional properties of the program. • AOP is well suited to the unit test problems. • Designers use AOTDL to build Application-Specific Testing Aspects. • JAOUT translates Testing Aspects to AspectJ aspects automatically. • JAOUT automatically generates JUnit test framework and uses the runtime messages thrown by Testing Aspects as test oracles.

17. Thank you… Questions?