Testing Cactus and JUnit By Bill Dudney and Jonathan Lehr

1. Neal O’Brien 10/27/04 Testing Cactus and JUnitBy Bill Dudney and Jonathan Lehr

2. Table of Contents Overview Article Outline Thesis JUnit Background Cactus Background Pitfalls 1-6 Comparable Products Conclusion

3. Overview Junit and Cactus are popular tools for automating testing of Java classes and web-based components Automation accomplished using ANT (Java Make utility) and often tested every time project built These tools enable testers to “verify” the code is written correctly – it was built right Automated tests are very useful to show code works correctly esp. after refactoring which is central to XP I picked this article because deals with JUnit & Cactus, offers a critique of the tools, and was practical, not just theoretical As this article describes, the tests themselves must be built right in order to validate the code being tested

4. Article Outline Theme: Pitfalls when testing with JUnit & Cactus Pitfall 1: No assert Pitfall 2: Unreasonable assert Pitfall 3: Console-Based Testing Pitfall 4: Unfocused Test Method Pitfall 5: Failure To Isolate Each Test Pitfall 6: Failure to Isolate Subject

5. Thesis – Analysis of article Thesis: critique was correct but wordy, repetive, and incomplete – missing some bigger pitfalls Pitfalls 1-3 are all the same – use assert correctly Pitfall 4 is programming rule – write focused method Pitfall 5 is JUnit rule – Use setUp and tearDown Pitfall 6 is incomplete analysis of Cactus vs MockObj Overview of Junit and Cactus, then discuss pitfalls, and finally look at unmentioned pitfalls & issues

6. JUnit Overview Popular and simple Java framework / library for automating testing Integrates well with ANT – Java Make utility General idea: write one test class per testee Write one method to verify each main feature Test class must extend TestCase and each test method must start with “test” Order of test method execution varies Use assertTrue() and assertEquals() to verify code Use setUp() & tearDown() prepare testcase testfixture

7. JUnit code example • Following test cases test the collection methods, isEmpty() and add() • import junit.framework.*; • public class SimpleTest extends TestCase { • private java.uti.Collection collection; • protected void setUp() { collection = new ArrayList(); } // instantiates collection test fixture • protected void tearDown() { collection.clear(); } • public void testEmptyCollection() { • assertTrue(collection.isEmpty()); • } • public void testOneItemCollection() { • collection.add("itemA"); • assertEquals(1, collection.size()); • } • }

8. Cactus Overview Built on Junit framework Intended to test JSP, Servlets, EJBs, Filters, and custom tags Complex architecture that has client JVM call the J2EE application server JVM via redirector Testcase classes must reside on client and server Adds two methods to Junit architecture, beginXX() and endXX() which get called on client, rest on server

9. Cactus System Diagram

10. Cactus Sequence Diagram – Jakarta website

11. Pitfalls 1-3 – No assert, Unreasonable assert, Console-Based Testing Pitfalls 1-3 are all the same – use assert correctly Very important and author points out a major guideline test what is written in the javadocs for the testee implies javadocs must be up-to-date with requirements Author also points out to write a test case if encounter a defect before it is corrected However, using assertTrue() and assertEquals() is obvious These are the prominent features of the JUnit

12. Pitfall 4 – Unfocused Test Methods Writing focused tests is really just writing good code General rule of programming to make methods succinct, this applies equally to test methods Writing focused test methods is the whole point

13. Pitfall 5 – Failure To Isolate Each Test Pitfall 5 is really saying to use setUp() and tearDown() to prepare/release test fixture, an obvious suggestion – example from JUnit site Bigger pitfall is automating creation of the test fixture in distributed environments import junit.framework.*; public class SimpleTest extends TestCase { private java.uti.Collection collection; protected void setUp() { collection = new ArrayList(); } // instantiates collection test fixture for 2 tests protected void tearDown() { collection.clear(); } public void testEmptyCollection() { assertTrue(collection.isEmpty()); } public void testOneItemCollection() { collection.add("itemA"); assertEquals(1, collection.size()); } }

14. Pitfall 6 – Failure to Isolate Subject Author points out one drawback of Cactus is that does not isolate test case as MockObjects does MockObjects simulates the Servlet container Mock Object framework does isolate test but at big expense Massive amount of stubs needed, more code to maintain While Cactus may be better than MockObjects, it may NOT be better than HttpUnit, why not compare these?

15. My pitfalls Does not mention major issues like creating this complex automated testing API is itself a software project and has high upfront resource commitments It will slow the development process Upkeep of test code is costly Tests must be documented, reside in source control, and be included in the project planning Junit testcases must reside in same package or parrallel package to access private/package methods – complicating code structure Testers often analysts, not programmers

16. My Pitfalls – continued Cactus can only test J2EE web objects - JSP/EJB/etc, cannot test ASP.NET, PHP, etc. Goal of Cactus not clear – Web or Unit tests? JUnit is better for unit testing business objects Cactus is unnecessarily complex mucky interface: test code must reside on two JVMs Should call via web server like user does, else white box ** Many business websites may have distributed nature of data/objects to be tested, it may be very difficult to setUp the test cases Possibly data to be tested is on server not owned/managed by the testing team, data could be volatile

17. Alternative tools HttpUnit cleaner, simpler tool than Cactus HttpUnit is black box testing by calling webserver Test code resides ONLY on client JVM Various interfaces like JWebUnit (Java API) and WebTest (XML) integrate well with ANT Use Junit for unit tests and HttpUnit for functional Features to analyze HTML, ie. table element tests Features to input HTML form elements http://www.junit.org/news/extension/index.htm

18. JWebUnit example – JWebUnit.sourceforge.net • // The tests perform a google search for the HttpUnit home page, navigate to that page from Google, • // and validate that there is a link to the user manual on the HttpUnit home page. • package net.sourceforge.jwebunit.sample; • import net.sourceforge.jwebunit.WebTestCase; • public class JWebUnitSearchExample extends WebTestCase { • public JWebUnitSearchExample(String name) { super(name); } • public void setUp() { getTestContext().setBaseUrl("http://www.google.com"); } • public void testSearch() { • beginAt("/"); • setFormElement("q", "httpunit"); • submit("btnG"); • clickLinkWithText("HttpUnit"); • assertTitleEquals("HttpUnit"); • assertLinkPresentWithText("User's Manual"); • } • }

19. Webtest– XML interface to HttpUnit, easy to automate test by inlining with ANT, verifies logon • This XML file can easily be called from ANT • Generic interfaces like XMLSPY can be used by non programmers to create the XML test files <target name="login" >  <testSpec name="normal" >      <steps>       <invoke         stepid="get Login Page"          url="login.jsp"   />       <verifytitle    stepid="we should see the login title"    text="Login Page" />       <setinputfield stepid="set user name" name="username“ value="scott" />       <setinputfield stepid="set password"  name="password"  value="tiger"  />       <clickbutton   stepid="Click the submit button"  label="let me in" />       <verifytitle    stepid="Home Page follows if login ok"  text="Home Page" />    </steps></testSpec> </target>

20. Conclusion Article lists some useful guidelines & pitfalls in an wordy fashion Many pitfalls were obvious and important ones not mentioned Important pitfalls not mentioned include Cost, complexity, difficulty of distributed tests not mentioned Performs white box tests, yet, JUnit already does this Does not test HTTP interface (tests presentation layer poorly) Test code must reside in same package as testee & both JVMs Testers must be programmers JWebUnit & WebTest better for web unit testing At times unclear when addressing Junit vs. Cactus and unnecessarily complex coding examples However, automating testing can save time and money in the long run These tools, while not perfect, are major players for automated Java testing and can verify functionality during development and refactoring