CS5123 Lecture 3 Unit Testing

1. CS5123 Lecture 3Unit Testing

2. Review: Test overview Test is the practical choice: the best affordable approach Concepts: test case, test oracle, test suite, test driver, test script, test coverage Granularity: unit, integration, system, acceptance Type by design principle: black-box, white-box Black-box-testing: boundary, equivalence, decision table White-box-testing: branch coverage, complexity 2

3. Unit testing Testing of an basic module of the software A function, a class, a component Typical problems revealed Local data structures Algorithms Boundary conditions Error handling 3

4. Unit test framework xUnit Created by Kent Beck in 1989 This is the same guy in XP, design patterns The first one was sUnit (for smalltalk) Junit The most popular xUnit framework There are about 70 xUnit frameworks for corresponding languages Never in the annals of software engineering was so much owed by so many to so few lines of code -------Martin Fowler 4

5. An example: writing test cases without a test framework Class Definition Consider a class that handles and manages a Fibonacci array Initialized the class Fibonacci f = new Fibonacci (); Extend to a certain length f. extend (length) Provide the number at certain index f. get(index) Get Length f.getLength() Provide a range of numbers f. getRange(start, end) 5

6. An example: writing automatic test cases without a test framework • public static void main(String args[]){ • Fibonacci f = new Fibonacci(); • if(f.getLenght != 0){ • System.err.print(“wrong results …”); • } • f.extend(50); • if(f.getLenght != 50){ • System.err.print(“wrong results …”); • } • int num = f.get(50); • if(num != 12586269025){ • System.err.print(“wrong results …”); • } • int[] range = f.getRange(40, 50) • if(!ValueEquals(range, …)){ • System.err.print(“wrong results …”); • } • … • } Not Enough! What happened if an exception is thrown 6

7. An example: writing automatic test cases without a test framework • public static void main(String args[]){ • Fibonacci f = new Fibonacci(); • if(f.getLenght != 0){ • System.err.print(“wrong results …”); • } • f.extend(50); • if(f.getLenght != 0){ • System.err.print(“wrong results …”); • } • int num = f.get(50); • if(num != 12586269025){ • System.err.print(“wrong results …”); • } • int[] range = f.getRange(40, 50) • if(!ValueEquals(range, …)){ • System.err.print(“wrong results …”); • } • … • } Potential test interference! Consider if the get method does an extension when length is smaller than index: while in getRange, such extension is forgotten to be added 7

8. Common Operations Comparison between actual results and expected results Report error messages Catch Exceptions Re-initialize the class to be tested 8

9. Unit Test Framework System under test: the system/module/component we are testing Test Fixture: SUT + DOC Test Method: The actual code of the test Test Case: A collection of tests with common purpose/setup 9

10. Writing a Test Case public class FibonacciTest extends TestCase { protected Fibonacci fTest; protected static int groudtruth[] = {0, 1,...,12586269025}; public void setUp(){ fTest = new Fibonacci(); } @Test public testInitialize(){ assertEquals(“Length of fTest after initialization”, 0, fTest.getLength()); } @Test public testExtend(){ fTest.extend(50); assertEquals(“…”, 50, fTest.getLength()); } @Test public testGet(){ fTest.extend(50); assertEquals(“…”, groudtruth[49], fTest.get(50)); } … } 10

11. State machine of JUnit try{ testMethod } catch {…} Initialization Of Test Class setUp tearDown 11

12. Benefits of using test framework Clear structure of testing Each test method for one feature Common initialization to setup Assertion methods You can always define new assertion methods Reduced Interference Try catch to run all test methods Always do re-initialization in setup 12

13. Tips of writing test cases Bottom Up If you know that obj.m1 calls obj.m2, test obj.m2 first Why? You may not understand the error in m1 if you don’t know whether m2 is working or not Externalize data Externalize expected results to files and load them at the test case constructor Why? Easier to maintain if you want to reuse them in several places 13

14. Tips of writing test cases Use development Database Do not use real database for testing Why? Obvious Do enough logging in the code during testing Why? Help debugging Testing for exceptions How? @Test(expected = xxxException.class) public testException(){ do(invalid) } • public testException(){ • try{ • do(invalid); fail(); • }catch(xxxException e){} • } 14

15. Writing Assertions public void testCapacity() { // a test method …. assertTrue(fFull.size() == 100+size); //assertion } If assertion fails: Assertion failed: myTest.java:150 (expected true but was false) Not so good! Try: assertEquals(100+size, fFull.size()); //expected value first Assertion failed: myTest.java:150 (expected 102 but was 103) Better! Try: assertEquals(“list length”, 100+size, fFull.size()); Assertion failed: myTest.java:150 (list length expected 102 but was 103) 15

16. Assertions Extend TestCase and write your own assertions AssertStringContains, AssertArrayEquals, … Use fail(String) to fail a test with certain message Feel free to change the fixture Each test will re-setup public void testRemoveAll() { fFull.removeAllElements(); fEmpty.removeAllElements(); assertTrue(fFull.isEmpty()); assertTrue(fEmpty.isEmpty()); } 16

17. Tear down Consider the following test code void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { try { use f and b } finally { clean&close f, b } } void testBBB() { try { use f and b } finally { clean&close f, b } } void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { use f and b } void testBBB(){ use f and b } Better? Problems?

18. Tear down • Consider the following test code void setUp() { File f = open(“foo”); File b = open(“bar”); } void testAAA() { use f and b } void testBBB(){ use f and b } void tearDown{ clean&close f, b } void setUp() { File f = open(“foo”); File b = open(“bar”); } … void tearDown{ try{ clean&close f, b }catch{ … } } void setUp() { File f = open(“foo”); File b = open(“bar”); } … void tearDown{ try{ clean&close f }catch{ … } the same for b } Better? Problems? 18

19. Tear down Do some cleanup job after the test is executed Close a file Clear a global data structure Revert changes to the database Close network connections … 19

20. Tear down • Be careful about tear down • If tear down is not complete, a test failure may affect the following test cases • Recover the changes done to global data that are not well handled by the setup • Database, files, network, global variables • Clean resources • Caution of exceptions in tear down itself 20

21. Demo on Unit Testing • Installation of JUnit • Writing and Running a Unit Test Case • Installation of ECLEmma http://www.eclemma.org/ • Interpretation of the coverage results • Loading and writing test case for Apache Email 21

22. The Practical Difficulty in Unit Testing • Decoupling • It is rare that you are writing test cases for a class with no dependencies (e.g., fibonacci array) • Dependencies are everywhere • Complex structures • System calls • Databases • File systems • Network • … 22

23. Problem of dependencies • Maintenance during setUp and tearDown • Expensive to setUp • Network, loading from file, start a database… • Hard to find bugs • Bugs may be related to the dependencies • They are still bugs, but should not be considered until system testing 23

24. Handling dependencies • Explicitly show dependencies • Hidden dependencies are dangerous • Dependency injection • Remove unnecessary dependencies • Reasonable design • Double dependencies • When dependency is large or not ready yet • Use mock object 24

25. A story from google A new developer go to a group and want to try on some methods testCreditCardCharge() { CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);} This code: Only works when you run as part of the suite. When run in isolation, throws NullPointerException. When you get your credit card bill, you are out $100 for every time the test runs. 25

26. A story from google After a lot of digging, you learn that you need to initialize the CreditCardProcessor. testCreditCardCharge() {CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);} Still not working 26

27. A story from google After a lot of digging, you learn that you need to initialize the OfflineQueue. testCreditCardCharge() {OfflineQueue.init(); CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);} Still not working 27

28. A story from google After a lot of digging, you learn that you need to initialize the Database. testCreditCardCharge() {Database.init(); OfflineQueue.init(); CreditCardProcessor.init(); CreditCard c = new CreditCard( "1234 5678 9012 3456", 5, 2008); c.charge(100);} But sometimes you have to find out the correct sequence! 28

29. Solution with dependency injection testCreditCardCharge() { Database db = Database(); OfflineQueue q = OfflineQueue(db); CreditCardProcessor ccp = new CreditCardProcessor(q); CreditCard c = new CreditCard("1234 5678 9012 3456", 5, 2008); c.charge(ccp, 100);} 29

30. Lessons Learned • Always try to make dependencies explicitly shown • Never do testing with real credit card number, • Or at least try $1 first 30

31. Remove Dependencies • Thin method interfaces • Example: • Get the total rent of a car • getRent(car, customer) -> getRent(car.getDays(), car.getPrice(), customer.getDiscount()) • Duplicate common dependencies • Example: • The class for Graduate students and under-graduate students both depend on a calendar module • It says, “if(student.isGraduate){…}else{…}” • Split it to two parts and put them into the class for both types of students 31

32. Double Dependencies • Removing Dependencies is not always doable • Lots of necessary dependencies • It is mostly something in design, so maybe unchangeable at the testing time • It is not always good to remove all unnecessary dependencies • increase maintenance effort • reduce the benefit of software reuse • make code more complex and hard to read 32

33. Double Dependencies • Solution: Double it! • Double: highly trained replacement, vaguely resembling the actor • No need to be a good actor • May require different skills for different cases 33

34. Types of test doubles • Dummies • Test stubs • Fake objects • Mock objects 34

35. Dummies • Simplest test double • No behavior, just serve as an argument • Null can be viewed as a simple double • Example public class ShopDummy extends Shop{ ... } public class OrderTest{ @Test public void testInit(){ Order o = new order(new ShopDummy()); ... } } public class Order{ private Shop sp; public Order (Shop sp){ this.sp = sp; } public Shop getShop(){ return sp; } … } 35

36. When to use dummies • SUT does not invoke any method on the dependency • The dependency class is an interface or abstract class with no implementation or • The dependency class is hard to initialize so that you can put the initialization in the dummy once • E.g., Shop sp = new Shop(new A(…), new B(…), new C(…), new D(…), new E(…)); • Can put the whole thing in ShopDummy

37. Test Stubs • More powerful than dummies • Provide a fix value or fixed behavior for a certain method invocation • Example: • Always return 0 for a integer method • Do nothing for a void method 37

38. Hard-coded Test Stubs • The value or behavior is hard coded in the Stub Class public class ShopStub extends Shop{ public void save(Order o){ } public double getShopDiscount(){ return 0.9; } } public class OrderTest{ @Test public void test(){ Order o = new order(new ShopStub()); o.add(1122, 3); ... AssertEquals(expect, o.getTotal()); o.save(); } } 38

39. Configurable Test Stubs • You may set different values for different test cases public class ShopStub extends Shop{ private Exception saveExc; private discount; public setException(Exception e){ this.saveExc = e; } public setDicount(Float f){ this.discount = f; } public void save(Order o){ if(this.saveExc!=null){throw saveExc;} } public double getShopDiscount(){ return this.discount; } } public class OrderTest{ @Test public void testAbnormalDiscount(){ ShopStub stub = new ShopStub(); stub.setDiscount(1.1); Order o = new order(stub); o.add(1122, 3); ... AssertEquals(expect, o.getTotal()); o.save(); } } 39

40. Fake Objects • More powerful than stubs • A simplified implementation of the DOC • Example: a data table to fake a database • Example: use a greed algorithm to fake a complex optimized algorithm • Guidelines • Slow -> Fast • Complex -> Simple

41. Fake Objects • Tips for fake objects • As simple as possible (as long as not too time-consuming) • Go to a higher level if some object is hard to fake • Example: URLStatus sts = HttpConnection.open("http://api.dropbox.com/files/myfile"); if(sts.status == 200){ return sts.data; }else{ return “Error”; } • Need to double • Difficult to reproduce • Maybe slow • Affected by lots of factors

42. Fake Objects Example (1) • Fake HttpConnection • We need to fake URLStatus also public class FakeDropBoxApi{ private files = { }; public FakeUrlStatus read(fname){ if(files.contain(fname)){ return FakeUrlStatus(200, files[fname]); }else{ return FakeUrlStatus(-1, "Error"); } } } public class FakeUrlStatus{ public FakeUrlStatus(int status, string data){...}; public int getStatus(){...}; public String getData(){...}; } FakeURLStatus sts = FakeDropBoxApi.read("myfile"); if(sts.status == 200){ …

43. Fake Objects Example (2) • Fake the HttpConnection + URLStatus public class FakeDropBoxHigherApi{ private files = { }; public String read(fname){ if(files.contain(fname)){ return files[fname]; }else{ return "Error"; } } } return FakeDropBoxHigherApi.read("myfile");

44. Fake Objects • Choose a reasonable abstraction level to do the faking • Do not need to fake lots of dependencies (put them in one box) • Write simpler code (less potential bugs)

45. Mock objects • Mock objects is not similar to any of the above • For the ability to imitate DOC, it is similar to configurable stubs • But it does some different things • Mock objects do behavior-based testing • Usually we only check return values or status • AssertEquals (expected, actual); • AssertEquals (expected, array.length); • Can we do something like this? Why? • Assert ( testObject.f1 calls DOC.f) 45

46. Mock objects • Go back to the order example public class ShopStub extends Shop{ public void save(Order o){ } public double getShopDiscount(){ return 0.9; } } public class OrderTest{ @Test public void test(){ Order o = new order(new ShopStub()); o.add(1122, 3); ... AssertEquals(expect, o.getTotal()); o.save(); } } Problems??? 46 46

47. Mock objects • Do behavioral assertions for SUT • Only cares about whether the unit to be tested is correct • Do not care about the correctness of DOC DON’T CARE!! SUT DOC Test Code

48. Mock objects @Test public void testOrder() { //initialize Shop sp = EasyMock.CreateMock(Shop.class); Order o = new Order(sp); o.add(1234, 1); o.add(4321, 3); //record EasyMock.expect(sp.getDiscount()).andReturn(0.9); sp.save(o); EasyMock.expectLastCall(); //replay EasyMock.replay(sp); AssertEquals(expect, o.getTotal()); o.Save(); EasyMock.verify(sp) } • Example • EasyMock

49. Mock objects I am a mock object, just generated • Procedure Record: What I am expected to be invoked, and what should I return Expect Methods Replay: See how SUT invoked me SUT Verification Results 49

50. Verification • Verifies whether the expected methods are actually invoked • Exception: missing, expected save(0xaaaa) • Verifies whether the expected methods are invoked in an expected way • Exception: save(null) expected save(0xaaaa) 50