CS428/9: Software Engineering II

1. CS428/9: Software Engineering II Darko Marinov (slides from Ralph Johnson)

2. Administrative info • HW 1 and Demo 1 grades sent today • HW 2 due next Tuesday, Feb 27 • Write several kinds of tests in subgroups of 2-3 • Will be graded by Thursday, Mar 1 • ACP revision 2 due on Thursday, Mar 1 • If you failed revision 1, meet with Tracy • Mar 1: Guest lecture on testing video games 428-11

3. Testing: Covered • Previously • Classic Testing Mistakes (by Brian Marick) • Kinds of testing (Hamlet & Maybee, ch. 6 & 19) • Black- and white-box techniques (H & M, ch. 20 & 21) • JUnit • The Impossibility of Complete Testing (by Cem Kaner) 428-11

4. Testing: To cover • Today • Testing non-functional properties, OO testing • Acceptance/system tests • Test doubles • Tuesday, Feb 27 • Assertions and debugging • Thursday, Mar 1: Guest lecture • Travis Tholen from Volition/THQ onQuality Assurance: General Software Testing 428-11

5. Some techniques for writing tests • Black-box (from specifications) • Equivalence partitioning • Boundary value analysis • White-box (from code) • Structural code coverage • Every statement, branch, condition, path… 428-11

6. More kinds of testing • Performance testing • Load testing • Configuration testing • OO testing 428-11

7. Performance testing • Testing requires specification • Performance tests must measure time of operations • Test fails if it takes too long • Keep database of past results, and track changes in performance over time 428-11

8. Load testing • Some problems only show up under heavy load • Includes performance, but more than performance • Often requires computers to generate load 428-11

9. Configuration testing • Some systems have many configurations • Do they all work? • How can you tell? 428-11

10. Example: Hardware and OS • Amount of memory: 512MB, 1GB, 2GB • Processors: single-core, dual-core, quad-core • OS: Win XP, Win Vista, Linux, Mac OS X • Each configuration is a triple of some amount of memory, some processor, and some OS • Widely-used approach: cover all pairs 428-11

11. Planning testing • Testing must be planned • Make sure important testing happens first • Make sure you have enough people • Make sure it starts soon enough 428-11

12. OO testing • Structure of tests should match structure of system • OO systems have extra structure • Units of OO systems are classes • Subclass of TestCase for each class being tested • Often one unit is a group of classes 428-11

13. Testing and reuse It might be argued that, as OOA and OOD mature, greater reuse of design patterns will mitigate the need for heavy testing of OO systems. Exactly the opposite is true. 428-11

14. Testing and reuse • Reuse reduces testing • Use discovers bugs. Reused components have less bugs. • Reuse does not reduce testing • Inherited methods can behave differently • Must test inherited methods 428-11

15. public class Foo { private int x; public String asString() { … writeOn(aStream); …} public abstract writeOn(Stream){} } public class Fee extends Foo { public writeOn(Stream aStream) {…} }

16. Continuous Improvement • Use testing to learn how to improve development • Why did this bug happen? • Where else might we have made this mistake? • What could we have done to prevent it? • Change reviews • Change tests we write 428-11

17. Summary of programmer’s tests • Lots of techniques for writing tests • Testing shows the presence of bugs, it never proves their absence • All testing is a guess • Manage testing • Make it effective • Make it cost-effective • Use what you learn to improve your process 428-11

18. Customer’s tests • Acceptance tests / System tests • Customer alternatives • Hire programmers to automate tests • Manual testing • Testing tool like SilkTest and WinRunner • FIT: Framework for Integrated Test 428-11

19. Customer tests • Most customers are not programmers • Customers don’t care about the structure of the program • Need to write lots of similar tests and know which ones work and which ones don’t 428-11

20. FIT • Framework for Integrated Test: http://fit.c2.com • Created by Ward Cunningham • Use HTML to write tests and view their outputs • Tests are in tables • Anything outside of a table is a comment • Output is same tables as input, but showing actual results and success/failure • Customers write tables and run programs 428-11

25. FIT Fixtures • A fixture is a Java class • It reads tests from the table and executes them • Fixture • ColumnFixture • ArithmeticColumnFixture • RowFixture • ActionFixture • PrimitiveFixture 428-11

26. public class ArithmeticFixture extends PrimitiveFixture { int x=0; int y=0; public void doRows(Parse rows) { super.doRows(rows.more); // skip column heads } public void doCell(Parse cell, int column) { switch (column) { case 0: x = (int)parseLong(cell); break; case 1: y = (int)parseLong(cell); break; case 2: check(cell, x+y); break; case 3: check(cell, x-y); break; case 4: check(cell, x*y); break; case 5: check(cell, x/y); break; default: ignore(cell); break; } } }

27. public class ArithmeticColumnFixture extends ColumnFixture { public int x; public int y; public int plus () { return x + y; } public int minus() { return x - y; } public int times () { return x * y; } public int divide () { return x / y; } public float floating () { return (float)x / (float)y; } public ScientificDouble sin () { return new ScientificDouble(Math.sin(Math.toRadians(x))); } public ScientificDouble cos () { return new ScientificDouble(Math.cos(Math.toRadians(x))); } }

28. Using FIT • HTML can be • In a file • On a web server (especially a Wiki) • Run from regular Java program • FileRunner • WikiRunner • Run from cgi script 428-11

29. Using FIT • Variety of languages • Java • .NET • Perl • Python • Lisp • Ruby • C++ 428-11

30. How to test with FIT • Customer writes tables with Excel to specify the system • Programmer figures out how to write a fixture for them • (Negotiation) • Customer exports tables to HTML, and thereafter maintains them with either Word or Excel 428-11

31. Test double • How can you test one class without having the whole system working? • Easy if class doesn’t depend on anything • Unimportant if class doesn’t do much except combine other objects • Use mock object / test double • http://xunitpatterns.com/Using%20Test%20Doubles.html 428-11

32. System under test • What is your “unit”? • A method? • A class? • A set of classes? • Write test objects for everything else needed to make the test run 428-11

33. Test double • Test double • An object written by a tester that acts like the real object • Fake object - is simpler / faster • Test stub - follows test script • Recording test stub - records interaction so that test can check it • Mock object – knows entire expected behavior 428-11

34. Test doubles • Often write test doubles for code that someone else is writing • Don’t want to wait for it to be done • Don’t want to test their code • Don’t trust them • Test doubles start small and grow • Refactor test doubles 428-11

35. Next time • Debugging • Assertions • Oracles 428-11