2003-09-19  Dan Garcia (cs.berkeley/~ddgarcia)

Computer Science 61BData Structures and Advanced ProgrammingLecture 11 – Testing 2003-09-19  Dan Garcia(www.cs.berkeley.edu/~ddgarcia) Kathy Yelick (www.cs.berkeley.edu/~yelick) inst.eecs.berkeley.edu/~cs61b/www.ucwise.org1 Handout:notes

Testing Overview • Purpose of testing: Provide evidence that specification matches implementation • Why not prove that the program works? • Because it’s impossible in general • Because it’s infeasible in practice for most programs • But there is interesting work here (OSQ, etc.) • Testing is hard • It cannot produce proof of correctness. • It can be tedious. • It is psychologically difficult • Testing is not debugging • Testing is to ensure there are no bugs, • Debugging is tracking down a known problem

Command Loop Testing • One way to testing a program is to write a “driver” in the main method, e.g., Enter month: __3___ Enter day: __4___ What method: __t__ Result is: “tomorrow is 3/5” • What’s wrong with this? • Hard to keep track of which tests you have run • A lot of work to redo testing • Don’t do this! Instead write test code. • More work the first time, but less in the long run

Regression Testing • Regression Testing: repeated testing, to make sure that your program has not “regressed”: • It worked yesterday, but not today (“bit rot”) • Because I fixed something else, or added a feature,… • Write a program to test your program • More work to write than the interpreter, but • Easy to run after that • Have someone else write test code if possible • Do not cut-and-paste from solution to test code • Example: Titanium has 2 sets of tests • Nightly “cron” job checks that compiler not broken • Before a major release, larger set of tests run on multiple machines

Testing Principles • Want tests to be easy to understand: Testing tomorrow method: tomorrow on 3/4 is 3/5 • Self-checking if possible: All tomorrow tests PASSED • How much test code? • At least 1x-3x more test code than “actual” code • More is not always better: not 2 pages, but the right 2 pages • Divide cases into normal, abnormal, boundaries • Boundary cases in the interface • Branches (and loop bounds) in the code

Black-box vs. Glass-box Testing • Black box tests derived from the specification • Usually done in a separate class, e.g., DateTester • Test all public methods • Use the specification to select cases: normal, exceptional, etc. • Do not test things that fail to satisfy the “requires” • Glass box tests derived from an implementation. • Usually done inside the class, in the main method java IslamicDate -- runs main • Can test private methods, e.g., gcd • Can test repOk on illegal representation values • Design tests looking at the code: • branches and loops: test bounds • test all paths through the code

Choosing Test Cases • Consider testing contains1MoreThan • Test different outputs (only 2: true, false) • Test for possible crashing • Nulls, substring/array/Vector bounds, 0-divide • Note re-use of test code across versions. • Useful in project 1 (although some behavior differs) • Test boundaries Placement of character (first, last, middle) No character inserted: different/same Length of strings (s1, shorter, longer, same) • Glass box testing (which could still be done “outside”) would include loop boundaries, recursion cases, etc. • Most errors occur at the boundaries!

Testing Classes • These ideas are fine for (static) methods, but how do we test classes? • Abstraction prevents you from seeing inside objects • Idea: • test all methods that build different objects • constructors, mutators, methods that return objects of the class • Using accessors/observers • All methods and return (or modify) a different type • What if constructors & observers are buggy? • Answer: It’s only a bug if you can write a program to see it.

Assume equals/toString OK Assume IslamicDate(int,int) is OK Example: IslamicDateTester check daySpans write daySpans Group related methods for testing (check non-mutation) test tomorrows write tomorrows test other constructors write other constructors test toString write toString test equals Next do most important accessors write equals Start with most important/simplest method: make sure it doesn’t crash test IslamicDate(int,int) write IslamicDate(int,int)

Test Framework for IslamicDate • Define a separate class for black-box testing public class DateTester public static int testBasicConstructor () public static int testEquals () public static int testToString () public static int testOtherConstructors () public static int testTomorrows () public static int testDaySpans () public static void main (String [ ] args) Checks that the simplest constructor doesn’t crash Checks equals using basic constructor Checks toString using basic constructor Checks other constructors using toString and equals Checks tomorrow and makeTomorrow using above Checks recDaySpan and iterDaySpan using above Calls all the other test routines

Abstraction Helps in Testing • The previous slides assume tight abstraction: • That no one can see inside and modify internals • Public fields complicate the picture: • Include all values of public non-final fields in creating the set of “all objects” • Include all public fields (final or not) in the set of all “observers” • Design principle: narrow interfaces: • Methods: keep the number of input parameters small • Classes, avoid public fields, except when necessary • This simplifies testing (and use) of your code

Testing repOk • repOk cannot be throughly tested outside the class (can’t get false out) Use glass-box testing: IsalamicDate date1 = new IslamicDate(1,1); for (int month = 1; month <= 2; month++) { date1.myMonth = month; for (int day = -1; day <=32; day = (day==0 ? 30+month%2 : day+1)) { date1.myDay = day; showVerbose("Testing repOk on " + date1); if (date1.repOk()) { System.out.println(“Unexpected repOk->true ” + “for ” + date1); allPassed = false; } } } Note use of loops for generating tests Avoid using same logic as in the other methods

Aside: Using repOk • If systems (your program) are going to fail, you want it to happen as soon as possible • Place repOk calls at then end of all constructors and mutators using assert: assert <boolean> [ : <String>] • Example: public IslamicDate(int month, int day) { myMonth = month; myDay = day; assert repOk() : "repOk failed on IslamicDate(int,int)"; }

More on Assert • Assert only works with Java 1.4 and beyond • New keyword “assert” causes problems for old code • Compile with javac –source 1.4 <JavaFileName> • Run with java –ea <ClassName> • ea = enable assertions • No cost for assert statement when the are not enabled • See more variations at: http://java.sun.com/j2se/1.4.2/docs/guide/lang/assert.html

Static Definitions: • Static: associated with a class, not its objects • Non-static: associated with an object of a class • Non-static fields: instance variables • Each object in the class gets its own copy • Static fields: class variables • One variable for all the objects in the class • Non-static methods • Operate on an object • Can access “this” and instance variables • Static methods • Do not operate on an object

Non-Static During the call to balance on the Mike object, there is a “this” variable Account kathy = new Account (100); Account mike = new Account (150); mike.balance()  150 this kathy mike myBalance myBalance 150 100

Add to all constructors public Account (int bal) { myBalance = bal; ourCount++; } Add an observer public static int getCount () { return ourCount; } Keep track of the number of Accounts that have been created Static Field/Method Example • Seen static/final fields for constants • Add a non-final one (rare!) to the Account class public class Account { private static int ourCount = 0; private int myBalance; }

kathy mike myBalance myBalance 150 100 Static Before any Account objects are created: Account kathy = new Account (100); ourCount Account mike = new Account (150); 0 2 1 Account.getCount()  2

Static Non-static fields: instance variables • Common case; accessed by “this” in non-static methods • Static fields: class variables • Shared by all objects; accessed as • classname.fieldname • object.fieldname (legal but not recommended) • Non-static methods • Common case; manipulate non-static fields (uses this) • Static methods • Do not operate on an object; no “this” or non-static fields can be accessed

PRS Question • Assume we have a Fraction containing public methods: public double toDouble ( ) { … } public static int gcd (int x, int y) { … } public boolean repOk () { … } • And a separate FractionTester class with: Fraction f1 = new Fraction (…); • How many of the following will cause a CT or RT error? 1: Fraction.toDouble() 2: Fraction.gcd(12,8) 3: f1.toDouble() 4: f1.gcd(12,8) 5: assert (f1.toDouble() > 0); 6: assert Fraction.repOk(); 7: assert f1.repOk(); 0: none 1: 2: 3: 4: 5: 6: 7: all

Extra Slides Read these as well!

PRS Answer • Assume we have a Fraction containing: public double toDouble ( ) { … } public static int gcd (int x, int y) { … } public boolean repOk () { … } • And a separate FractionTester class with: Fraction f1 = new Fraction (…); • How many of the following will cause a CT or RT error? 1: Fraction.toDouble() 2: Fraction.gcd(12,8) 3: f1.toDouble() 4: f1.gcd(12,8) 5: assert (f1.toDouble() > 0); 6: assert Fraction.repOk(); 7: assert f1.repOk(); 0: none 1: 2: 3: 4: 5: 6: 7: all

Administrivia • Quiz • Review session Saturday 2-4pm in 306 Soda • Not on the quiz: Exceptions, repOk, invariants, lab3 • For TRUE & FALSE questions, you will be graded #right – #wrong • No homework this week or next week • Start the project now! • Watch newgroup, errata.txt in proj1, announcements

equals toString recDaysBetween iterDaysBetween dayOfYear Testing Classes • Testing in the ideal world Use all “observer” methods in the class that return (or modify) a different type Use constructors, plus mutators, and any method that returns the type to construct: Check that all of these return the expected value Set of all possible objects of a given type All of this is done using the previous rules for checking normal, unusual, and boundary inputs IslamicDate (all constructors) tomorrow makeTomorrow

Two More Categories of Testing • Liskov distinguishes between • Modular testing • One class at a time • Integration testing • Testing multiple classes together • E.g., the entire program • Integration testing is even harder! • The test cases are harder to design (too large) • Debugging at this level is also hard • Reduce the errors in this phase by doing more modular testing.

2003-09-19  Dan Garcia (cs.berkeley/~ddgarcia)