Lecture 5: Testing

CSC 213 – Large Scale Programming Lecture 5:Testing

Today’s Goal • Understand why testing code is important • Result of poor or no testing & embarrassment caused • Learn basics of writing good suite of test cases • What test cases should do (& should not do) • Managing test cases to find as many bugs as possible • Make debugging easier with clear, focused tests • Analyzing tests to not waste time on useless ones • Learn types of test cases that most often used • Useful and helpful ideas from coders in real-world

Why Does Testing Matter?

Testing Drives Debugging Does it work correctly? Who gets to decide? How do they know? Can YOU Offer Any Proof?

Testing Drives Debugging Does it work correctly? Can YOU Offer Any Proof?

Does It Work Correctly? • You have two options with all of this testing: • Show code works by writing working tests • Stop complaining & accept your punishment

What Is “Correct”? • Could check code against design being used • Verification is name for these checks • These tests are easy to create • Verification still assumes design is correct • Validation checks code does what it should

What Is “Correct”? • All roads lead back to requirements

What Is The Error? • Make sure each test method looks for 1 error • Easier debugging when you also use good names • Can you figure out when each of these tests fail?testAverageOnNullArray()testInsertFirstWhenEmpty()testPopOnEmptyStack()testOnTimeUsingTim()testDeposit2()

Critical Property of Test At start, test must FAIL

Could We Find an Error? • Write tests with known, constant answers • Use constants in your assert* statements • Relying on other methods’ results propagate errors • Which would you want, if your money’s at stake? assertEquals(cut.addTwo(10), cut.addOne(11)); assertEquals(cut.addTwo(10), 12); assertEquals(cut.winner(), cut.runners[0]); assertEquals(cut.winner(), cut.getRunner(0)); assertEquals(cut.winner(), “Cheetah”);

Are We Sure There Is No Error? • Test all aspects of results to look for any errors • Should begin by looking that data returned correct • Check any fields and see if they are what is expected • Add assertions that array entries & other data okay • Problems caused when assumewithout full data

Where Is The Error? • Check methods from simplest to most complex • Check easiest methods first and get them working • Once this done, test methods calling the easiest ones • Most complex methods tested last after all else works • Fixes method with bug & prevents wasting time • ONLY call method being tested to also help with this

Why Is This An Error? • Always check your test cases • Test cases are not perfect; may have errors • Many hours lost looking for non-existent bugs • But, at the same time, bad test cases lame excuse

How To Write Tests • Cannot prove there are no bugs • Only ever show no bugs exist on those tests • Unlikely & boundary scenarios are vital to test

How To Write Tests • Cannot prove there are no bugs • Only ever show no bugs exist on those tests • Unlikely & boundary scenarios are vital to test • Keep in mind why Willie Sutton robbed banks That’s where themoneyis

How To Test Boundaries • Each test should provide additional information • Why write two tests which expose same bug? • Before writing test ask: what error will this expose? • Must Include boundary checks for objects – null • Test each parameter unless pre-condition says no • Good set of tests should also check all fields used • Boundaries for arrays & Strings also very simple • Unless forbidden, should also check if these null • Lengths of 0, 1, & nalso important to try

Interesting Boundary Cases • Number are much harder; no simple rules exist • To find boundaries, must look at variables used • Try negative, 0, & 1when used in arithmetic • Check values around limit in if(…) statements public intgcd(intap, int a, int b) {if (b == 7) { return ap; }ap = a;a = b;b =ap % a;return gcd(ap, a, b); }

Types of Loops Simple Loop Concatenated Loops Nested Loops Unstructured Loops

Simple Loop • For all simple loops, try inputs that: • Skip loop entirely • Make 1 pass through the loop • Make 2 passes through the loop • Make m passes through the loop, where (m > 2) • If loop executed at most n times, try inputs that: • Make n-1, n, & (n+1) passesthrough the loop

Nested Loops • First test set runs all outer loops exactly once • Inner loop runs (min+1), average, (max-1) & maxtimes • Then run all but two innermost loops exactly once • Inner loops run (min+1), average, (max-1) & maxtimes • Tests should continue growing loop-by-loop

Concatenated Loops • If loops are entirely independent • No conditions, variables, or values in common • Woo-hoo! Just perform single loop tests on each • Otherwise treat as nested loops & make life easier • Work as if the first loop is outermost loops

Types of Loops

Unstructured Loops • Figure out the process leading to this decision • Burn the artifacts creating this abomination • Anyone involved should terminated immediately

Unstructured Loops • Figure out the process leading to this decision • Burn the artifacts creating this abomination • Anyone involved should terminated immediately • Rewrite “missing” documentsfrom scratch

For Next Lecture • Next weekly assignment available on Angel • Due as usual on Tuesday at 5PM • Use virtual extensions to delay this due date • Reading on advanced testing techniques • Does un-testable code exist? • What do we do with this flaky, scary code? • What tricks are there to help us work with it?

Lecture 5: Testing