Lecture 9: Debugging & Testing

Lecture 9:Debugging & Testing

Objectives “Unfortunately, errors are a part of life. Some are detected by the compiler, some are detected through testing, and the remainder go uncaught. Modern languages and tools like J#, Visual Studio .NET, and NUnit provide many techniques for quickly identifying, dealing with, and eliminating errors. But you have to apply them…” • Debugging options • Testing • Testing with NUnit

Part 1 • Debugging options…

Print debugging • Everyone understands print debugging • In .NET, you have various options: • console app: System.out.println(...); • GUI app: System.Windows.Forms.MessageBox.Show(...); • inside VS: System.Diagnostics.Debug.WriteLine(...);

Source-level debugging • Most development environments supply a debugger • allow you to debug your program line-by-line, look at vars, etc. • Visual Studio .NET contains a very sophisticated debugger • Basic idea: • you set a breakpoint to pause your program at a certain point • run program in order to reach breakpoint • once paused, you can look at variables, single-step, etc. click in margin to set breakpoint

Debug menu • Once you reachbreakpoint, you can: • single-step • view local vars • change values • Example: • Watch window let’syou expand &inspect objects! • See Debug menu forthings you can do…

Examples • When using constructors, set a breakpoint where you do “new Class1(…)”, run to hit breakpoint, and then single-step, showing how constructor is automatically called… • When using dynamic binding, single-step through calls to see how different method implementations are called dynamically…

Part 2 • Testing…

Testing • Good testing is a skill that students must develop: • algorithm boundary cases • single calls & sequences of calls • parameter validation • exception handling • etc. • Good testing is automated • so there’s no reason not to do it • Good testing is repeatable (regression testing) • so you can retest after any change to the code

class C testharnessTC Basic testing methodology • For every class C, create a test harness TC to test it • TC = one or more methods that exercise C • Test harness should: • exercise all fields & methods in C • strive for 100% codecoverage (exercise all paths of execution) • generate no output other than a final testing summary • i.e. test harness should *not* rely on user to determine correctness • user will forget what the “correct” output should be…

Example • StudentsIO class reads student info from a file • returns an ArrayList of Student objects… public class StudentsIO { /** * Reads student info from given file, returning arraylist */ public static java.util.ArrayList read(String filename) ... { java.util.ArrayList students; students = new java.util.ArrayList(); . . . return students; }

Example test harness • Tests StudentsIO.read( ) against various input files… public class StudentsIOTest { // tests against an empty input file… public static void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); } // tests against an input file with one student, "Jim Bag" public static void readOne() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("one.txt"); . . .

Running test harness • Common approach is to define a main method in test harness • Reconfigure VS to startup app using this main. Right-click on project in Solution Explorer (or use Project menu), select Properties, then Common Properties, then General, and set "Startup Object" in this case to "StudentApp.StudentsIOTest". public class StudentsIOTest { … public static void main(String[] args) { int failed = 0; System.out.println("** Testing StudentsIO **"); try { readEmpty(); } catch(Exception ex) { System.out.println(ex.toString()); failed++; } … } }//class

Part 3 • Testing with NUnit…

NUnit • NUnit is based on the highly-successful Java tool JUnit • NUnit is for .NET code • NUnit is for testing "units", i.e. classes • available for free from http://nunit.org/

.EXE / .DLL NUnit methodology • NUnit automates the running of the test harness • it doesn't write the test harness for you (that's our job) • it runs the test harness & collects statistics…

How does NUnit help? • NUnit helps in many ways: • you don't have to write the code to collect & output statistics • tells you exactly which tests failed / skipped • catches exceptions that testing methods may throw • provides a framework for regression testing, i.e. re-testing after source code has been modified

Using NUnit • Five-step process: • Download & install NUnit • Create test harness, with instance test methods & no main( ) • Set a reference to NUnit: • Project menu, Add Reference… • Browse… • Navigate to C:\Program Files\NUnit V2.1\bin • Open "nunit.framework.dll" • Ok • Add NUnit attributes to test harness, rebuild project • Startup Nunit-Gui via Start menu, open compiled code, & run! • "Green" light means test passed • "Yellow" light means test was skipped • "Red" light means test failed

Example • Let's revisit test harness for StudentsIO.read( )… • Same as before, except: • no main method; test methods should be instance methods • addition of NUnit attributes to identify test harness & methods /** @attribute NUnit.Framework.TestFixture() */ public class StudentsIOTest { /** @attribute NUnit.Framework.Test() */ public void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); } . . .

.EXE / .DLL Working with NUnit • Idea: • compile test harness, etc. in Visual Studio .NET • startup Nunit-Gui, file >> open compiled code, run test harness • Green is good, Yellow is caution, Red is bad • leave both tools open, compiling in VS & testing in NUnit

Debugging with NUnit • Visual Studio's debugger is still available • Process for debugging: • open Visual Studio & Nunit-Gui as usual • set breakpoints in VS as usual • in VS, attach debugger to Nunit-Gui as follows: • Debug menu • Processes… • select "nunit-gui.exe" from list • click "Attach" • check the box for "Common Language Runtime" & click OK • close Processes window, notice that VS is now in "Run" mode • in Nunit-Gui, click Run button! • when you are done testing/debugging, you'll need to stop VS…

Writing a test method • If test method returns, counted as success • If test method throws an exception, counted as failure • Implication: • test method must throw an exception to denote failed test… /** @attribute NUnit.Framework.Test() */ public void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); if (students == null || students.size() != 0) throw new java.lang.Exception("readEmpty failed!"); }

Assertions • Assertions are a common way of denoting failure • Idea: • you assert that a particular boolean condition is true • if condition turns out to be true, nothing happens • if condition turns out to be false, an exception is thrown /** @attribute NUnit.Framework.Test() */ public void readEmpty() throws java.lang.Exception { java.util.ArrayList students; students = StudentsIO.read("empty.txt"); System.Diagnostics.Debug.Assert(students != null); System.Diagnostics.Debug.Assert(students.size() == 0); }

Part 4 • Other tools & ideas…

Other tools & ideas • NUnit short demo videos by Martin Schray: http://www.msdnacademicalliance.net/curriculum/pfv.aspx?5935 • WinForms GUI testing? • NUnitForms, http://sourceforge.net/projects/nunitforms/ • Web app testing? • NUnitASP, http://sourceforge.net/projects/nunitasp/ • Other strategies? • programming with invariants, listing pre & post-conditions • these are based on the idea of using assertions in your code: … // at this point in the program, count should be 1, if not through an exception & stop now System.Diagnostics.Debug.Assert(count == 1, "Count==1 failed");

Summary • Unfortunately, errors are a part of being human • Errors can be eliminated, but it takes a great deal of effort: • good logic • good design • good programming • good testing • You have the tools: • J#, Visual Studio .NET, NUnit, etc.

Lecture 9: Debugging & Testing