Readings

1. Practical Software TestingTest Generation from Requirements using JUnitVahid Garousihttp://maralsoft.garousi.net

2. Readings • Chapter 2

3. Test automation • Test automation is using software tools to automate any aspect of testing • Generating test inputs and expected results • Running test suites without manual intervention • Evaluating pass/no pass • Testing must be automated (as much as possible) to be effective and repeatable

4. JUnit • JUnit is a framework for writing unit tests • Written by Erich Gamma (of Design Patterns fame) and Kent Beck (creator of XP methodology) • JUnit helps the programmer: • define and execute tests and test suites • formalize requirements • write and debug test code • integrate code and always be ready to release a working version

5. Equivalence Class Testing - A Test design Technique Motivation: we would like to have a sense of complete testing based on specification and we would hope to avoid test redundancy (having several copies of the same test cases) One popular solution to this: Equivalence Class Testing The equivalence classes are created assuming that the program under test exhibits the same behavior on all similar elements, i.e., tests within a class. Equivalence classes: partitions of the input set in which input data result in the same behavior of the program Does “same behavior” necessarily mean “same output”? No! e.g., abs(x). Let’s draw the equivalence class x>=0: abs(x)=x x<0: abs(x)=-x EC5 EC4 EC6 EC1 EC3 A SUT’s input set EC2

6. Equivalence Class Testing – In JUnit • Create a new Java Project: File-> New -> Java Project • Call your project “JUnitExample”. Click Finish. • Right click on your newly-created project and choose: • New->JUnit Test Case • Write the following code and click the run button Run

7. Equivalence Class Testing - Example 2 NextDate is a function with three variables: month, day, year. It returns the date of the day after the input date. E.g., if we give Jan. 1, 2009 as input, it will provide the output as: Jan. 2, 2009 Limitation (simplification): years between 1812-2012. It just needs to support that range. Treatment Summary: If the input date is not the last day of the month, the next date function will simply increment the day value. At the end of a month, the next day is 1 and the month is incremented. At the end of the year, both the day and the month are reset to 1, and the year incremented. Finally, the problem of leap years makes determining the last day of a month interesting! (What is a leap year? next)

8. Equivalence Class Testing - Example 2 Definition of leap years (http://en.wikipedia.org/wiki/Leap_years): The Gregorian calendar adds a 29th day to February in all years evenly divisible by 4, except for centennial years (those ending in -00) which are not evenly divisible by 400. Thus 1600, 2000 and 2400 are leap years, but 1700, 1800, 1900, 2100, 2200 and 2300 are not. Only the year 1900 falls in the 1812-2012 period Derive the equivalence classes of NextDate. Hint: Determine the program’s input variables first? How many variables do we have?

9. Back to the NextDate Example Number of WECT test cases=maximum partition size (D)=4 An example test suite? Also include the expected output for each test case.

10. NextDateStrong Equivalence Class Testing (SECT) - Test Cases • Number of SECT test cases= partition size (D) x partition size (M) x partition size (Y) = 3x4x3=36 test cases

11. Consider the Print feature in PowerPoint. • What will be the equivalence classes and test cases for SECT for the print feature? • In this case, the SUT is tested via its GUI, i.e., data is input using commands available in the GUI. • The GUI might disallow invalid inputs by offering a palette of valid inputs only. (example: above two figures)

12. Equivalence Class Testing - Example JFreeChart – We will use this in our lab assignment • http://www.jfree.org/jfreechart • The JFreeChart framework is a free/open-source library intended to be integrated into other systems as a quick and simple way to add charting functionality to other Java applications.

13. JFreeChart – Requirements • Reminder: SW Requirements can come in many different formats, e.g., plain English, UML models, Javadoc, … • In lab #1, we will use the Javadoc documentation of JFreeChart as its requirements

14. Lab #1 - JFreeChart – Classes to test based on requirements • org.jfree.data.DataUtilities: Has 5 methods (details next) • org.jfree.data.Range… • org.jfree.data.io.CSV…

15. JFreeChart – Class org.jfree.data.DataUtilitiesHas 5 methods… Let’s apply equivalence partitioning testing to this method (NEXT). You will do the rest in the lab #1! Note: What is this method supposed to do?

16. Systematic procedure for equivalence partitioningJFreeChart – Class org.jfree.data.DataUtilities • Identify the input domain • Equivalence classing • Combine equivalence classes • Identify infeasible equivalence classes

17. Systematic procedure for equivalence partitioningClass org.jfree.data.DataUtilities • Identify the input domain • We should start from the Requirements (in Javadoc format) • Exercise: What is the input domain of method calculateColumnTotal?

18. Systematic procedure for equivalence partitioningJFreeChart – Class org.jfree.data.DataUtilities • Equivalence classing • Let’s draw the equivalence classes diagram for method calculateColumnTotal?

19. Systematic procedure for equivalence partitioningJFreeChart – Class org.jfree.data.DataUtilities • Combine equivalence classes: • We should use either Weak Equivalence Class Testing (WECT) (uni-dimensional), or Strong … (SECT) (multi-dimensional) • Combine the equivalence classes (derive test cases) in two ways: uni-dimensional and multi-dimensional. • Also, specify the expected output for each test case. • Hint: In Java, InvalidParameterException (which extends IllegalArgumentException) should be thrown (by our code) when an invalid parameter is passed to a method.

20. Test Generation from Requirements - Outline • Types of requirements-based Test Generation Techniques • Test Selection Problem • Exhaustive (complete) Testing • Large and Complex Input Domains • Black-box Testing: Basic Principles • Black-box Testing: Different Techniques • Equivalence Class Partitioning • GUI design and equivalence classes • Boundary-Value Analysis (BVA) • Decision tables • Cause-Effect Graphs

21. -ve +ve Boundary-Value Analysis (BVA) • Experience indicates that, sometimes, programmers make mistakes in processing values at or near the boundaries of equivalence classes. • For example, suppose that method M is required to call the function f1 when x 0 is true, and function f2 otherwise. • However, M has an error: it calls f1 for x<0, and f2 otherwise. • Obviously, this fault will be revealed when M is tested against x=0. But not if the input test set is, for example, {-4, 7}, derived using equivalence partitioning. • In this example, the value x=0, lies at the boundary of the equivalence classes x0and x>0.

22. More General Boundary-Value Analysis: Approach Recall from our earlier discussion: “BVA focuses on tests at or near the boundaries of equivalence classes.” So certain failures may occur with values just next to the boundary, and not exactly on it. Approach: For each equivalence class, setting values for input variable at their minimum, just above the minimum, a nominal value, just below their maximum, and at their maximum. Convention for the above notions: min, min+, nominal (typical), max-, max A usual strategy for all input variables: Holding the values of all but one variable at their nominal values, letting one variable take each of the above five combinations Examples next… EC1 min min+ nom max- max

23. Boundary-Value AnalysisInput Domain of Function F • Assume a function F, with two variables x1 and x2, domains X1 and X2 • Boundaries: a <= x1 <= b, c <= x2 <= d X2 d c X1 b a

24. Boundary-value Analysis Test Cases Test set ={ <x1nom, x2min>, <x1nom, x2min+>, <x1nom, x2nom>, <x1nom, x2max->, <x1nom, x2max>, <x1min, x2nom,>, <x1min+, x2nom,>, <x1max-, x2nom>, <x1max, x2nom>}. Number of test cases=9 X2 d c X1 b a min min+ nom max- max

25. Boundary-Value Analysis Even further: Robust Worst-Case Testing for 2 variables • How many test cases will a function with n variables have? X2 d c X1 b a

26. We will now start the JUnit Lab (#1) • JUnit is a framework for writing unit tests