Class Testing

1. Class Testing ECEN5033 University of Colorado

2. Overview • Class Testing • Testing Interactions between objects • Testing Class Hierarchies ECEN5033 University of Colorado -- Class Testing

3. Basics – How to test a single class • Class Testing • How to Test a Class • Aspects of Class Testing • How to Construct Test Cases • When is a class test suite adequate? • How to Construct a Test Driver • Testing Interactions between objects • Testing Class Hierarchies ECEN5033 University of Colorado -- Class Testing

4. Definition of class testing • Verifying implementation of a class = verifying the specification for that class • If so, each of the instances should behave properly. • Assumption: • The class in question has a complete and correct specification that has been tested earlier in the context of models • Spec is expressed in natural language or as a state transition diagram ECEN5033 University of Colorado -- Class Testing

5. Ways to test a class • Code can be tested effectively by • inspection (preferable when construction of a test driver is too difficult) • execution of test cases (lends itself to easy regression testing later on) • Remember: When you test a class, you are really: • creating instances of that class and • testing the behavior of those instances. ECEN5033 University of Colorado -- Class Testing

6. Aspects of Class Testing • Decide • test independently • test as a component of a larger part of the system • How decide – combination of the following: • Role of the class in the system – degree of risk • Complexity of the class • Amount of effort associated with developing a test driver • Sometimes, needs so many collaborators, makes more sense to test in a cluster ECEN5033 University of Colorado -- Class Testing

7. Who Tests? How much? • Who – class usually tested by its developer • Understands class’ spec • Familiar with the code • Test driver can be used by the developer to debug code while writing it • Perpetuates misunderstanding – needs inspection at model stage to head that off • What – ensure that the code for a class exactly meets the requirements – no more, no less ECEN5033 University of Colorado -- Class Testing

8. When are tests written? • When – A test plan that identifies test cases should be developed soon after a class is fully specified and ready for coding. • Especially if developer is the class tester • Why? • (What’s the danger in developer alone writing and reviewing test cases for the class?) • When again? • Iterative development – the driver and the test cases will be available to supplement or change as the class is enhanced or modified ECEN5033 University of Colorado -- Class Testing

9. How can you test “just a class”? • How – • Create a test driver that • creates instances of the class • sets up suitable environment around those instances to run a test case • sends one or more messages to an instance as specified by a test case • checks the outcome based on a reply value, changes to the instance, or parameters to the message • deletes any instances it creates if responsible for storage allocation ECEN5033 University of Colorado -- Class Testing

10. How do we test static data in a class? • How – continued: • Static data members or operations • testing required • they belong to the class itself rather than to each instance of the class • the class can be treated as an object • If the behavior of the instances of a class is based on the values of class-level attributes • test cases for testing the class-level attributes must be considered as an extension of the state of the instances ECEN5033 University of Colorado -- Class Testing

11. How MUCH testing is done at this level? • How much – Adequacy of testing measured in terms of • how much of the spec has been tested • how much of the implementation has been tested • Want to test operations and state transitions in many combinations • Objects maintain state. “State” affects the meaning of operations. ECEN5033 University of Colorado -- Class Testing

12. Constructing test cases - identification • Identification of test cases • Should be made from the spec that has been reviewed rather than from the implementation which may embody developer’s misunderstandings • Best: develop from spec and augment to test boundaries introduced by the implementation • If no spec exists: create one from the code and verify it with the developer ECEN5033 University of Colorado -- Class Testing

13. Constructing test cases – identification2 • Identify requirements for test cases for all possible combinations of situations in which • a precondition can hold • post conditions can be achieved • Create test cases for those requirements • specific input values – typical and boundary • determine correct outputs • eliminate conditions that are not meaningful • Add test cases to show what happens when a precondition is violated ECEN5033 University of Colorado -- Class Testing

14. Constructing Test Cases – from STD’s • State Transition Diagrams • They show behavior associated with instances of a class • Each transition represents a requirement for one or more test cases ECEN5033 University of Colorado -- Class Testing

15. Constructing test cases from STD’s 2 • Suppose 6 transitions between states • Plus 1 constructor and 2 destructors • That makes 9 requirements • Select representative values • Select boundary values on each side of a transition • If the transition is guarded, select boundary values for the guard condition • Boundary values are based on the range of attribute values associated with a state ECEN5033 University of Colorado -- Class Testing

16. Top-level Statechart for Elevator Control ECEN5033 University of Colorado -- Class Testing

17. Adequacy of a Class Test Suite • Ideally – exhaustively test each class • Practically – impossible or too hard • Worth it to exhaustively test some classes with high risk • Measures of adequacy to increase confidence that we have tested enough • state-based coverage • constraint-based coverage • code-based coverage ECEN5033 University of Colorado -- Class Testing

18. State-based coverage • How many of the transitions in a state transition diagram are covered by the test suite? • “Covered” = touched at least once • May reveal each transition was covered test values do not adequately cover value ranges • If test cases were generated correctly from a state transition diagram with typical values and good boundary values, the test cases will achieve adequate state-based coverage • If test cases generated from pre- and post conditions, then it is useful to check them against the state transition diagram to ensure each transition is covered. ECEN5033 University of Colorado -- Class Testing

19. State-based coverage: object interaction • Note how operations interact w.r.t. transitions Test cases for the transition from Sc to Sd may work if Sc was reached from Sa but not if Sc was reached from Sb. “State” is a function of history. ECEN5033 University of Colorado -- Class Testing

20. State-based coverage: transition pairs • Concerning problem on previous page: • Check that the test cases cover all pairs of transitions in the state transition diagram. • In previous table, create test cases to test: • SaSc and ScSd • SbSc and ScSd ECEN5033 University of Colorado -- Class Testing

21. Statechart for Elevator Control ECEN5033 University of Colorado -- Class Testing

22. Hierarchical statechart for Elevator Control ECEN5033 University of Colorado -- Class Testing

23. Portion enlarged ECEN5033 University of Colorado -- Class Testing

24. Constraint-based coverage • How many pairs of pre- and post conditions have been covered? • Using the technique described earlier for generating test cases from pre and post conditions, if one test case is generated to satisfy each requirement, then the test suite meets this measure ECEN5033 University of Colorado -- Class Testing

25. Constraint-Based coverage: object interaction • For each operation opn that is not an accessor operation, • identify the operations op1, op2, etc. for which their preconditions are met when the post conditions for opn hold. • That is, post condition(opn) satisfies precondition(op1), etc. • Then execute test cases for operation sequences opn-op1, opn-op2, etc. ECEN5033 University of Colorado -- Class Testing

26. Code-based coverage • How much of the code that implements the class is executed across all test cases in the suite? • Minimum: Ensure that every line was executed at least once by the time all test cases (for that class) have completed execution. • Commercial tools • Add test cases to the suite if some lines have not been reached. • Might still be inadequate – interaction between methods may fail to exercise interactions between methods • Measure coverage for operation sequences ECEN5033 University of Colorado -- Class Testing

27. Test Driver Construction ECEN5033 University of Colorado -- Class Testing

28. Testing Interactions • Class Testing • Testing Interactions between objects • Identifying & specifying object interactions • Testing object interactions • Collection classes • Collaborator classes • Testing and Design Approach • Sampling • COTS testing • Patterns • Testing exceptions • Testing Class Hierarchies ECEN5033 University of Colorado -- Class Testing

29. Testing Interactions with other classes • An OO program is made of a collection of objects that collaborate to solve a problem. • Correct collaboration is necessary for program correctness. • In this section, we assume interactions are sequential, not concurrent (that’s a subject for next semester) • Focus: Ensure that messaging occurs correctly between objects whose classes have already been tested separately. • Can happen embedded in application or in special environment using a test harness ECEN5033 University of Colorado -- Class Testing

30. What is an object interaction? • Object interaction is: • request by a sender object to a receiver object • to perform one of the receiver’s operations • all of the processing performed by the receiver to complete the request • Includes messages between • an object and its components • an object and other objects with which it is associated ECEN5033 University of Colorado -- Class Testing

31. Object interaction impact • During the processing of any single method-invocation on a receiving object • Multiple object interactions can occur • Want to consider impact of these on • the internal state of the receiving object • those objects with which it has an association • Impact may be • “no change” • changes in attribute values in 1 or more objects involved • state changes in 1 or more objects • including state changes of creation or deletion of objects ECEN5033 University of Colorado -- Class Testing

32. Primitive and Non-Primitive Classes • Interactions implied in a class spec where there are references to other objects • Primitive class can be instantiated and used without needing to create any other instance • Relatively few primitive classes in a program • Since an OO program should model the objects in a problem and all the relationships between them, non-primitive classes are common & essential • Non-primitive classes require the use of other objects in some or all of their operations • Identify the classes of those objects ECEN5033 University of Colorado -- Class Testing

33. Identifying Interactions • Identify interactions by association relationships in the class diagram • regular associations • aggregation • composition • Associations translate to class interfaces by: • public operation names 1 or more classes as the type of a formal parameter • public operation names classes as the type of a return value • A class method creates an instance of another class or refers to a global instance of some class ECEN5033 University of Colorado -- Class Testing

34. Recognizing collaborators • Collaborators may be addressed • directly, e.g. using a variable name • by a pointer or a reference • dynamic type of the object may be different from the static type associated with the pointer • pointers and references are polymorphic, bound to an instance of any number of classes • Pre- and post conditions for operations in the public interface typically refer to states and/or specific attribute values of any collaborating objects ECEN5033 University of Colorado -- Class Testing

35. Collection and collaboration • Collection class • maintains associations with instances of other classes but never actually interacts with those instances • that is, never requests a service • Collaborating class • Class with more extensive interactions • Collection classes are less common ECEN5033 University of Colorado -- Class Testing

36. Testing Collection Classes • What collection classes do • store references to these objects, representing one-to-many relationships between objects in a program • create instances of these objects • delete instances of these objects • Specification • refers to other objects • does not refer to values computed based on those objects ECEN5033 University of Colorado -- Class Testing

37. Identifying Collaborating Classes • Non-primitive classes that are not collection classes • Use other objects in 1 or more operations as part of their implementation • When a post condition of an operation in a class’ interface • refers to the state of an instance of an object • and/or specifies that some attribute of that object is used or modified, that class is a collaborating class. ECEN5033 University of Colorado -- Class Testing

38. Choosing testing “chunk” size • Number of potential collaborations can grow too large very quickly • During class test: test composing object’s interactions with its composed attributes • As successive layers of aggregation are integrated, • test the interaction between an object and its associated objects • Defect visibility issue – If too large a chunk is chosen, intermediate results may be incorrect but not seen at the level of test-result verification • The more complex the objects, the fewer one should integrate prior to a round of testing ECEN5033 University of Colorado -- Class Testing

39. What factors increase object complexity? • number of parameters for each method • number of methods • number of state attributes in each object Trying to test a chunk that is too complex can result in defects that are hidden from the tests. ECEN5033 University of Colorado -- Class Testing

40. What are the implications of defensive and contract design approaches when testing interactions? ECEN5033 University of Colorado -- Class Testing

41. Specifying interactions • Remainder of slides assume • Operations defined by a class are specified by preconditions, post conditions, and class invariants • contract design approach ECEN5033 University of Colorado -- Class Testing

42. Testing Collection Classes • Collection classes are testing in ways similar to primitive classes • Test driver creates instances passed as parameters in messages to a collection • Test cases ensure that • those instances are incorporated into and • removed from the collection • address limitations placed on collection capacity • Test sequences of operations – the way modifier operations on a single object interact with one another • What additional test cases might be needed in a defensive approach? ECEN5033 University of Colorado -- Class Testing

43. Testing Collaborator Classes • More complex than testing collection or primitive • For example, if certain objects must be passed as parameters to a constructor, we can not test the constructor or the instance that requests the services of the constructor without ensuring there are instances of the objects to be passed as parameters ECEN5033 University of Colorado -- Class Testing

44. Contract More responsibility on human designer Less class-level testing due to fewer paths due to less error-checking code MORE interaction testing required to ensure human designer complied with client side of contract using precondition constraints – are preconditions in receiver met by sender “illegal” for test cases to violate preconditions Defensive More responsibility on error-checking code More class-level testing due to increased paths due to more code Once the class-level error-checking code is tested, the interaction testing is simplified since there is no need to test if preconditions are met Appropriate to violate preconditions to see if receiver catches that Contract vs. defensive approach & testing ECEN5033 University of Colorado -- Class Testing

45. Sampling • Exhaustive testing: every possible test case covering every combination of values • Not reasonable much of the time • Want to select the ones that will find the faults in which we are the most interested • Without prior information, random selection is as good as we can do • A sample is a subset of a population (e.g. all possible test cases) that has been selected based on some probability distribution ECEN5033 University of Colorado -- Class Testing

46. Use profile • Gave a way to associate each use of the system with a frequency • These can be ranked by frequency • Ranks can be turned into probabilities • The higher the frequency of use, the larger the probability of selection ECEN5033 University of Colorado -- Class Testing

47. Operational profile – stratified sample • Stratified sample is a set of samples in which each sample represents a specific subpopulation • Example • Select test cases that exercise each component of the architecture • Divide a population of tests into subsets so that a subset contains all of the tests that exercise a specific component. • Sample on each subset independent of the others • Need a basis for stratifying ECEN5033 University of Colorado -- Class Testing

48. A basis for stratification of test cases • Use the actors from the use cases • Select a sample of test cases from the uses of each actor in the use cases • Each actor uses some subset of the possible uses with some frequency; rank by frequency • Stratifying the test case samples by each actors provides a way to increase the reliability of system • Running selected tests • uses the system the way it will be used in typical situations; • finds defects likely to be found in typical use • gives largest possible increase in reliability with least effort ECEN5033 University of Colorado -- Class Testing

49. Test data generation for a range of values • Generate test data first using the specification as basis • Select values based on boundary values • just within and just on the boundaries if contract • just outside the boundaries also if defensive • Select values within intervals by sampling • Consider a random function over range 0.. N such as int (random( ) * N) which generates a pseudo random value between 0 and 1 according to a uniform distribution. • +: many values will be tested over many iterations • Need to log actual values used in case of failure so that the failed test case can be re-created. • A randomly chosen value causing failure is explicitly added to test suite and used to test the repaired software ECEN5033 University of Colorado -- Class Testing

50. Systematic sampling • Want to be able to increase level of coverage systematically. • We use boundary values coupled with sampling over ranges • We never omit boundary value test data ECEN5033 University of Colorado -- Class Testing