Software Testing

1. Software Testing Department of Computing Science University of Alberta CMPUT 402 Software Engineering

2. Outlines • Testing overview • Testing object-oriented systems • Testing object-oriented frameworks • Testing object-oriented framework instantiations.

3. Testing Overview

4. Introduction • Software testing • Verification activity • Time consuming and costly. • No correctness proof. • Increase user confidence

5. Testing Phases • Specification testing • Design testing • Implementation testing (software testing) • Dynamic • Static

6. Test Suite … Test case Software test plan • Document to explain testing approach Test case

7. Elements of a Test Case • Reference number • Description • Preconditions • Input • Expected output

8. Test Case (TC) Example TC#:S-221 TC Description:This test is to check the response to an invalid input selection. TC Precondition:Go to screen Flight_Management TC Input:Enter <F7> TC Expected Output:Error message: “Invalid input, Enter “1” to “4” or ESC”

9. Implementation testing levels • Unit testing • Integration testing • System testing • Regression testing

10. Unit testing • Module testing • Testing approaches • Black-box • White-box

11. Example: problem statement Given two integers, find the larger. If larger is less than 1, return 1.

12. Example: Black-box testing Given two integers, find the largest. If largest is less than 1, return 1.

13. White-box testing • Statement coverage • Graph based • Branch coverage • Condition coverage • Path coverage

14. Example: Statement Coverage 1 int maxPositive(int n1, int n2) { 2 if (n1<=1 && n2<=1) 3 n1=1; 4 if (n1<=n2) 5 n1=n2; 6 return n1; } TC Inputs: n1=n2=1

15. 1,2 3 4 5 6 TC#1 Inputs: n1=n2=1 TC#2 Inputs: n1=2, n2=0 Example: Branch Coverage 1 int maxPositive(int n1, int n2) { 2 if (n1<=1 && n2<=1) 3 n1=1; 4 if (n1<=n2) 5 n1=n2; 6 return n1; }

16. TC#3 Inputs: n1=0, n2=2 Example: Condition Coverage 1 int maxPositive(int n1, int n2) { 2 if (n1<=1 && n2<=1) 3 n1=1; 4 if (n1<=n2) 5 n1=n2; 6 return n1; } 1,2 3 4 5 TC#1 Inputs: n1=n2=1 6 TC#2 Inputs: n1=2, n2=0

17. TC#4 Inputs: n1=n2=0 Example: Path Coverage 1 int maxPositive(int n1, int n2) { 2 if (n1<=1 && n2<=1) 3 n1=1; 4 if (n1<=n2) 5 n1=n2; 6 return n1; } 1,2 3 4 5 TC#1 Inputs: n1=n2=1 6 TC#2 Inputs: n1=2, n2=0 TC#3 Inputs: n1=0, n2=2

18. Integration Testing • Non-incremental (big-bang) • Incremental • Top-down (stubs needed) • Bottom-up (drivers needed) • Sandwich

19. System Testing • Performed in terms of Inputs and outputs of the system. • Performed on the targeted platform. • Goals: • Reveal bugs that are represented only at system level. • Demonstrate that the system implements all required capabilities. • Answer the question: Can we ship it yet?

20. Regression Testing • Ensuring that modifications have not caused unintended effects. • Retesting is required

21. Testing Object-Oriented Systems

22. Object-oriented testing • No sequential procedural executions. • No functional decomposition. • Unique problems • Encapsulation • Inheritance • Polymorphism • Why it is different?

23. Object-oriented testing levels • Method testing. • Class testing. • Cluster testing. • System testing. • Regression testing

24. Class testing • Tests classes by sending messages to methods one at a time. • Requires drivers and stubs. • May have to test more than one method at a time to test the collaboration of methods. • Inherited method have to be tested.

25. Alpha-Omega Cycle Class Testing • Alpha state: object declaration state. • Omega state: object destruction state. • Alpha-omega cycle testing: takes objects from alpha to omega states by sending messages to every method at least once. • Method ordering (constructor, accessor, predicate, modifier, destructor)

26. Example Class person { person(String n, int a) { name=n; age=a; } constructor void setName(String n) { name=n; } modifier void setAge(int a) { age=a; } modifier String getName() { return name; } accessor int getAge() { return age; } accessor int isOld() { if age>50 return 1; return 0;} predicate String name; int age; }

27. Example (Cont’d) Class driver { … void test() { person Tom= new Tom(“Tom”, 20); String name=Tom.getName(); int a=Tom.getAge(); if (Tom.isOld()) { … } setName(“old Tom!!”); setAge(60); … } … }

28. Cluster testing • Cluster: collection of classes cooperating with each other via messages. • Goal: test the interaction among the objects of the classes that forms the cluster. • Techniques: • Message Quiescence • Event Quiescence

29. Message Quiescence • Method/Method path (MM-Path): sequence of method executions linked by messages. • Starts with a method and ends with another method that does not issue a message.

30. MM-Paths 1 2 3 MM-Path Message Event Class 3 Class 1 meth2 meth1 meth1 meth2 meth3 Class 2 meth1 meth3 meth2

31. Event Quiescence • Atomic System Function (ASF): MM-Path Input Event Output Event

32. INPUT PORT EVENT A A B B OUTPUT PORT EVENT Class 3 Class 1 meth2 meth1 1 meth1 meth2 meth3 INPUT PORT EVENT OUTPUT PORT EVENT Class 2 2 meth1 meth3 3 MM-Path meth2 Message Event

33. Example • ATM pin entry • Customer enters card (event). • Card is validated. • Screen requesting pin entry is displayed. • Digits are touched and displayed. • Pin is checked.

34. ASF Starts here CardSlot Screen memberCard showMessage ValidateCard Security NumKeypad checkPin getKeyEvents parseKeyEvent ASF ends here Customer inserts card Message is displayed Key pushers

35. Testing Object-Oriented Frameworks

36. Definitions • OO-Framework • Reusable design and implementation of a system or subsystem [Beck+ 94] • Design + code + hooks • Hooks • Places at which users can add their own components.

37. Importance • Why do we have to care about testing frameworks? • Framework defects passes on to all instantiations. • Hard to discover framework defects at the instantiation development stages. • Little research has been carried out.

38. Framework testing • Components (classes and small clusters) • black-box • class associations • Hooks • Compatibility and completeness • Problem Solving

39. Testing OO-framework Problems • Testing framework implementation • Developing testable model for the framework specifications. • Developing a test suite generator technique. • Testing framework hooks • …

40. Testing OO-framework Problems (Cont’d) • Testing framework hooks • Formalizing hook requirements. • Developing testable model for hook methods. • Integrating hook method model with framework specification model. • Extending framework implementation test suite generator technique. • Developing an approach to test hook compatibility and completeness

41. Testing Object-Oriented Framework Instantiations

42. Framework instantiation parts • Framework used components. • Interface components. • Other instantiation components

43. Importance • Why do we have to deal with framework instantiations in a special way? • Part of the code is tested. • Part of the code is not needed. • Part of the code is extended.

44. Testing framework instantiation problems • Testing the used portion of the framework code. • Testing the extensibility of the framework. • Testing the framework interfaces. • Testing other instantiation components. • Integration testing

45. Summary • Testing importance • OO-technology effects testing. • OO-Framework technology effect testing.

46. References • G. Myers, The art of software testing, Wiley-Interscience, 1979. • B. Beizer, Software Testing Techniques, Van Nostrand reinhold, 2nd edition, 1990. • R. Binder, Testing object-oriented systems, Addison Wesley, 1999. • M. Hanna, Testing object-oriented systems for QA professionals, PSQT/PSTT 2000 North Minneapolis, MN, conference tutorial H4, 2000. • M. Petchiny, Object-oriented testing, http://www.cs.queensu.ca/home/shepard/599.dir/petchiny/OOPRESENTATION.ppt