Blackbox Testing

1. Blackbox Testing Equivalence Partitioning:The ATM Simulation System

2. Software Testing • “The” Testing Quote Process of executing a program aiming to reveal defects or, failing in this objective, to increase the confidence that the program is correct. Testing cannot show the absence of defects, it can only show that software defects are present. We cannot test quality into a product,we have to design it in.

3. Test Objectives • Tests intended to find errors • Good test cases have high probability for finding a yet undiscovered error • Successful tests cause program failure, i.e. find an undiscovered error • Minimal set of test cases needs to be developed • Exhaustive testing (test the program with all possible values of input data) is not possible!!

4. Where does testing fit in? LEVEL 0: Testing == Debugging LEVEL 1: Testing shows the software works LEVEL 2: Testing shows the software doesn’t work LEVEL 3: Testing doesn’t prove anything, it reduces risk of unacceptable software delivery LEVEL 4: Testing is not an act, but a mental discipline of quality

5. Terminology • Test Case • Specification of an input of the program and its correspondent expected result • Test Suite • Set of test cases • Testing Criterion • Systematic way to generate test cases and/or evaluate their effectiveness in revealing errors • Based upon a testing technique

6. Testing “Joe the Box” • joe’s size is 50 • What would happen if we made: • joe grow: -60 (try it three times!!) • If we found no errors: High-quality software??? Low-quality testing???

7. Test Activities • Planning • Test Case Design • Test Case Execution • Result Analysis

8. Testing Techniques • Functional – Blackbox Testing • Uses the functional requirements (specification)to derive test cases • Equivalence Partitioning, Boundary Value Analysis • Structural – Whitebox Testing • Uses the internal structure of the program (implementation details) to derive test cases • Control-flow, Data-flow • Error-Based • Uses information about the most common errors committed during the software development process • Mutation Analysis

9. Equivalence Partitioning • Divide input domain into classes of data • Equivalence classes • Single test case can cover all “equivalent” data elements • Partitions consist of valid and invalid sets • Equivalence Class • A set of valid or invalid states for input conditions • Input Condition: usually a sentence or a clause from the specification

10. ATM Simulation System • Specification (simplified) • The customer will be required to enter the account number and a PIN. • There is no need of an ATM card. • The ATM must provide the following transactions to the customer: • Cash Withdrawals, Deposits, Tranfers and Balance Inquiries. • Only one transaction will be allowed in each session. • The ATM will communicate the transaction to the bank and obtain verification that it was allowed by the bank. • If the bank determines that account number or PIN is invalid, the transaction is canceled. • The ATM will have an operator panel that will allow an operator to start and stop the servicing of customers. • When the machine is shut down, the operator may remove deposit envelopes and reload the machine with cash. • The operator will be required to enter the total cash on hand before starting the system from this panel.

11. ATM Simulation System • Classes ATM CashDispenser EnvelopeAcceptor OperatorPanel Session Transaction WithdrawalTransaction DepositTransaction TransferTransaction InquiryTransaction Bank • Some Scenarios • System Startup • Session • Cash Withdrawal Transaction • Deposit Transaction • Transfer Transaction • Balance Inquiry • CRC Cards...

12. Defining Input Conditions Analyse the scenarios and the set of responsibilities for each class/CRC card. 1.1. Consider the responsibilities in a macroscopic way. • Focus on what each class really has to do. 1.2. Look for the input data provided by the user. • Analyse the responsibilities of the collaborators and of the other classes too. • Sometimes the input data are, in fact, being treated by some of the collaborators. 1.3. Write down all the input data for the CRC you are analysing.

13. CRC Cards • Class: ATM Responsibility Collaborator • Start up system operation (initial cash) OperatorPanel • Start a session for each customer Session • Shut down on operator request OperatorPanel • Get PIN from the customer • Get transaction choice from the customer • Get account from the customer • Get amount entry from the customer (Class xxxTransaction) • Verify the availability of cash for withdrawal CashDispenser • Dispense cash CashDispenser • Accept deposit envelope EnvelopeAcceptor

14. CRC Cards • Class: CashDispenser Responsibility Collaborator • Set initial cash on hand at startup (Operator Panel) • Report cash available • Dispense cash • Class: EnvelopeAcceptor Responsibility Collaborator • Accept deposit envelope • Class: OperatorPanel Responsibility Collaborator • Get initial cash on hand from operator

15. CRC Cards • Class: Session Responsibility Collaborator • Perform session use case ATM, Transaction • Perform invalid data ATM, Bank • Furnish account to Transaction • Furnish PIN to Transaction • Class: Transaction Responsibility Collaborator • Perform a particular transaction use case WithdrawalTransaction, DepositTransaction, TransferTransaction, InquiryTransaction, Session, Bank

16. CRC Cards • Class: WithdrawalTransaction Responsibility Collaborator • Get specifics from customer ATM • Send to bank Session, Bank • Dispense cash and notify bank when complete ATM, Bank • Class: DepositTransaction Responsibility Collaborator • Get specifics from customer ATM • Send to bank Session, Bank • Accept envelope and notify bank when complete ATM, Bank • Class: TransferTransaction Responsibility Collaborator • Get specifics from customer ATM • Send to bank Session, Bank

17. CRC Cards • Class: InquiryTransaction Responsibility Collaborator • Get specifics from customer ATM • Send to bank Session, Bank • Class: Bank Responsibility Collaborator • Initiate withdrawal • Finish withdrawal • Initiate deposit • Finish deposit • Do transfer • Do inquiry

18. Defining Input Conditions • Input Data • Class ATM: account, PIN, transaction • Class OperatorPanel: initial cash • Class WithdrawalTransaction: amount • Class DepositTransaction: amount • Class TransferTransaction: receiver account, amount

19. Defining Input Conditions 2. Identify the main characteristics of each input data you found. 3. Identify the “interactions” among the input data. 3.1. Think about how a specific input can be related to the other inputs of the system. • Focus on the system’s operation. • Scenarios can help you. Input Conditions !!!

20. Defining Input Conditions • Class ATM • account • matching with bank • PIN • matching with account • transaction • type • Class OperatorPanel • initial cash • valid characters • availability of money in cash dispenser

21. Defining Input Conditions • Class WithdrawalTransaction • amount • valid characters • availability of money in the account for withdrawal • Class DepositTransaction • amount • valid characters • Class TransferTransaction • receiver account • matching with bank • amount • valid values • availability of money in the sender account for transfer

22. Identifying Equivalence Classes 1. Analyse each input condition you defined. 1.1.Think about the valid values to satisfy the input condition. If necessary, refine the input condition into more details. • They will correspond to the elements of aValid Equivalence Class. 1.2. Think about other possible values associated to the input condition – invalid values. • They will correspond to the elements of aInvalid Equivalence Class. 2. Enumerate each equivalence class. • Assign a unique number to each class.

23. Identifying Equivalence Classes Class ATM Input ConditionValid Invalid account matches with bank size (a) of account a = 8 (1) a> 8 (2) 0  a< 8 (3) valid characters for account numerical (4) alpha, special (5) PIN matches with account size (p) of PIN p = 4 (6) p> 4 (7) 0  p< 4 (8) valid characters for PIN numerical (9) alpha, special (10) type of transaction withdrawal (11) none (“blank choice”) (15) deposit (12) transfer (13) inquiry (14)

24. Identifying Equivalence Classes Class WithdrawalTransaction Input ConditionValid Invalid valid characters for amount numerical (1) alpha, special (2) availability of money balance - amount  0 (3) balance - amount < 0 (4) in the account for withdrawal

25. Defining the Test Cases 1. Derive test cases associated with valid classes to cover all of them • A test case should cover a number of valid equivalence classes, as large as possible 2. Derive test cases associated with invalid classes to cover all of them • A test case should be written for each invalid equivalence class 3. Write a “specification” for your test cases 4. Write “the” test cases

26. Defining the Test Cases Class Equivalence Class # Test Case Actual Result Expected Result WithdrawalTransaction (3) Balance: US$ 100,00 Amount: US$ 90,00 Transaction Canceled Transaction OK Balance: US$ 10,00 Error: Debugging • Example of a “specification” for a test case Actual Result: fill in after the test case execution

27. Our “Testing Process” The testing activities should start in the analysis phase, going through design and implementation phases. • OO Analysis (CRC cards and Scenarios) • Definition of input conditions • Identification of equivalence classes • OO Design (UML class diagrams) • Definition of test cases • OO Implementation (Squeak) • Execution of the test cases against the system • Use the SUnit framework • Check of the obtained results Interative Process !!! You can always return to the previous activities if necessary.

28. At the End… • A well-documented test suite • Basis for regression tests • New features or changes are added to the system • Agreement with eXtreme Programming philosophy Build a little, test a little. Test first, code later.