410 likes | 770 Views
Object-Oriented Software Testing. ICS 224A Joachim Feise jfeise@ics.uci.edu. A Brief Meditation on the Essential Nature of Software Testing. Object-Oriented Testing Papers. Object-Oriented Integration Testing Jorgensen and Erickson CACM Sept. 1994
E N D
Object-Oriented Software Testing ICS 224A Joachim Feise jfeise@ics.uci.edu
A Brief Meditation on the Essential Nature of Software Testing OO Software Testing
Object-Oriented Testing Papers • Object-Oriented Integration Testing • Jorgensen and Erickson • CACM Sept. 1994 • Object-Oriented Software Testing - A Hierarchical Approach • Shel Siegel • J. Wiley and Sons, 1996 OO Software Testing
Structure vs. Behavior • most popular notations in software development portray software structure • examples: E/R models, data flow diagrams, structure charts, etc. • used to show relationships between components, interfaces, control and dataflow, etc. • not very useful for testing • SW testing is fundamentally concerned with behavior, i.e. what it does OO Software Testing
Traditional Testing Notions • Traditional software is • written in an imperative language • described by functional decomposition • developed in a waterfall life cycle • separated into three levels of testing • unit level, integration level, system level • these points do often not apply directly to object-oriented software OO Software Testing
Imperative vs. Declarative • Imperative languages lend themselves to a description as a directed graph • declarative languages suppress sequentiality • the event-driven nature of OO systems forces a “declarative spirit” on testing • not evident at unit level since most object-oriented languages are imperative • but pronounced at integration and system levels OO Software Testing
Unit Testing Overview • Units can be defined in several ways • a single, cohesive function • a function which, when coded fits on one page • the smallest separately compilable segment of code • the amount of code that can be written in 4 to 40 hours • a task in a work breakdown structure • code that is assigned to one person • code that one person designs, codes and test in a three-month period OO Software Testing
Unit Testing Overview (cont.) • Many organizations have not chosen their definition of a software unit • Units are always tested “by itself” • stubs and drivers are used to emulate inputs and outputs • goal: verification that, taken by itself, the unit functions correctly OO Software Testing
Integration Testing Overview • Once the units are separately tested, they are integrated together • gradual replacement of stubs and drivers by separately tested units • pairwise integration, i.e. each unit is integrated with its adjacent units • bottom-up integration guided by functional decomposition tree • “big bang” integration where all units are thrown together at once OO Software Testing
System Testing Overview • Conducted exclusively in terms of inputs and outputs that are visible at the port boundary of a system • Tester only has access to those port events that are visible to the customer/user • Can occur on the target platform only OO Software Testing
System Testing Overview (cont.) • Examples of system testing activities • sequences of machine instructions • sequences of source instructions • scenarios of normal usage • system-level test cases • stimulus/response pairs • behavior that results from sequences of system-level inputs • sequences of transitions in a state machine description of the system OO Software Testing
Object-Oriented Software • within an object, individual methods are imperative • declarative aspect because of event-driven nature • dynamic binding creates indefiniteness that resembles declarative programs • it is impossible to ever know the full set of “adjacent” objects with which a given object may be composed • two object may be correct, but errors might result when they are composed OO Software Testing
Composing Errors Illustrated OO Software Testing
Levels of OO Testing • Unit testing • method testing • Integration testing • Message quiescence • Event quiescence • System testing • Thread testing • Thread interaction testing OO Software Testing
Integration Testing • Method executions are linked by messages • Quiescence provides natural endpoints • Message quiescence • Method/Message path (MM-Path) • sequence of message executions linked by messages • starts with a method • ends when method is reached that does not issue any messages of its own • endpoint is called message quiescence OO Software Testing
Integration Testing (cont.) • Atomic System Function (ASF) • input port event, • followed by a set of MM-Paths, • terminated by output port event • ASFs constitute the point at which integration and system testing meet • elemental function visible at the system level OO Software Testing
MM-Paths and ASFs Illustrated OO Software Testing
ATM System Class Hierarchy OO Software Testing
Example: ATM System • Examples of • MM-Path: Digit entry • ASF: PIN Entry • Thread (sequence of ASFs): • simple transaction: PIN entry, transaction type selection, account detail presentation, operation execution, result reporting • Thread sequence: • a session consisting of multiple simple transactions OO Software Testing
Digit Entry MM-Path Illustrated OO Software Testing
PIN Entry ASF Illustrated OO Software Testing
Observations • Many-to-many relationships dominate (see E/R diagram) • an object may be involved in many threads • threads entail many objects • an object may be involved with many ASFs • ASFs may entail many objects OO Software Testing
Observations (cont.) • The five levels of OO testing result in distinct, useful testing goals • Bottom-up testing order • Seamless juncture between OO integration testing and system testing • Constructs are directly usable as basis for test coverage metrics • Works well with composition OO Software Testing
The Hierarchical Approach • Everything is a system • Defines and applies testing standards for several levels of software components • objects • classes • foundation components • systems OO Software Testing
Hierarchical Approach Illustrated OO Software Testing
Safe Components • Components that meet the testing standards for that kind of components • only safe components can be integrated with other safe components to produce the next-level component • Safe is a relative state • depends on risk management practices, attitude toward risk, what is enforced, the application, etc. OO Software Testing
Foundation Components • Can be one complete class hierarchy • or some cluster of classes • that performs a core function, • that represents a logical or physical architectural component, etc. • After testing to a safe level, can be integrated with other foundation components OO Software Testing
Integration Testing of Foundation Components • Needs to address the interconnections between the foundation components and any new composite functionality ONLY • Eliminates the need to test all of the combinations of states during integration testing • Improves productivity OO Software Testing
Integration Testing Problem Illustrated OO Software Testing
Test Suites • Conditional Test Suite • Hierarchical Incremental Test Suite • Integration Test Suite • System Test Suite • Regression Test Suite OO Software Testing
Conditional Test Suite • Four types of test scripts • Assertions: abort if condition fails • Exceptions: raise exception if condition succeeds • Concurrent test operations: monitor condition • Message polling: evaluate condition on demand • Conditions • class invariant, precondition, postcondition, loop invariant OO Software Testing
Pre/Postconditions Illustrated OO Software Testing
Hierarchical Incremental Test Suite • Structural models • control-flow model • models class as series of control-flow graphs deriving from the method implementations • data-flow model • models class as series of control-flow graphs with data annotations showing the flow of data values through the method implementations OO Software Testing
Hierarchical Incremental Test Suite (cont.) • Functional models • state-transition model, aka OMT dynamic model • models states and transitions for a class in the context of its position in the inheritance hierarchy • transaction-flow model • models object life cycle as transaction from construction to destruction • exception model • specifies what methods raise what exceptions under what conditions OO Software Testing
Integration Test Suite • Functional Test Suite • tests vertical integration of a cluster of components, i.e. the way the cluster implements specific functions from the requirements • Abstraction Test Suite • tests horizontal integration of a cluster of components, i.e. the way the cluster abstracts a service contract into an API OO Software Testing
Integration Test Suite (cont.) • Possible test scripts • class-to-class test script • interaction between classes based on message relationship • protocol-interface, synchronization, exception handling tests • UI test script • tests UI components of cluster and the components on which they depend • true-time, character-recognition, widget-playback tests OO Software Testing
System Test Suite • System Acceptance Test Suite • derived from the set of use cases covering the system requirements • User Options Test Suite • based on probable usage patterns of the features and functions delivered with the system • designed to simulate the way the end users will combine the features and functions OO Software Testing
Regression Test Suite • Series of tests run against a previous version of the CUT • Almost always automated • performing manual tests is usually not cost effective OO Software Testing
Epilog • “Testing proves the presence, not the absence, of bugs.” • E.W. Dijkstra • “Absence of evidence is not evidence of absence.” • Source unknown • From: E.V. Berard, Issues in the Testing of OO Software • Slides available at http://www.ics.uci.edu/~jfeise/ICS224A OO Software Testing