390 likes | 528 Views
TIME BOXED TESTING. BCS SIGIST 13 th July 1998 Graham Thomas - OSI Group. Abstract.
E N D
TIME BOXED TESTING BCS SIGIST 13th July 1998 Graham Thomas - OSI Group
Abstract There is great pressure upon developers today to improve productivity and effectiveness. To achieve this there is a move away from the traditional structured methodologies towards more dynamic, iterative and RAD approaches. This is being combined with Object and Component based techniques, and delivered with a new generation of IDE’s, to produce thin client, web based, voice and data products. So how do we test it? • Propose Time Boxed Testing • Outline an approach • Discuss the problems • Give a flavour of the fun ?
Agenda • Development Lifecycle • Testing Lifecycle • Testing Helix • Testing Stage by Stage • Traceability • Testing Process • How to sell time boxed testing • Lessons Learnt
The Project • Telephony based project • 2 types of proprietary hardware • Client Server running Windows NT • Interfaces to legacy systems • Rational Rose case tool (using UML) • Objectory process • Visual Studio • Was considering Java & thin clients • Incremental/Iterative development
Rqmts. UAT Sys Test Analysis Depth Design Int Test Build Unit Test Breadth Classic V Model Iterative (RAD) Model Incompatible Models?
Component Integration unit unit Component Integration Depth unit Component Integration System Breadth Acceptance Difficult to Represent Test Stages
Project Lifecycle ¬ ® . . .
Testing Lifecycle ¬ ® . . .
A dual spiral life cycle ? ¬ ® R B . . . P E A D Rqmts Analysis Design Build D C Plan Design Construct Execute
Testing Lifecycle • Unit Testing • Component Integration Testing • System Testing • Acceptance Testing • Year 2000 Millenium Compliance • Testing Processes and Activities • Tools & Traceability • Resources • Planning
Scope of Test Phases Conical Model Component Integration Testing Unit Testing User Acceptance Testing System Testing Release Iterative
Tuned Test Lifecycle Testing Activities Brought Forward System Testing - Non functional Requirements - Performance Analysis User Acceptance Testing - Scenario Testing - Acceptance Criteria Component Integration Testing Unit Testing Release Iterative
Sub Unit Testing Unit Testing Unit Integration Testing Unit Testing External Interface Testing Static Analysis Dynamic Analysis Quality Assurance Unit Testing • Iterative Time-Boxed Activity • Specific to delivered components • ‘White Box’ testing techniques employed • External Interfaces tested • Use of tools for Static & Dynamic analysis
Unit Package Sub Unit Unit Sub Unit Sub Unit Package Sub Unit Unit Unit Sub Unit Sub Unit Unit Sub Unit Sub Unit Package Sub Unit Unit Unit Integration Testing Sub Unit Testing Unit Testing Previously tested & checked-in Stubs, Drivers, Harnesses Unit Testing
Package Integration Component Integration Performance Analysis Component Integration Testing Non Functional Requirements Functional Requirements Regression Test Scenario Regression Test Component Integration Testing • Iterative Time-Boxed Activity • Testing prioritised by Business • End to End process threads where possible • Contains elements of following test phases • Build regression test packs
Component Package Package Component Package Package Component Component Integration Testing Step 1 - Package Integration Package Integration must precede Component Integration
Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Drop1 Drop2 . . . Component Integration Testing Step 2 - Component Integration
® ¬ Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component . . . Component Integration Testing Drop by Drop Automated Regression Test Pack
Non Functional Requirements Functional Requirements Regression Test System Testing Performance Analysis Design Verification System Testing • One off activity • Structured testing techniques • End to End process threads • Ties together the development iterations • Regression test already constructed
Scenario Regression Testing Acceptance Criteria Testing Acceptance Testing Business Process Testing Data Take On & Conversion Testing Usability Testing Acceptance Testing • One off activity • Test that the computer system integrates with business processes • Structured testing techniques • End to End process threads • Regression test already constructed
Y2000 - Millennium Compliance 6 Date Scenarios • Separate test runs • Carried out in sequence • Some tests can be exercised post Y2000 Across Century Boundary Wholly next Century This Century Jan 1st Feb 29th Dec 31st 2 Business Tests Financial Year ? Fiscal Periods ?
User Requirements Development Repository TestPlanning Testing Lifecycle Traceability High Level Function Functional Requirement Non FR Use Case Scenario Test Script Test Case
Process Quality Interface Problem Mgmt Conf. Mgmt. Interface Activity Entry Testing Processes and Activities Test Management Test Stage Sign off Test Execution Test Automation Regression Test Test Reporting Monitor Measure Manage Exit Test Planning Test Design Test Preparation Analysis of Change
Processes • Essential • Configuration Management • Impact Analysis using a Test Planning repository • Quality Management • Good practice • Fault Reporting - on the Intranet - Live • Gathering metric information
Tools • Full set of testing tools is required • Static analysis • Dynamic Analysis • Interactive debugging • Test Automation/Capture Replay essential • Test Planning Repository • Performance monitoring (stress/load) • Fault Reporting system
Test Resources Component Integration Testing User Acceptance Testing Unit Testing System Testing TEST TEAM Business Users Business Users Developers Developers Testers Secondment
Test Approach Decision Test Strategy Definition Testing Process Test drop ¬ Test drop Development Test drop ® Non Functional Testing System Test Acceptance Test Delivery Y2000 Test Time-boxed Testing Test Planning
Selling the Approach • Target the following; • Test Team • Systems Architect • Development team • Standards & Methods • Business Representatives • Project Sponsor • Via • Animated Presentation • Test Strategy document
So What went well? • Systems Architect liked it • The presentation was well received
What was difficult? • Initial cool response from other members of the test team • Development Team were uneasy • Dev. Why test anyway, our code is wonderful and our test harnesses will do all the testing? • Business Representatives were unhappy about; • Anything less than 100% testing • Bringing their effort forward in the development lifecycle • The difference between the ‘Use Case’ modeling they had done, and the test planning that they would need to carry out
What was difficult? (2) • Standards & Methods Saw nothing wrong with their approach, it was either; • structured in which case their approach worked fine, or • it was incremental, in which case their approach worked fine. • They did not do iterative development, but if they did their approach would still work. This took several meetings to gain approval. • Project Sponsor • Cancelled the project. (Time Boxed Testing was not cited as the reason !!!)
What did I learn from this? • Organisations change their development approach more quickly than their testing approach • There is a reluctance to do anything less than 100% testing • Testing is a key and integral part of any development lifecycle • The more you think about it, the more sense Time BoxedTesting makes
Contact Information Graham Thomas OSI the management support company e-mail : gthomas@osi.co.uk Phone : 0171 323 3353 Copies of slides including animated gifs at www.osi.co.uk