1 / 39

TIME BOXED TESTING

TIME BOXED TESTING. BCS SIGIST 13 th July 1998 Graham Thomas - OSI Group. Abstract.

neviah
Download Presentation

TIME BOXED TESTING

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TIME BOXED TESTING BCS SIGIST 13th July 1998 Graham Thomas - OSI Group

  2. Abstract There is great pressure upon developers today to improve productivity and effectiveness. To achieve this there is a move away from the traditional structured methodologies towards more dynamic, iterative and RAD approaches. This is being combined with Object and Component based techniques, and delivered with a new generation of IDE’s, to produce thin client, web based, voice and data products. So how do we test it? • Propose Time Boxed Testing • Outline an approach • Discuss the problems • Give a flavour of the fun ?

  3. Agenda • Development Lifecycle • Testing Lifecycle • Testing Helix • Testing Stage by Stage • Traceability • Testing Process • How to sell time boxed testing • Lessons Learnt

  4. The Project • Telephony based project • 2 types of proprietary hardware • Client Server running Windows NT • Interfaces to legacy systems • Rational Rose case tool (using UML) • Objectory process • Visual Studio • Was considering Java & thin clients • Incremental/Iterative development

  5. Rqmts. UAT Sys Test Analysis Depth Design Int Test Build Unit Test Breadth Classic V Model Iterative (RAD) Model Incompatible Models?

  6. Component Integration unit unit Component Integration Depth unit Component Integration System Breadth Acceptance Difficult to Represent Test Stages

  7. Project Lifecycle ¬ ­ ® . . .

  8. Testing Lifecycle ¬ ­ ® . . .

  9. A dual spiral life cycle ? ¬ ­ ® R B . . . P E A D Rqmts Analysis Design Build D C Plan Design Construct Execute

  10. Testing Lifecycle • Unit Testing • Component Integration Testing • System Testing • Acceptance Testing • Year 2000 Millenium Compliance • Testing Processes and Activities • Tools & Traceability • Resources • Planning

  11. Scope of Test Phases Conical Model Component Integration Testing Unit Testing User Acceptance Testing System Testing Release Iterative

  12. Tuned Test Lifecycle Testing Activities Brought Forward System Testing - Non functional Requirements - Performance Analysis User Acceptance Testing - Scenario Testing - Acceptance Criteria Component Integration Testing Unit Testing Release Iterative

  13. Sub Unit Testing Unit Testing Unit Integration Testing Unit Testing External Interface Testing Static Analysis Dynamic Analysis Quality Assurance Unit Testing • Iterative Time-Boxed Activity • Specific to delivered components • ‘White Box’ testing techniques employed • External Interfaces tested • Use of tools for Static & Dynamic analysis

  14. Unit Package Sub Unit Unit Sub Unit Sub Unit Package Sub Unit Unit Unit Sub Unit Sub Unit Unit Sub Unit Sub Unit Package Sub Unit Unit Unit Integration Testing Sub Unit Testing Unit Testing Previously tested & checked-in Stubs, Drivers, Harnesses Unit Testing

  15. Package Integration Component Integration Performance Analysis Component Integration Testing Non Functional Requirements Functional Requirements Regression Test Scenario Regression Test Component Integration Testing • Iterative Time-Boxed Activity • Testing prioritised by Business • End to End process threads where possible • Contains elements of following test phases • Build regression test packs

  16. Component Package Package Component Package Package Component Component Integration Testing Step 1 - Package Integration Package Integration must precede Component Integration

  17. Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Drop1 Drop2 . . . Component Integration Testing Step 2 - Component Integration

  18. ® ­ ¬ Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component Component . . . Component Integration Testing Drop by Drop Automated Regression Test Pack

  19. Non Functional Requirements Functional Requirements Regression Test System Testing Performance Analysis Design Verification System Testing • One off activity • Structured testing techniques • End to End process threads • Ties together the development iterations • Regression test already constructed

  20. Scenario Regression Testing Acceptance Criteria Testing Acceptance Testing Business Process Testing Data Take On & Conversion Testing Usability Testing Acceptance Testing • One off activity • Test that the computer system integrates with business processes • Structured testing techniques • End to End process threads • Regression test already constructed

  21. Y2000 - Millennium Compliance 6 Date Scenarios • Separate test runs • Carried out in sequence • Some tests can be exercised post Y2000 Across Century Boundary Wholly next Century This Century Jan 1st Feb 29th Dec 31st 2 Business Tests Financial Year ? Fiscal Periods ?

  22. User Requirements Development Repository TestPlanning Testing Lifecycle Traceability High Level Function Functional Requirement Non FR Use Case Scenario Test Script Test Case

  23. Process Quality Interface Problem Mgmt Conf. Mgmt. Interface Activity Entry Testing Processes and Activities Test Management Test Stage Sign off Test Execution Test Automation Regression Test Test Reporting Monitor Measure Manage Exit Test Planning Test Design Test Preparation Analysis of Change

  24. Processes • Essential • Configuration Management • Impact Analysis using a Test Planning repository • Quality Management • Good practice • Fault Reporting - on the Intranet - Live • Gathering metric information

  25. Tools • Full set of testing tools is required • Static analysis • Dynamic Analysis • Interactive debugging • Test Automation/Capture Replay essential • Test Planning Repository • Performance monitoring (stress/load) • Fault Reporting system

  26. Monitoring

  27. Monitoring

  28. Monitoring

  29. Monitoring

  30. Test Resources Component Integration Testing User Acceptance Testing Unit Testing System Testing TEST TEAM Business Users Business Users Developers Developers Testers Secondment

  31. Test Approach Decision Test Strategy Definition Testing Process Test drop ¬ Test drop ­ Development Test drop ® Non Functional Testing System Test Acceptance Test Delivery Y2000 Test Time-boxed Testing Test Planning

  32. Selling the Approach • Target the following; • Test Team • Systems Architect • Development team • Standards & Methods • Business Representatives • Project Sponsor • Via • Animated Presentation • Test Strategy document

  33. So how easy was this?

  34. So What went well? • Systems Architect liked it • The presentation was well received

  35. What was difficult? • Initial cool response from other members of the test team • Development Team were uneasy • Dev. Why test anyway, our code is wonderful and our test harnesses will do all the testing? • Business Representatives were unhappy about; • Anything less than 100% testing • Bringing their effort forward in the development lifecycle • The difference between the ‘Use Case’ modeling they had done, and the test planning that they would need to carry out

  36. What was difficult? (2) • Standards & Methods Saw nothing wrong with their approach, it was either; • structured in which case their approach worked fine, or • it was incremental, in which case their approach worked fine. • They did not do iterative development, but if they did their approach would still work. This took several meetings to gain approval. • Project Sponsor • Cancelled the project. (Time Boxed Testing was not cited as the reason !!!)

  37. So what fun did we have ?

  38. What did I learn from this? • Organisations change their development approach more quickly than their testing approach • There is a reluctance to do anything less than 100% testing • Testing is a key and integral part of any development lifecycle • The more you think about it, the more sense Time BoxedTesting makes

  39. Contact Information Graham Thomas OSI the management support company e-mail : gthomas@osi.co.uk Phone : 0171 323 3353 Copies of slides including animated gifs at www.osi.co.uk

More Related