1 / 34

Testing Workflow

Testing Workflow. In the Unified Process and Agile/Scrum processes. Testing Workflow: Purpose. Verify the result from implementation by testing each build Plan the tests in each iteration Integration tests for every build within the iteration System tests for the end of the iteration

aziza
Download Presentation

Testing Workflow

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing Workflow In the Unified Process and Agile/Scrum processes

  2. Testing Workflow: Purpose • Verify the result from implementation by testing each build • Plan the tests in each iteration • Integration tests for every build within the iteration • System tests for the end of the iteration • Design and implement tests by creating • test cases that specify what to test • test procedures that specify how to perform the tests • executable test components to automate the tests

  3. Review: Kinds of testing • Unit testing • Does an individual software component (unit) live up to its contract? • Typically done by the same person who codes the given unit • Tests become part of regression test suite • Integration testing • System testing • Regression testing

  4. Where do unit tests fit into UP? • Unit tests are part of the implementation workflow • They come before the integration testing • They are typically designed and written by the programmer who writes the given module/unit • (See separate slides on unit testing.)

  5. Review: Kinds of testing • Unit testing • Integration testing • Does a collection of modules function correctly together as a group? • System testing • Regression testing

  6. Review: Kinds of testing • Unit testing • Integration testing • System testing • Does the entire system meet the end user’s requirements? • Regression testing

  7. Review: Kinds of testing • Unit testing • Shows that individual software components (units) live up to their contracts • Typically done by the same person who codes the given unit • Tests become part of regression test suite • Integration testing • System testing • Regression testing

  8. Testing Workflow: Flow

  9. Testing Workflow in UP Phases

  10. Testing Workflow in Phases Generally, when there is an implementation result, there is a test --- so Test at each build Inception phase: initial test planning, prototype testing Elaboration phase: test architectural baseline Construction phase: significant testing at each build Transition phase: re-test fixes and regression tests

  11. Testing Workflow: Other Tasks • Regression tests: • in a new build, re-apply tests from old builds • to make sure nothing “broke” in new build • Evolve test model • create new test cases for next build • refine prior test cases into regression tests • remove obsolete tests and corresponding • test procedures and components

  12. Testing Workflow: Models Test model artifacts Test cases Test procedures Test components Test subsystem packages for complex test Other artifacts Test Plan List of product Defects Test Evaluation

  13. Testing Workflow: Model Describes how executable components (e.g., builds) in the implementation model are tested by integration tests and system tests Describes how specific system aspects are to be tested e.g., usability testing of user interface or user manual

  14. Testing Workflow: Test Case Specifies one way to test the system What to test (requirements) With which inputs Expected results Test conditions and constraints

  15. Test Case Examples Test a use case or a scenario in a use case (black box) actor interaction, pre-conditions, action sequences, post-conditions Test a use-case realization (white box) verify interaction between components installation tests, configuration tests, negative (abuse) tests, stress tests Organize similar tests in input/result matrix Family of tests for different paths through a use case

  16. Special Tests Verify the system can be installed on the customer platform and that it executes correctly when installed Configuration tests verify the system works correctly in different configurations Negative tests (abuse tests) discover what causes failure Test outside the intended design, e.g., wrong configuration, wrong input, etc. Reveals weaknesses in design Stress tests show problems with insufficient or shared resources

  17. Testing Workflow: Test Procedures • Specify how to perform one or several test cases or • parts of them • Instructions for tester in manual test case • Instructions for interacting with test automation tool to • create executable test components, then instructions to integrate and execute those test components • One test procedure may cover several test cases • One test case may require several test procedures • Test procedure instructions often reflect flow-of events descriptions, including input values, how to enter input values, and how to verify results

  18. Testing Workflow: Test Components Automate one or several test procedures or parts Provide test inputs, control and monitor test execution, and possibly analyze and report results Developed with scripting languages, programming languages, or test automation tools test drivers, test harnesses, and test scripts Complex test cases and procedures may require formal analysis, design, and implementation modeling of the test components

  19. Testing Workflow: Test Plan Describes testing strategies, resources and schedule Strategy What kind of tests to perform for each iteration Test objectives Required level of test and code coverage Test acceptance criteria Trade-off cost vs. improved quality and reduced risk Test Plan may be defined in conjunction with the model Artifacts from other workflows analysis (black box) design and implementation (white box) “Tests” may include artifact reviews

  20. Testing Workflow: Test Results • Results are the output data of the test and/or the behavior of the system during test • Analysis of results: pass or defect • Defect: a system anomaly (software fault or a • defect found in artifact review) • Capture symptom in sufficient detail to allow problem resolution • Record defect in problem and resolution tracking system • Test evaluation: summary of overall test results • test-case coverage, code coverage • defects and defect status

  21. Testing Workflow: Test Results

  22. Testing Workflow: Workers Test designer: responsible for integrity of the test model plan tests: test goals and schedule select and define test cases, test procedures evaluate test results Component engineer: responsible for development of test components that automate some of the test procedures consider independent test

  23. Testing Workflow: Workers Integration tester: perform integration tests and report defects may be same person as system integrator in implementation workflow System tester: perform system tests and report defects focus on externally observable behavior consider users, beta customers, etc. as testers

  24. Testing Workflow – THE PLAN Plan the Tests Strategy and Resources Define Integration Tests Define System Tests Design Regression Tests Define and Mutate Test Cases Implement Tests Evaluate Tests

  25. Testing Workflow – THE PLAN Plan the Tests Strategy and Resources Describe testing strategy Inputs: use case model, supplemental requirements, design model Develop test cases and procedures that have a minimum of overlap test the most important test cases test high risk requirement items Estimate resources Estimate human and system resources Schedule the testing effort

  26. Testing Workflow – THE PLAN Define Integration Tests Derive test cases from use-case realizations – design (component interaction) Develop tests with minimal overlap that test an interesting path or scenario through use case Primarily consider interaction diagrams Combinations of actor input, output and system start state that lead to interesting scenarios that employ the classes that participate in the diagrams

  27. Testing Workflow – THE PLAN Define System Tests Test system as a whole Test combinations of use cases instantiated under different conditions Hardware configurations System loads Number of actors Database size, etc. Prioritize combinations required or are likely to function in parallel likely to influence each other if in parallel involve multiple processes frequently use system resources or use them in complex and unpredictable ways

  28. Testing Workflow – THE PLAN Design Regression Tests Modify test cases from earlier builds to be used as regression tests in subsequent builds Make more flexible and resilient to changes to the system under test Specifically address previously found defects Automate where possible

  29. Testing Workflow – THE PLAN Define and Mutate Test Cases Work test case by test case and suggest testing procedures for each one Reuse existing test cases May need to modify them to be reusable Align test procedures to subsystems under test Minimize overlap Isolates change to test resulting from change to subsystem

  30. Testing Workflow – THE PLAN Implement Tests Create test components from test procedures With test automation tools Perform or specify actions Record actions as a test component When programming explicitly Design and implement using test procedures Provide input and output data generation, visualization, interpretation and management utilities Use spreadsheets and database applications

  31. Testing Workflow – THE PLAN Evaluate Tests Perform the integration tests relevant to a build Manually perform test procedures Compare test results with the expected test results Investigate test results that deviate from the expected Faulty system and/or faulty test? Report defects to the component engineer responsible for the components likely to contain the fault Report defects to the test designers for evaluation of overall test results

  32. Testing Workflow – THE PLAN Evaluate Tests (cont) Test designers prepare metrics of test acceptance with respect to system quality level Test completeness and coverage Percentage of test cases executed Percentage of code and paths tested Reliability Categorize defect severity Analyze trends of discovered defects Compare with prior experience to predict amount of testing still required to reach an acceptable defect level

  33. Testing Workflow – THE PLAN Evaluate Tests (cont) Suggest actions Suggest further tests Relax test criteria Deliver parts of the system that passed; rework and retest parts that failed Document test completeness, reliability and actions

  34. Testing Workflow – SUMMARY Test Model Test cases specify what to test Test procedures specify how to perform tests Test components automate test procedures Test Plan Test Evaluation Defects to be fed into other core work

More Related