1 / 16

Validating and Improving Test-Case Effectiveness Yuri Chernak

Validating and Improving Test-Case Effectiveness Yuri Chernak. Presented by Michelle Straughan. Overview. Introduction Test-Case Effectiveness Metric (TCE) Improving Test-Case Effectiveness 5 steps to improving TCE Summary. Introduction.

amir-hardin
Download Presentation

Validating and Improving Test-Case Effectiveness Yuri Chernak

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validating and Improving Test-Case EffectivenessYuri Chernak Presented by Michelle Straughan

  2. Overview • Introduction • Test-Case Effectiveness Metric (TCE) • Improving Test-Case Effectiveness • 5 steps to improving TCE • Summary

  3. Introduction • Software testing is crucial to the success of product • Evaluation certifies product usage • Verification of test specifications • Validation of test cases • New metric for evaluating test case • Framework for test-case effectiveness

  4. Improving Test-Case Effectiveness – Test-Case Effectiveness Metric • Formulation of metric to measure the effectiveness of testing • Metric Definition: TCE = Ntc /Ntot * 100% • TCE value compared to a baseline to determine effectiveness (eg 75% baseline)

  5. Improving Test-Case Effectiveness • Analyze causes and areas improvements are needed • Proposed improvement framework based on test-escapes (defects test case suite did not find but were side effects found in the same test cycle) • Approach relies on analysis of missed defects by test cases

  6. 5 steps to Improvement • Thoroughly understanding and properly documenting the test process • Make Assumptions • Gathering defect data and performing causal analysis • Identifying main factors • Implementing corrective actions

  7. Step 1: Thoroughly understanding and properly documenting the test process

  8. Step 2: Make Assumptions • After completing step 1, project team makes assumptions about factors affecting TCE • Test Planning • Test objectives defined in test plan should be traceable to functional specifications • Incomplete specifications lead to an incomplete test plan and leads to reduced TCE

  9. Step 2: Make Assumptions • Test Design • Necessary test cases to test logic of functional features need to be defined, if not adequate coverage will be lacking • Test case specifications should correspond to business rules • Deficiencies could be caused by: • Incomplete test suite (all possible test cases are not defined) • Incomplete test cases – proper logic is not tested – negative test cases • Incorrect test-specifications • Test Preparation and Execution • Test cases not executed • Test cases improperly executed

  10. Factors affecting test-case effectiveness Figure 1: Factors affecting test-case effectiveness

  11. Figure 2: Factors based on causal analysis logic Step 3: Gathering defect data and performing causal analysis

  12. Step 3: Gathering defect data and performing causal analysis • Testers need to track defects and categorize defects. Test-escapes are further categorized according to (fig. 2) • Understand reasons for missed defects • Verify completeness of test specifications • Verify coverage of test-cases • Check for test deficiencies • Verify the correctness of test cases

  13. Step 4: Identifying main factors • Identifying factors that are causing major defects • Analyze these defects and focus on these as most needing corrective measures

  14. Step 4: Identifying main factors Figure 3: Pareto chart

  15. Step 5: Implementing corrective actions • Depending on project at hand, corrective measures could vary. These may include: • Revisiting and revising functional specifications, which results in revising test case specifications • Use of traceability matrix to meet coverage needs of the project – ensures that test cases meet business rule • Training for testers on proper design techniques, using checklists to help better design test case specifications (involves revisiting and revising test case specifications) • Training for testers on proper way to execute tests and verifying test results

  16. Summary • New framework is to aid in better effectiveness of test-cases, however its not applicable to some projects as it requires large number of defects • Framework would theoretically help to reduce defects present in a product when released to production, since it first checks the test process and if its not effective then provides steps for achieving effectiveness • Formalized way of achieving effectiveness, other ways for improving effectiveness could include the use of other testing techniques

More Related