Introduction

1. Prioritizing Test Cases for Regression Testing Based on Boundary Value CoverageTao Xie and David NotkinComputer Science & Engineering, University of Washington What test cases are most likely to expose faults ? Introduction Standard characterizations of test cases include: Problem: If resources are constrained, how can we still perform regression testing effectively? Structural coverage: exercising structural entities (e.g., statements, edges) that encode faults is essential to exposing those faults  try to choose test cases that achieve high coverage of those entities. Data Coverage: exercising entities with special values (e.g., boundary values) increases the ability to expose many faults  try to choose special values that achieve high coverage of those entities. Solutions: Test Case Selection selects a subset of test cases that exercise those changed entities Test Case Prioritization orders the test cases with the intent of executing the test cases most likely to expose faults earlier What special values are dangerous? Boundary values tend to be dangerous. Extensively discussed in practitioner and researcher's testing handbooks: “The Art of Software Testing” by G. J. Meyers, 1979 “Software Testing Techniques” by B. Beizer, 1990 …… and research literatures …. Related Work:Boundary Value Coverage Related Work: Test Case Prioritization Most existing applications of boundary value coverage are in test case generation. Most existing test case prioritization techniques are based on structural coverage. Supportive Evidence Challenging Problem is based on Usually the specification or an assertion-like annotation is required to infer boundary value. Few programs are equipped with them. Lack of tool support to collect boundary value coverage information without a priori specification. Previous empirical studies showed data coverage techniques, e.g. boundary value coverage, outperformed structural coverage ones in testing effectiveness Complementing structural coverage techniques with data coverage ones are suggested in practice. Our Solution: Test Case Prioritization Based on Boundary Value Coverage without a priori specification Boundary predicates (used to infer boundary values) are extracted from invariants dynamically detected by Daikon, an invariant detection tool. The boundary value coverage for a test case execution is collected based on the data trace of that execution and the boundary predicates. Then test case prioritization can incorporate this boundary value coverage information to improve performance. Our initial experiments show promise for boundary value coverage; in particular, boundary value coverage dominated structural coverage for these small programs. This work was supported in part by the National Science Foundation under grant ITR 0086003.