1 / 1

Specification-Based Unit Test Data Selection: Integrating Daikon and Jtest Tao Xie and David Notkin Computer Science &a

Manual basic test #. Iteration. Type. Auto generated test # (length 1). Manual JAX test #. Auto generated test # (length 2). Test call length 1. Auto generated test # (length 3). Test call length 2. Test call length 3.

tieve
Download Presentation

Specification-Based Unit Test Data Selection: Integrating Daikon and Jtest Tao Xie and David Notkin Computer Science &a

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Manual basic test # Iteration. Type Auto generated test # (length 1) Manual JAX test # Auto generated test # (length 2) Test call length 1 Auto generated test # (length 3) Test call length 2 Test call length 3 Specification-Based Unit Test Data Selection: Integrating Daikon and JtestTao Xie and David NotkinComputer Science & Engineering, University of Washington 8 (67%) 1. Basic Full 14 (63%) 96 (86%) 16 (97%) 0/1: 2 1745 (86%) 0/3: 3 0/0: 0 1. Basic No Pre 5/7: 15 8/13: 15 1/2: 2 1. JAX Full 1/3: 3 0/0: 0 1/1: 1 1. JAX No Pre 1/3: 6 10/24: 41 0/0: 0 2. Basic Full 0/0: 0 0/0: 0 0/0: 0 2. Basic No Pre 0/0: 0 0/1: 1 1/1: 1 2. JAX Full 0/0: 0 0/0: 0 1/2: 2 2. JAX No Pre 0/0: 0 0/1: 1 1/1: 1 Problems Which test inputs are more “valuable” ? • Developers manually write unit tests, which are usually insufficient to achieve satisfactory quality assurance of the unit. (Test insufficiency problem) • Unit test generation tools automatically produce a large number of test inputs to exercise the unit. But how to know these automatically generated tests produce correct results? (Test oracle problem) • How do we select “valuable” test inputs from automatically generated ones, equip them with oracles, and augment manually created unit tests? (Test selection problem) Those new test inputs that exercise at least one new feature not being exercised by any test case in the existing test suite. Black-box features: functionalities, predicates/states in specifications, etc. White-box features: methods, statements, branches, def-use pairs, etc. In traditional black box testing, , either formal or informal specifications are usually needed to help identify features, but in practice programs lack of specifications most of the time. A specification-based approach without a priori specification • Infer likely specifications from existing test executions • They reflect both the program behavior and the test history • Any new tests that violate the inferred specifications are used as the candidates for test data selections – These tests possibly exercise new program behavior Jtest(http://www.parasoft.com/) Daikon(http://pag.lcs.mit.edu/daikon/) A popular commercial unit test tool that can produce a large number of test inputs for a Java class An invariant detection tool that can infer likely specifications from program executions Bounded Stack Example: The number of selected tests, violating tests, and violated specifications Precondition Guard Removal Iterations reaching fixed point This work was supported in part by the National Science Foundation under grant ITR 0086003.

More Related