1 / 20

Department of CS and Mathematics, University of Pitesti

TAIC PART 2007. State-based Testing is Functional Testing !. Florentin Ipate, Raluca Lefticaru University of Pitesti, Romania. Department of CS and Mathematics, University of Pitesti. TAIC PART 2007. Background

elewa
Download Presentation

Department of CS and Mathematics, University of Pitesti

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TAIC PART 2007 State-based Testing is Functional Testing ! Florentin Ipate, Raluca Lefticaru University of Pitesti, Romania Department of CS and Mathematics, University of Pitesti

  2. TAIC PART 2007 • Background • Over the last 10-15 years we have developed specification based test generation techniques proven to determine system correctness under certain “design for test” conditions. • State-based specifications – stream X-machine (SXM). • Industrial evidence of the power and practicality of these techniques. • Joint research with the Verification & Testing Research Group of the University of Sheffield, UK (Mike Holcombe). Department of CS and Mathematics, University of Pitesti

  3. TAIC PART 2007 • Class testing • State based languages (e.g. UML statecharts) are widely used to model class behaviour. • New challenge: extend our SXM based techniques to class testing. • Gather empirical evidence about their effectiveness. Department of CS and Mathematics, University of Pitesti

  4. TAIC PART 2007 • Empirical evidence • Empirical studies show that state-based testing is more effective than random testingbut it is not likely to be sufficient unless the state diagram is straightforward. • They advocate the use of functional methods to complement state-based testing. • Our experiments (in which we evaluated our own SXM based techniques) also back these findings. Department of CS and Mathematics, University of Pitesti

  5. TAIC PART 2007 • Questions • Why ? • Can functional testing and state-based testing be separated ? Department of CS and Mathematics, University of Pitesti

  6. TAIC PART 2007 • Our answers • The modest fault detection of state-based class testing is actually due to the inappropriate state diagram used. • Functional testing of a class can be reduced to state-based testing, provided that the right state model is produced. Department of CS and Mathematics, University of Pitesti

  7. push[0 < L < k-1] pop[1 < L < k] popErr[L = 0] pushErr[L = k] push[L = 0] push[L = k-1] 0 1 2 pop[L = 1] pop[L = k] TAIC PART 2007 • How is a state diagram constructed ? • Usually: ad-hoc process, using conventional wisdom. • Example: a stack is associated with a 3 state diagram. Department of CS and Mathematics, University of Pitesti

  8. push[0 < L < k-1] pop[1 < L < k] count popErr[L = 0] count pushErr[L = k] count push[L = 0] push[L = k-1] 0 1 2 pop[L = 1] pop[L = k] TAIC PART 2007 • How is a state diagram constructed ? • Suppose we add a new method, count(l) that finds the • number of distinct elements among the last l. • A 3-state diagram is no longer appropriate. Department of CS and Mathematics, University of Pitesti

  9. TAIC PART 2007 • Our approach • Construct a diagram for each class method to be tested. • The construction of the diagram – based on a partitioning of the domain of the class method. • Thus, state-based testing is functional testing ! Department of CS and Mathematics, University of Pitesti

  10. TAIC PART 2007 • State diagram construction • Partition the domain according to some functional criterion. • Derive (not necessarily disjoint) data sub-domains by “hiding” the inputs. • Derive the states of the diagram by removing the overlapping of data sub-domains. • Construct the transitions by considering each pair of states and checking whether the pair is related to a method. • Split the transitions according to the original partitioning criterion. Department of CS and Mathematics, University of Pitesti

  11. TAIC PART 2007 • State diagram construction (example) • push :3 state diagram • pop : 3 state diagram • count • domain partition • count(l) = 0 • count(l) = 1 • 1 <count(l) < L • count(l) = L Department of CS and Mathematics, University of Pitesti

  12. count[p2] count[p1] push[0→1] push[1→1] pop[1→1] 2 1 pop[1→0] push[1→2] pop[2→1] push[i→i,2≤i≤k-1] push[i→i+1,2≤i≤k-2] pop[i→i,2≤i≤k-1] pop[i→i-1,3≤i≤k-1] pop[k→k-1] 4 3 push[k-1→k] count[p2] count[p3] count[p4] count[p2] count[p3] TAIC PART 2007 • State diagram construction (example) Department of CS and Mathematics, University of Pitesti

  13. TAIC PART 2007 • Previous approaches • Construct a composite (all-in-one) diagram rather than a separate diagram for each method. • A method may induce at most one transition between a given pair of states  Original partitioning criterion is not fully used. • Stack example: • 9-state composite diagram but • One loop-back count transition for each state, so important test cases can be missed out. Department of CS and Mathematics, University of Pitesti

  14. TAIC PART 2007 • Test generation • Adaptation of the W-method. • Transition cover: only the transitions labeled by the method in question are considered. • Characterization set: identifies each state of the diagram. Alternative : an abstract oracle associates with each state a predicate rather than a sequence of transitions. Department of CS and Mathematics, University of Pitesti

  15. TAIC PART 2007 • Experimental results • A series of experiments to evaluate the cost effectiveness of state-based testing. • Java implementation of several classes, e.g. bounded stack, book from a library, vending machine. • Test suites were executed against mutant programs automatically generated by MuJava. • Test cost measure = number of mutants killed / total length Department of CS and Mathematics, University of Pitesti

  16. TAIC PART 2007 • Experimental results • Bounded stack: approximately 90 LOC, 219 mutants. Our approach Composite (all-in-one) diagram Department of CS and Mathematics, University of Pitesti

  17. TAIC PART 2007 • Experimental results • Book class (library): approximately 110 LOC, 136 mutants. Our approach Full predicate Department of CS and Mathematics, University of Pitesti

  18. TAIC PART 2007 • Mutants left undetected • Equivalent mutants. • For which stronger criteria are needed. • Caused by attributes (dueDate) for which no obvious partitioning criterion exists: precise oracle needed. • Class mutants (JSI): simple tests in which more than one instance is used. Department of CS and Mathematics, University of Pitesti

  19. TAIC PART 2007 • Further work • Evolutionary methods for automatically converting sequences of methods into actual test data. • Deriving the state model using evolutionary methods ? Department of CS and Mathematics, University of Pitesti

  20. TAIC PART 2007 Thank you for your attention ! Department of CS and Mathematics, University of Pitesti

More Related