1 / 16

Towards Automatic Generation of Parameterized Test Cases from Abstractions

Towards Automatic Generation of Parameterized Test Cases from Abstractions. Jens R. Calamé Natalia Ioustinova Jaco van de Pol Centrum voor Wiskunde en Informatica, Amsterdam. Agenda. Testing Theory Data Abstraction Test Case Parameterization Some Lemmas Conclusion. Testing Theory.

sidney
Download Presentation

Towards Automatic Generation of Parameterized Test Cases from Abstractions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards Automatic Generation of Parameterized Test Casesfrom Abstractions Jens R. Calamé Natalia Ioustinova Jaco van de Pol Centrum voor Wiskunde en Informatica, Amsterdam

  2. Agenda • Testing Theory • Data Abstraction • Test Case Parameterization • Some Lemmas • Conclusion IPA Lentedagen 2006

  3. Testing Theory • Conformance Testing: • Implementation Imp conforms specification Spec, ifffor all traces t in Spec holds: all input should be accepted after t, but at most the specified output should be generated by Imp. • Based on Tretmans‘ ioco theory • Test Generation by the tool TGV(Test Generation with Verification Techniques) • Based on state enumeration • Limitation: Data often leads to state-space explosion • Solution: Data abstraction mitigates this limitation IPA Lentedagen 2006

  4. Our Test Generation Process System Specification (e.g. UML) System Specification (formal) Data Abstraction Abstract System Specification Test Case Generation TGV Abstract Test Case Test Purpose TTCN-3 Generation Constraint Generation Parameterizable Test Case (TTCN-3) Rule System for Data Selection IPA Lentedagen 2006

  5. Data Abstraction • Motivation: Input and output data from large (infinite) domains leads to state explosion Application of state-based test generation tools impossible • Solution (from model-checking): data abstraction • Introduction of a chaotic value ╥ for each datatype D • Lifting functions for original values from D • Replacement of input variables by ╥and propagation through the system • Introduction of may functions for guards over three-value-logic (semantics: may(╥) = true) • Consequence: Introduction of extra traces bynon-determinism IPA Lentedagen 2006

  6. ? init Pin ( p ) ? initBalance ( b ) 0 1 2 ? getPin ( x ) ¹ ( x p ) ! pinIncorrect > 3 == ( x p ) > ! pinCorrect 4 > ( y b ) > ? getBalance ! LowBalance ( b ) ? getAmount ( y ) 8 7 5 ? getAmount ( y ) £ ( y b ) ! Money ( y ) ; ! Balance ( b ) > 6 b := b - y Data Abstraction – Example IPA Lentedagen 2006

  7. Data Abstraction – Test Case by TGV IPA Lentedagen 2006

  8. Test Case Parameterization • To be solved after data abstraction: • Pruning of traces introduced by overapproximation • Finding possible value ranges for test data • Rule System (Prolog) • Represents SUT specification • Defines rules on data (addition, substraction; and, or etc.) • Defines rules for process behavior of Spec • Query (Prolog) • Represents the abstract test case(one rule per trace) • All transitions in trace taken as query body • Alternative: one query for whole CTG IPA Lentedagen 2006

  9. ? init Pin ( p ) ? initBalance ( b ) 0 1 2 ? getPin ( x ) ¹ ( x p ) ! pinIncorrect > 3 == ( x p ) > ! pinCorrect 4 > ( y b ) > ? getBalance ! LowBalance ( b ) ? getAmount ( y ) 8 7 5 ? getAmount ( y ) £ ( y b ) ! Money ( y ) ; ! Balance ( b ) > 6 b := b - y Testing with Abstraction – Rule System … getPin(state(2,P,B,X,Y), state(3,P,B,X1,Y), param(X1)). pinIncorrect(state(3,P,B,X,Y), state(8,P,B,X,Y),param(_)) :- X ≠ P. pinCorrect(state(3,P,B,X,Y), state(4,P,B,X,Y), param(_)) :- X = P. … IPA Lentedagen 2006

  10. Testing with Abstraction – Query oracle(P,B,X,Yin,Yout) :- initPin(state(0,0,0,0,0), G1, param(P)), initBalance(G1, G2, param(B)), getPin(G2, G3, param(X)), pinCorrect(G3, G4, _), getAmount(G4, G5, param(Yin)), money(G5, _, param(Yout)). P=P{-1.0inf..1.0inf} B=B{-1.0inf..1.0inf} X=P{-1.0inf..1.0inf} Yin=Yout{-1.0inf..1.0inf} Yout=Yout{-1.0inf..1.0inf} Yout{-1.0inf..1.0inf}-B{-1.0inf..1.0inf}=<0 IPA Lentedagen 2006

  11. Test Execution • Pre-solve one trace to pass statically(skip internal steps) • Execute this trace until the SUT leaves it • Try to find a trace to pass solve and execute it • If no trace to pass:Try to find a trace to inconc  solve it • If no trace to inconc:set test verdict fail !initPin(P) !initBalance(B) 0 1 2 !getPin(X) ?pinInCorrect INCONC 3 ?pinCorrect ?lowBalance 4 !getAmount(Yin) !getBalance PASS 7 5 ?Money(Yout) ?Balance(x50) !getAmount(x60) ?eatCard 6 IPA Lentedagen 2006

  12. Some Lemmas • The original system Spec is simulated by Spec╥ in all details (and some more). • The control flow of the synchronous productSpec x TP is simulated by Spec╥ x TP. • The set of accepting/refusing traces of Spec x TP is a subset of those of Spec╥ x TP. • If the test oracle holds for a trace in the CTG under a certain valuation, this trace is valid in the original system under the given valuation. • The test algorithm terminates with a sound verdict. For further details and proofs see the technical report. IPA Lentedagen 2006

  13. Case Study CEPS • Common Electronic Purse Specifications: protocol for electronic payment using a multi-currency smart-card • Input and output parameters of card actions mainly natural numbers  (in)finite domain • Variables partially arrays(up to 16 elements in simplified µCRL realization) • As µCRL specification: • 54 summands • 44 process variables (netto) • 207 process variables (brutto), due to arrays of structures IPA Lentedagen 2006

  14. Case Study CEPS (cont'd) • Instantiation and reduction (abstracted specification): ca. 16 min. on five 2.2GHz Athlon 64bit single CPU computers (1 GB RAM each) • Generation of two test cases: 594 and 109 states, resp. in less than one second (one 2.2GHz AMD Athlon XP 32 bit CPU and 1 GB RAM) • Constraint solving produces results in negligable time IPA Lentedagen 2006

  15. Conclusion • Data abstraction makes state-based test generation applicable to systems with large data domains • Approach successfully evaluated on CEPS case study • Ongoing and future work: • Redesign of tools (add "real" constraint-solving)  • On-the-fly constraint-solving • Treatment of -steps • (Generation and) execution of TTCN-3 test cases • Integration of UML as specification language IPA Lentedagen 2006

  16. TT-Medal Project: www.tt-medal.org Test generator TGV: www-verimag.imag.fr/~async/TGV www.inrialpes.fr/vasy/cadp Data abstraction tools: www.cwi.nl/~calame/dataabstr.html Technical report and papers:www.cwi.nl/~calame/works.html Jens R. Calamé jens.calame@cwi.nl Natalia Ioustinova ustin@cwi.nl Jaco van de Pol vdpol@cwi.nl Related Links IPA Lentedagen 2006

More Related