330 likes | 461 Views
An Approach to Testing Distributed Systems. Ina Schieferdecker, Mang Li, Stephan Pietsch, Theofanis Vassiliou-Gioles, Andreas Hoffmann. GMD FOKUS, Berlin. Contents. Introduction Testing of Distributed Systems Test Case Development Test Validation: TSsim Test Implementation: TCgate
E N D
An Approach to Testing Distributed Systems Ina Schieferdecker, Mang Li, Stephan Pietsch, Theofanis Vassiliou-Gioles, Andreas Hoffmann GMD FOKUS, Berlin
Contents • Introduction • Testing of Distributed Systems • Test Case Development • Test Validation: TSsim • Test Implementation: TCgate • Test Execution: TTman • An Example: TINA Retailer Reference Point • Conclusions
Introduction • Distributed systems are complex, heteregeneous, and concurrent systems • Validated properties of the system specification might not be fulfilled by a system implementation • CTMF is a well-established and widely used test method for conformance and interoperability testing of OSI systems (including TTCN as a test notation) • Goal: • Application of TTCN to distributed systems • Automated CORBA based distributed test execution
Local Testing Distributed Testing Test Manager Test Manager TC TC TC TC TC TC IUT IUT • One Testing Device • Common Time • Coordination Message Exchange is Local • Many Testing Devices • Distributed Time • Coordination Message Exchange crosses Boundaries Local and Distributed Testing
TS OperationsTest StepsTimers TTCN Test Cases ODL / IDL SDL Adaptor CoDec TTCN ETS TCgate SUT ORB ORB ORB The Overall Approach
Test Case Development • Declarations part: • TTCN types: • Translation from ODL types is supported by mapping rules. • Mapping rules can be used for manual or semi-automatic test development. • Mapping rules support generic encoding/decoding. • Others: • Operations, parameters, constants, variables, timers.
Test Case Development (cont‘d) • Dynamic part, constraints part : • Currently manually developed, based on interpretation of textual description. • Alternative: semi-automatic, based on SDL specificationI.e. with TTCNLink, AutoLink or TGV.
ODL-to-TTCN Mapping • Mapping for an ODL interface declaration: • Interface inheritance is flattened. • An interface is comparable to a SAP. • Requested and provided interfaces are distinct two PCO types: • PCOtoSUTServer • PCOtoSUTClient • Multiple PCOs derived from the same interface declaration can be used in a test (multi-server or multi-client).
ODL-to-TTCN Mapping (cont‘d) • Mapping for a typical ODL operation declaration: • Operations are mapped to ASPs. • Three ASP types are derived from an operation declaration: • A request-related ASP type. • A reply-related ASP type. • A raise-related (exception-related) ASP type.
PA Consumer Domain Retailer Domain IA requestNamedAccess() UAP i_RetailerInitial UAP: User ApplicationPA: Provider AgentIA: Initial Agent Example:Test for Service Access Session Retailer Reference Point
Example: IDL Specification • module TINARetRetailerInitial { • interface i_RetailerInitial { • voidrequestNamedAccess ( • in TINACommonTypes::t_UserId userId, • in TINACommonTypes::t_UserProperties userProperties, • out Object namedAccessIR, • out TINAAccessCommonTypes::t_AccessSessionSecretId asSecretId, • out TINAAccessCommonTypes::t_AccessSessionId asId • ); • }; • };
Distributed TestExecution Distributed System TestValidation Test Specification TTman SUT ITEX TSsim TCgate The Test Tool Chain
Motivation for a TTCN Simulator • Testing the test problem • Validation of test suites is essential • Complexity of Concurrent TTCN test cases • Analyser tools provided by TTCN editors provide only syntax and static semantic checks • Manual dynamic semantic check by experts inefficient and error-prone TSsim
Test Suite Simulator TSsim • Basis for test suite validation • Generic test suite simulator • Supports Concurrent TTCN • Interactive step by step execution of test cases (PDUs from the IUTs, Timeouts) • Verification of dynamic aspects (e.g. test coordination procedures, timer settings) • Validation of logical correctness, deadlocks TSsim
Test Suite Simulator GUI TSsim
Simulation Validated TTCNATS Simulation Results Correction Simulation Report Validation Process with TSsim TSsim
Error Detection with TSsim • Timer Errors deadlock, if no receive event is evaluated to true • Test Component Coordination Errors timeout, if a test component waits for a CM • Indentation Errors unexpected dynamic behavior • Constraint Errors non-matching of correct PDUs or CMs TSsim
Distributed TestExecution Distributed System TestValidation Test Specification TTman SUT ITEX TSsim TCgate The Test Tool Chain TCgate
Motivation for a TTCN/CORBA GW • Acceptance of TTCN • Support of parallel test components via Concurrent TTCN • Test access to CORBA-based systems. • Uniform interface to TTCN-based test systems • Separated from TTCN-based test systems TTCN compiler-independent. • Problem: TTCN OSI paradigm vs. CORBA OO paradigm TCgate
TTCN/CORBA GW: TCgate • Architecture of the gateway: • Supports testing of interfaces provided by the SUT. • Supports emulation of interfaces required by the SUT. • Is a generic request-level bridge. • Uses dynamic interfaces of ORB type-independent. • Is a CORBA-compliant application. • Uses public interfaces of ORB easily portable. TCgate
ETS GatewayServer Supervisory functions TTCN run-time behaviour Help functions gwAdaptor gwCoDec GatewayMain TTCN / CORBA Gateway Interface Repository IDLSkeleton SUT IDLStubs GatewayClient DII DSI DSI DII BOA BOA ORB ORB Test Interface for CORBA Based Systems TCgate
0100011000101101011111010101000100011101 ITEX-GCI Generic CoDec TTCN TTCN/CORBA-Gateway Adaptor TTCN ATS C++ ETS ETS Generator Test Suite Adaption to TCgate • Is based on GCI interface. • Has generic encoding/decoding for all IDL types. • Is reusable for other TINA and CORBA applications. TCgate
PA ETS Retailer Reference Point Consumer Domain Retailer Domain SUT requestNamedAccess() TINA Server TINA Client UAP i_RetailerInitial UAP: User ApplicationETS: Executable Test SuiteSUT: System Under Test Example:Test for Service Access Session
ETS TTCN / CORBA Gateway IUT / SUT GatewayMain GatewayServer GatewayClient IDLSkeleton IDLStubs DII DSI ORB ORB Example: Test Execution 1. Test Request from ETS to IUT 2. IUT Parameter Request to ETS 3. ETS Parameter Reply to IUT 4. IUT Test Reply to ETS 5. Test Result: Verdict Assignment => PASS / FAIL TCgate
Distributed TestExecution Distributed System TestValidation Test Specification SUT TTman ITEX TSsim TCgate The Test Tool Chain TTman
Test Sychronizations Protocol 1 • Defines Procedures for • Test Setup, • Setting Up Configuration • Distributing Parameters, etc. • Test Execution • Start, Stop and Cancel • Routing of Coordination Messages • Test Reporting TTman
CMI TSP1 FrontEnd1 TC TC TC TC System Supervisor FrontEnd2 IUT CMI - Campaign Managment Interfacce TSP1 - Test Synchronization Protocol 1 TC - Test Component TSP1 Architecture TTman
The Test Manager TTman • Portable Library for Implementing • The System Supervisor • The Front End • State Driven TCL/TK GUI at the CMI TTman
Test Execution • Generation of MSC logs: • Uses a library supplied by Telelogic. • Display selected test events in an MSC editor. • Benefits: visualization, problem location.
Conclusions • Testing of CORBA based systems • ETS Generator for transformation of abstract test suites into executable ones, validation with TSsim • TCgate and TTman for automated distributed test execution • Testing within the development methodology for distributed systems • Future work • Enhanced coordination features for parallel test components • Investigation of real-time aspects of the TTCN/CORBA Gateway • TTCN extensions to better address the TINA object model