TTCN

1. TTCN • TTCN: Testing and Test Control Notation • Original meaning: Tree and Tabular Combined Notation • Original purpose: Specification of abstract test suites for conformance testing of International Telecommunications Union protocols. • Now promoted as a general purpose test language for distributed / communicating systems. • Goal: Provide a complete, internationally standardized, specification for test case behaviour and execution that is repeatable in any test lab. • Predictable execution requires very precise semantic definitions.

2. History (1) • Initial discussions on a standardized test language as part of a conformance test framework for communications protocols started in 1984. • In 1994, the first version of the conformance test framework standard, ISO-9646, was published. • TTCN-1 eventually was part 3 of this standard, published in 1995. • Language assumed that the entire protocol testing methodology was being used. • In 1997, TTCN-2 was issued. • Most used in Europe; only limited use in North America.

3. TTCN-2 • “Tree and Tabular Combined Notation” • Tables describe data formats, signal specifications, etc. • Trees used to describe test case behaviour • Execution starts at the root of the tree • Tree branches represent alternative execution paths. • Implication: no loops • Alternatives explicitly specified for valid and invalid behaviour.

4. Implementation of TTCN Tests • TTCN describes abstract tests that are completely platform independent • The TTCN standard also provides operational semantics, which describe the exact execution behaviour of a test case • Implementations are often in high-level language (C++, Java…) • TTCN primitives are mapped to actual test environment. • Implementations must obey the operational semantics

5. TTCN-2 Tables • Example: an “abstract service primitive type” definition:

6. TTCN-2 Tables • This table describes the data values to be used in one instance of a message of the type of the previous table, sent during a test case.

7. TTCN-2: Trees expected forward progress in time alternatives

8. TTCN-2: Test Suite Overview

9. TTCN-2: Test Suite Declarations

10. TTCN-2 Signal Declarations message data format

11. TTCN-2: Table declarations constraints expected message data

12. TTCN-2: Test case

13. Implementing TTCN-2 tests user interface compiled TTCN test case timer management executor decoder encoder form extract manually- written code operating system service PCO queues send/receive events test equipment- specific to system under test

14. Implementing TTCN-2 tests user interface compiled TTCN test case timer management executor decoder encoder ! A ? B form extract 011001…. 110010…. PCO queues send/receive events to system under test

15. History (2) • Discussions after TTCN-2 centred on making the language more flexible and “user-friendly”. • Standardization taken over by ETSI, the European Telecommunications Standards Institute. • First version of TTCN-3 issued in 2001 • Radical change from the previous version

16. Credits • Much of the following has been adapted from: • “TTCN-3: An Introductory Overview” • Anthony Wiles, ETSI • Conference presentation at TestCom • “About TTCN-3” • Tutorial available at www.etsi.org

17. What is TTCN-3? • Testing and Test Control Notation Version 3 • Internationally standardized testing language • Product of the European Telecommunications Standards Institute (ETSI) • A programming language that has been used for more than 15 years in standardization as well as industry • Specifically designed for black box testing and certification • Constantly developed and maintained at ETSI by a team of leading testing experts from industry, institutes, and academia • A testing technology that applies to a variety of application domains and types of testing • Proven to work in very large and complex industrial tests, e.g., 3G network elements

18. TTCN-3 Standards • Standards from ETSI [some cross-listed as ITU standards] • ES 201 873-1 (Z.140): TTCN-3 Core Language • ES 201 873-2 (Z.141): TTCN-3 Tabular Presentation Format (TFT) • ES 201 873-3 (Z.142): TTCN-3 Graphical Presentation Format (GFT) • ES 201 873-4 (Z.143): TTCN-3 Operational Semantics • ES 201 873-5: TTCN-3 Runtime Interface (TRI) • ES 201 873-6: TTCN-3 Control Interfaces (TCI) • ES 201 873-7: Using ASN.1 with TTCN-3 • ES 201 873-8: The IDL to TTCN-3 Mapping • Proposed: Mappings to XML, C++ • Currently: Edition 3 of TTCN-3 (2007/8)

19. What makes TTCN-3 different … • From conventional programming or scripting languages? • Rich type system including native list types and support for subtyping • Embodies powerful build-in matching mechanism • Snapshot semantics, i.e., well defined handling of port and timeout queues during their access • Concept of verdicts and a verdict resolution mechanism • Support for specification of concurrent test behaviour • Support for timers • Allows test configuration at run-time • Tests focus only on implementation to be tested

20. What makes TTCN-3 different … • From a test tool or vendor proprietary testing language? • Not tied to a particular application or its interface(s) • Not tied to any specific test execution environment, compiler or operation system • TTCN-3 as such is not executable and requires a compiler/interpreter, adapter as well as codec implementations

21. (s + s) ( ) TTCN-3 test systems in a nutshell • TTCN-3 specifies a test but a test system is needed for test execution • TRI and TCI standards define test system architecture • TTCN- 3 tools are required to support internal interfaces • Allows reuse of test platforms with different tools but also for different SUTs • A test system requires • A TTCN-3 tool = TTCN-3 compiler and execution environment • A test platform for a specific device under test • Note: Tools come with default Test Control & Logging Test System Executor TTCN-3 Test System Test Control Logging Codecs TCI [compiled] TTCN-3 Executable TRI SUT Adapter Platform Adapter System Under Test (SUT) TCI = TTCN-3 Control InterfaceTRI = TTCN-3 Runtime Interface

22. An example adaptation: A IPv6 test system Test System Executor TTCN-3 Test System ToolLogging ParameterFile IPv6 Codecs TCI [compiled] IPv6 ATS TTCN-3 Executable TRI Ethernet Adapter Real-time Adapter Open Source FreeBSD Router

23. Minimal Test Configuration • All test behavior is executed on one (main) test component TTCN-3 Test System TCI Test Control TTCN-3 Executable Logging MTC Communication with IUT Codecs TRI SUT Adapter Platform Adapter Communication with SUT System Under Test (SUT) Communication with IUT Implementation Under Test

24. Example Concurrent Test Configuration • A test involves execution of many parallel test components • Dynamic instantiation of components and communication links TTCN-3 Test System TCI Test Control TTCN-3 Executable MTC Logging InternalCommunication PTC PTC Codecs TRI SUT Adapter Platform Adapter System Under Test (SUT) Implementation Under Test

25. Basic Concepts

26. Main Capabilities of TTCN-3 • Dynamic concurrent testing configurations • Various communication mechanisms (synch and asynch) • Data and signature templates with powerful matching mechanisms • Specification of encoding information • Display and user-defined attributes • Test suite parameterization • Test case controland selection mechanisms • Assignment and handling of test verdicts • Harmonized with ASN.1 and IDL (XML and C++ coming) • Different presentation formats • Well-defined syntax, static semantics and operational semantics

27. Text format Tabular Format Graphical Format PresentationFormat3 PresentationFormatn The Core Languageand Other Presentation Formats • Core format is text based (most popular) • TTCN-3 can be edited or viewed in other formats • Tabular format (for TTCN-2 people) • Graphical format (good for visual overview) • Other standardized formats in the future? • Proprietary formats possible TTCN-3 Core Language

28. Example Core (Text) Format testcase TC_resolveEtsiWww() runson DnsClient { timer t_ack; serverPort.send(m_dnsQuestion("www.etsi.org")); t_ack.start(1.0); alt { [] serverPort.receive(mw_dnsAnswer("172.26.1.17")) {setverdict (pass); } [] serverPort.receive { // any other message setverdict(fail); } [] t_ack.timeout { setverdict(inconc); } } t_ack.stop; }

29. Example Graphical Format testcase TC_resolveEtsiWww() runson DnsClient mtc serverPort DnsClient DnsPort timer t_ack m_dnsQuestion("www.etsi.org") t_ack alt mw_dnsAnswer("172.26.1.17") pass ? fail t_ack inconc t_ack

30. Example Tabular Format

31. ASN.1 Types & Values IDL Types & Values XML Types & Values Other types & Valuesn Use of TTCN-3 With Other Languages • TTCN can be integrated with types systems of other languages • Fully harmonized with ASN.1 • Harmonized with other languages • IDL, XML, C/C++ TTCN-3 Core Language

32. Test System Architecture Test Data Data Types Test Behaviour Major Elements of TTCN-3 • Built-in and user-defined generic data types (e.g., to define messages, service primitives, information elements, PDUs) • Actual test data transmitted/received during testing • Definition of the components and communication ports that are used to build various testing configurations • Specification of the dynamic test system behaviour TTCN-3 Core

33. Structure and Contents of a TTCN-3 Test Suite

34. Module Definitions Module Control Attributes TTCN-3 Modules • Modules are the building blocks of all TTCN-3 specifications • A test suite is a module • A module has a definitions part and a control part • Modules can be parameterized • Modules can import definitions from other modules • Modules can have attributes Module (…)

35. Example of Module Structure module MyModule (integer MyTSPar1) { : // Definitions part control { : // Control part } } with display″ETSI format″;

36. RP Signatures Data Types Data Templates Constants Communication Ports Test Components Functions Named Alternatives Test Cases Module Definitions • Definitions are global to the entire module • Data Type definitions are based on the TTCN-3 predefined types • Templates and Signatures define the test data • Ports and Components are used in Test Configurations • Functions, Named Alts and Test Cases define behaviour Signature Templates

37. Importing Definitions // Importing single definitions import template MyTemplate from MyModule; // Importing definitions of the same kind import all template from MyModule; // Importing all definitions import all from MyModule; // inhibiting recursive import import template MyTemplate from MyModule nonrecursive; /* To avoid name clashes the imported definition is prefixed with the identifier of the source module */

38. External Definitions // External constants may be defined external const integer MyExternalConst; // External functions may be defined external function MyFunction4() return integer;

39. Groups of Definitions // A nested group of function definitions group MyTestStepLibrary { group MyGroup1 { function MyTestStep11 { … } : function MyTestStep1n { … } } group MyGroup2 { function MyTestStep21 { … } : function MyTestStep2n { … } } }

40. TTCN-3 Data Types

41. TTCN-3 Base Types

42. Example of Integer Type /* integer sub-type RANGE definition */ type integer MyIntegerRange (1 .. 100) type integer MyIntegerRange (0 .. infinity) /* integer sub-type LIST definition */ type integer MyIntegerList (1 , 2, 3, 4) /* integer sub-type as a mixed LIST and RANGE */ type integer MyIntegerListAndRange (1 ..5, 7, 9)

43. Example of a Structured Type // record type definition type record MyMessageType { integer field1 optional, boolean field2 } // record value notation var MyMessageType MyMessage { field1 1, field2 true } // Individual field assignment MyMessage.field2 := false

44. Arrays /* Arrays are not considered to be types in TTCN-3. Instead, they are specified at the point of variable declaration */ // integer array of 5 elements varinteger MyArray[5]; // indexing starts at zero MyArray[0]:= 1; // An initialiser may be used to set array values MyArray := {1,2,3,4,5};

45. Defining Test Data

46. Uses of Data in TTCN-3 Test Specifications • Data in TTCN-3 is used • internally to the test component • as information passed between test components within the test system • as test data transmitted to and received from the System Under Test (SUT) • The grouping mechanism can be used to collect data definitions of a similar ‘kind’ e.g., • Service Primitives (ASP) • Protocol Data Units (PDU) • internal coordination messages

47. Constants (and Variables) /* Constants are defined in the definitions part of a module (and inside functions and test components) */ constant integer MyConstant := 1; /* Variables can only be declared in the module control part (and inside functions and test components) TTCN-3 does not support global variables*/ var hexstring MyHexString;

48. Example Message Definition // Complex messages can be defined by records type recordMyMessageType MyMessage := { field1 integeroptional, field2 integer, field3 integer, field4 integer, field5 booleanoptional, field6 hexstringoptional, field7 AnotherMessageType, field8 bitstring, field9 charstring (1 .. 20) }

49. Data Templates • Templates are special data values that have the following characteristics • matching mechanisms • they can be parameterized • they can be modified to provide a simple form of inheritance • Templates can be applied to simple types as well as complex types • Are most often used to define the possible received values of messages and responses to remote procedure calls

50. Matching Mechanisms