1 / 16

An overview on the Testing and Test Control Notation version 3 (TTCN-3)

An overview on the Testing and Test Control Notation version 3 (TTCN-3). Helmut Neukirchen Háskóli Íslands helmut@hi.is http://www.hi.is/~helmut. Outline. About me What is TTCN-3? TTCN-3 example Test automation with TTCN-3 Conclusions. 1. About me. Helmut (í. Helmút) Neukirchen:

tommy
Download Presentation

An overview on the Testing and Test Control Notation version 3 (TTCN-3)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An overview on the Testing and Test Control Notation version 3 (TTCN-3) Helmut Neukirchen Háskóli Íslands helmut@hi.is http://www.hi.is/~helmut

  2. Outline • About me • What is TTCN-3? • TTCN-3 example • Test automation with TTCN-3 • Conclusions Helmut Neukirchen: TTCN-3

  3. 1. About me • Helmut (í. Helmút) Neukirchen: • Born and educated in Germany: University of Aachen, University of Lübeck, University of Göttingen. PhD in Computer Science. • Since 2008: • Associate professor for Computer Science and Software Engineering at University of Iceland. • Working in the field of software quality and distributed systems, in particular testing distributed systems and quality of tests. • Test methodology, test languages, test tools, test metrics, test “smells”, test refactoring, test patterns, standardisation of tests. • If you have interesting testing problems for students to work on as part of their BSc, MSc, PhD project: contact me! • Office in Tæknigarður, best contacted via e-mail: helmut@hi.is Helmut Neukirchen: TTCN-3

  4. 2. What is TTCN-3? • Testing and Test Control Notation version 3 • Test specification and test implementation language. • Test cases specified at an abstract level. • Hardware and operating system independent. • Can be turned into automated test cases by • compiling into executable code, • adding an adaptation layer. • Standardised technology: • European Telecommunication Standards Institute (ETSI). • International Telecommunication Union (ITU). • Strength in functional black-box testing of distributed systems. • However, applicable to various other domains, levels, and types of tests: • Embedded systems, distributed systems, pure software systems. • From unit test to acceptance test level. • From functional to non-functional (real-time, load, performance) tests. Helmut Neukirchen: TTCN-3

  5. 3. TTCN-3 example:Scenario • Black-box test of a message-based weather service: • Purpose of a very simple example test case: • Send weather request for Reykjavik to SUT. • Check that weather response is received containing: • location “Reykjavik” • any date, • temperature within a reasonable range (-60..+60°C) • wind speed within a reasonable range (0..100m/s). Test System System Under Test (SUT) 1. Weather request(“Reykjavik”) Test case Weather service 2. Weather response(“Reykjavik”, 25.11.2009, 3°C, 9m/s) Helmut Neukirchen: TTCN-3

  6. Test component Test component Real Test System Interface TTCN-3 example:Test architecture TTCN-3 supports distributed, parallel test components. Test behaviour is executed by a test component. • Weather service example: • SUT=Weather service, • Simple example test case: only one test component needed. • However, for some load test, we could, e.g., run the same test case in parallel on multiple test components to generate some load. Test component Specified and implemented using TTCN-3 language TestSystem Out In Abstract Test System Interface Ports Test adapter (implemented using Java, C, C++ etc.) System Under Test (SUT) Black-box Helmut Neukirchen: TTCN-3

  7. TTCN-3 example:Definition of data types and test data module WeatherServiceTestSuite { type record weatherRequest { charstring location } template weatherRequest ReykjavikWeatherRequest := { location := "Reykjavik" } type record weatherResponse { charstring location, charstring date, integer temperature, integer windVelocity } template weatherResponse ReykjavikWeatherResponse := { location := "Reykjavik", date := *, temperature := (-60..60), windVelocity := (0..100) } Definition of data types. Templates=Definition of test data. Definition of test data: powerful matching mechanism allows fuzziness (wildcards, ranges, regular expressions). Helmut Neukirchen: TTCN-3

  8. TTCN-3 example:Definition of ports and interfaces type port weatherPort message { in weatherResponse; out weatherRequest; } typecomponent TestComponentType { port weatherPort weatherOffice; } Definition of a port used for communication: allowed messages (=data types). Definition of a test component type: ports that may be used by test behaviour running on an instance of that component type. Helmut Neukirchen: TTCN-3

  9. TTCN-3 example:Definition of test behaviour testcase testWeather() runson TestComponentType { weatherOffice.send(ReykjavikWeatherRequest); alt { [] weatherOffice.receive(ReykjavikWeatherResponse) { setverdict(pass) } [] weatherOffice.receive { setverdict(fail) } } } // End of testcase control { execute(testWeather()) } } // End of module Test component type used to run test case behaviour. Send message specified by template via port. Two expected alternatives: Either expected response (i.e. matched by template) is received (PASS) or any other response is receive (FAIL). Specification of order (and, e.g., conditional execution) of test cases. Helmut Neukirchen: TTCN-3

  10. TTCN-3:Further concepts • Not presented in the previous example: • Procedure-based communication, • Timer and catching timeouts, • Distributed testing using parallel test components, • Concepts from general-purpose programming languages: • Conditional statements (if-then-else, for, while, …), • Local variables, • Functions, • Parametrisation. Helmut Neukirchen: TTCN-3

  11. 4. Test automation with TTCN-3 • TTCN-3 test cases are abstract:weatherOffice.send(ReykjavikWeatherRequest)makes no statement • how the actual communication shall be implemented: • UDP datagram? • SOAP over http? • SMS over GSM? • …? • how the data to be communicated shall be encoded: • XML? • Proprietary ASCII-based format? • Proprietary binary format? • …? Helmut Neukirchen: TTCN-3

  12. Test adaptation • Automatically executable TTCN-3 test suite = • Compiled TTCN-3 test cases + • CoDecs (encode data to be sent to SUT / decode datareceived from SUT) + • SUT adapter (communication with SUT) + • Platform adapter (operating system specific) Test tool user interface TTCN-3 Test System Test Control Logging Codecs [compiled] TTCN-3 Executable SUT Adapter Platform Adapter System Under Test (SUT) Note: TTCN-3 execution tools come with default test control, logging, and platform adapter. Helmut Neukirchen: TTCN-3

  13. TTCN-3 & Re-usability • Separation of concerns (abstract test case description vs. concrete adaptors): • Re-use TTCN-3 test cases for different SUTs: • Same, unmodified weather service test case can be used for SOAP/http-based system under test, for SMS/GSM-based system under test, etc. • Re-use adaptors for different test cases: • A SOAP/http SUT adaptor can be used for testing all kinds of Web services, an SMS/GSM SUT adaptor can be used for testing all kinds of mobile services, etc. Helmut Neukirchen: TTCN-3

  14. TTCN-3 tools • Tools for, e.g., editing, compiling, deploying, executing, debugging TTCN-3: • Telelogic Tester (IBM/Rational/Telelogic, Sweden), • TTworkbench (Testing Technologies, Germany), • OpenTTCN Tester for TTCN-3 (OpenTTCN, Finland) • TTCN-3 toolbox (Danet, Germany), • Exhaustif/TTCN (Métodos y Tecnología, Spain), • MessageMagic (ELVIOR, Estonia), • TTCN-3 Express (Fraunhofer FIRST/Metarga, Germany). • TRex TTCN-3 Refactoring and Metrics tool (University of Göttingen/University of Iceland, Germany/Iceland) • Standardised language and test-tool interfaces: • No vendor lock-in! Helmut Neukirchen: TTCN-3

  15. 5. Conclusions • TTCN-3 benefits: • Standardised test technology supporting automated tests. • Well documented, many tools, no vendor lock-in. • Compatible with any test methodology and test management approach. • Fosters re-use. • Mature technology used by industry and standardisation. • Nokia, Ericsson, Siemens, Motorola, Huawei, ETSI, WiMax forum, Open Mobile Alliance, Automotive Open System Architecture, … • TTCN-3 standard freely available (via http://www.ttcn-3.org): • ETSI European Standard 201 873, ITU-T Recommendation Z.140 • TTCN-3 risks: • Heavyweight approach (may only pay off for big, distributed systems): • Compilers are expensive (>1000€). • Another language and technology to learn. • While Unit Testing is possible (and done) with TTCN-3, I would in most cases recommend a xUnit framework for Unit Testing. Helmut Neukirchen: TTCN-3

  16. Thank you for your attention! • Make sure to use the right tools! Helmut Neukirchen: TTCN-3

More Related