1 / 49

Protocol Testing

Protocol Testing. Lesson 5. Outline for today. Guidelines for testing protocols TTCN – A Standard Language For Testing Protocols Motivation The Language The usage. Guidelines for protocol testing. Guidelines. Unit Testing tests if individual units of source code are fit for use.

elma
Download Presentation

Protocol Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Protocol Testing Lesson 5 IHA præsentation

  2. Outline for today • Guidelines for testing protocols • TTCN – A Standard Language For Testing Protocols • Motivation • The Language • The usage IHA præsentation

  3. Guidelines for protocol testing IHA præsentation

  4. Guidelines • Unit Testing • tests if individual units of source code are fit for use. • A unit is the smallest testable part of an application • Integration Testing • individual software modules are combined and tested as a group • Occurs after unit testing • Coverage Testing • describes the degree to which the source code has been tested IHA præsentation

  5. Guidelines • Smoke Testing • Collection of written tests that are performed on a system prior to being accepted for further testing • Regression Testing • seeks to uncover software regressions • previously working software functionality stops working as intended • Conformance Testing • to determine whether a system meets some specified standard IHA præsentation

  6. Conformance Testing Framework • Conformance testing is the process of testing the extent to which implementations adhere to the requirements stated in relevant standard or specification • Conformance Testing is functional black-box testing • Functional refers to the correct functional behavior of an Implementation Under Test (IUT) • Black-box means that the internal structure of the IUT remains hidden IHA præsentation

  7. Conformance Testing Framework IUT – Implementation Under Test UT – Upper Tester function LT – Lower Tester function PCO – Point of Control & Observation TCP – Test Coordination Procedures ASP – Abstract Service Primitive IHA præsentation

  8. Conformance Testing Framework UT plays the role of a user that makes use of the service provided by the IUT LT plays the role of the peer entity of the IUT, i.e. the LT and the IUT communicate in order to provide the service to the IUT IHA præsentation

  9. Conformance Testing Framework PCO is a standardized interface. Typically the lower interface of an IUT is accessible only from the remote Communication is always meant to be asynchronous => PCO is modeled as two FIFO queues IHA præsentation

  10. Conformance Testing Framework TCP can be used to coordinate the actions of LT and UT. LT and UT can be on separate testers. IUT and LT communicates by means of ASP. PDUs are encoded in ASPs of the underlying service. IHA præsentation

  11. Conformance Testing Framework • Test case development starts with: • Identification of a test purpose • Description of a single requirement or set of related requirements • A test case is an implementation of a test purpose IHA præsentation

  12. Conformance Testing Framework A test case starts and ends in stable testing states. Preamble to take the IUT from stable testing state to test state. Test body is performed in order to check the test purpose. Verification to check test body Postamble to drive the IUT into a stable testing state again Test Case Scheme IHA præsentation

  13. Protocol Testing • Until now we have: • Developed our protocol layer including: • Timers • State - event model • PDUs to be exchanged • Interfaces to upper and lower layers HOW DO WE TEST OUR PROTOCOL LAYER? IHA præsentation

  14. Protocol Testing • Protocol Testing is mainly about: • Testing the state – event model. Our protocol layer Test System PDU (message) ?? Observe response IHA præsentation

  15. Protocol Testing • Testing the state – event model • Difficult part is when we send an unexpected message into our layer Our protocol layer Test System Unexpected PDU (message) Standard testing languages and tools available ?? • Ignore – take no action • Reject – return an explicit denial to the sender • Allow – take action to process the event IHA præsentation

  16. Test & Testing Control Notation Edition 3(TTCN-3) IHA præsentation

  17. Introduction • What is TTCN? • Why use TTCN? • The TTCN-3 Language • Background • Example • TTCN-3 Usage • Where is it used? • Future Outlook IHA præsentation

  18. What is TTCN? • Internationally standardized testing language for formally defining test scenarios and their implementation. • Designed purely for testing • Tool independent • Flexible and powerful language • Combined use with other languages like ASN.1 • Well-defined syntax, interchange format • Different presentation formats (e.g. tabular and graphical) • Support text-based protocols • Support for IP protocols IHA præsentation

  19. What is TTCN? • In TTCN we can…. • Define Test Suites • Define Test Cases • Define Test steps • Declare variables (test suite & test case variables) • Declare timers • Create PDUs • Etc.. IHA præsentation

  20. What is TTCN? • TTCN Edition 3 is documented in a serie of ETSI Specification • ES 201 873-1 (TTCN-3 Core Language) • ES 201 873-2 (Tabular presentation format) • ES 201 873-3 (Graphical presentation format) • etc. • etc. IHA præsentation

  21. Why Use TTCN? • Quality • a well-tested product increases confidence in the product’s quality • Time/Cost Improvement • TTCN is a language designed for protocol conformance testing • TTCN is standardized • TTCN is fairly simple • Formal • strict rules, complete BNF and operational semantics • Abstract • TTCN test suites are independent of an implementation IHA præsentation

  22. Application Areas • New application areas • Software Testing • Text-based protocols… • Additional communication paradigm • Message-based communication • Procedure-based communication • Different kinds of testing • Functional testing • Conformance testing (behavioral testing) • Scalability testing… IHA præsentation

  23. Send Stimuli Observe Responses Result TTCN Basics • Behavioral testing • send stimulus, observe response • was response received within specified time? Test System SUT IUT Implementation Under Test TTCN Test Suite PASS, FAIL or INCONCLUSIVE IHA præsentation

  24. TTCN – Example Minimal Test Configuration testcase MyTestCase() runs on DNSTester { P.send(query); P.receive(answer); setverdict(pass); stop; } IHA præsentation

  25. (s + s) ( ) TTCN-3 test systems in a nutshell • TTCN-3 specifies a test but a test system is needed for test execution • TRI and TCI standards define test system architecture • TTCN- 3 tools are required to support internal interfaces • Allows reuse of test platforms with different tools but also for different SUTs • A test system requires • A TTCN-3 tool = TTCN-3 compiler and execution environment • A test platform for a specific device under test Note: Tools come with default Test Control & Logging Test System Executor TTCN-3 Test System Test Control Logging Codecs TCI [compiled] TTCN-3 Executable TRI SUT Adapter Platform Adapter System Under Test (SUT) TCI = TTCN-3 Control InterfaceTRI = TTCN-3 Runtime Interface IHA præsentation

  26. An example adaptation: A IPv6 test system Test System Executor TTCN-3 Test System ToolLogging ParameterFile IPv6 Codecs TCI [compiled] IPv6 ATS TTCN-3 Executable TRI Ethernet Adapter Real-time Adapter Open Source FreeBSD Router IHA præsentation

  27. User’s view of TTCN-3 IHA præsentation

  28. Text format Tabular Format Graphical Format PresentationFormat3 PresentationFormatn The Core Language and Other Presentation Formats • Core format is text based (most popular) • TTCN-3 can be edited or viewed in other formats • Tabular format (for TTCN-2 people) • Graphical format (good for visual overview) • Other standardized formats in the future? • Proprietary formats possible TTCN-3 Core Language IHA præsentation

  29. Example Core (Text) Format testcase TC_resolveEtsiWww() runson DnsClient { timer t_ack; serverPort.send(m_dnsQuestion("www.etsi.org")); t_ack.start(1.0); alt { [] serverPort.receive(mw_dnsAnswer("172.26.1.17")) {setverdict (pass); } [] serverPort.receive { // any other message setverdict(fail); } [] t_ack.timeout { setverdict(inconc); } } t_ack.stop; } IHA præsentation

  30. Example Graphical Format testcase TC_resolveEtsiWww() runson DnsClient mtc serverPort DnsClient DnsPort timer t_ack m_dnsQuestion("www.etsi.org") t_ack alt mw_dnsAnswer("172.26.1.17") pass ? fail t_ack inconc t_ack IHA præsentation

  31. Example Tabular Format IHA præsentation

  32. ASN.1 Types & Values IDL Types & Values XML Types & Values Other types & Valuesn Use of TTCN-3 With Other Languages • TTCN can be integrated with types systems of other languages • Fully harmonized with ASN.1 (1997) • Harmonized with other languages • IDL, XML, C/C++ TTCN-3 Core Language IHA præsentation

  33. TTCN SDL ASN.1 • Abstract Syntax Notation One • language for describing structured information • standardized internationally (ISO/IEC 8824, ITU-T X.680) • TTCN includes ASN.1 • used for creating data descriptions and constraints • can be part of TTCN script or imported as a module • SDL also includes ASN.1 • ITU-T Z.105 • ASN.1 can be re-used across design and test! ASN.1 IHA præsentation

  34. TTCN – Example testcase MyTestCase() runs on DNSTester { P.send(query); P.receive(answer); setverdict(pass); stop; } • P.receive(answer) blocks until it receives a message that matches answer • Any other message does not unblock the tester, which then blocks forever • If no message is received, the tester will also block forever IHA præsentation

  35. TTCN Test Case Example testcase MyTestCase2() runs on DNSTester { timer t := 5.0; P.send(query); t.start; alt { [] P.receive(answer); { setverdict(pass); } [] P.receive { // any message setverdict(fail); } [] t.timeout { setverdict(inconc); } } stop; } IHA præsentation

  36. Test Suite Test Data Types Building blocks of a TTCN-3 Test Suite • Data types which specify • Structure of messages or calls and their information elements (fields, parameters) • Internal data structures (e.g., for computation) • Possibly encoding or display information • Built-in basic types • integer, boolean, float, • bitstring, hexstring, octetstring, • charstring, universalcharstring • ... and structured types • record, recordof, set, set of • union, enumerated • ... and special types such as • component, port, verdicttype, default, etc IHA præsentation

  37. Test Suite Test Data Types Actual Test Data Building blocks of a TTCN-3 Test Suite • Actual test data (values) used during testing • Constants or Templates for specific message or call parameter values • Matching expressions for allowing multiple message or call parameter values • value range, value list, wildcards, presence, length, size, permutation • regular expressions • Using also template decomposition, parameterization and modification IHA præsentation

  38. Test Suite Test Data Types Test Configuration Actual Test Data Building blocks of a TTCN-3 Test Suite • Static aspects • Test component and port types • Dynamic aspects • Dynamic instantiation and management of test components • Mappings of test components to abstract test system interfaces • Connections between test component interfaces • Management of test components IHA præsentation

  39. Test Suite Test Data Types Test Configuration Actual Test Data Test Behaviour Building blocks of a TTCN-3 Test Suite • test cases • specify sending/receiving messages, computation (e.g., checksums), and verdict assignment • can be decomposed with functions and altsteps • can (re)use default behaviour • can use timers and timeouts • test execution control (optional) • order, repetition, conditions, etc IHA præsentation

  40. Module Definitions Module Control TTCN-3 Module module EtsiDnsTests { // Test definition part control { // Test execution part // (optional) } } Module (…) IHA præsentation

  41. Definitions Part module EtsiDnsTests { // Message structure // Actual test data // Test configuration // Test Case definitions } IHA præsentation

  42. Structured Definitions Part module EtsiDnsTests { groupMessageStructure { // Defintions of message types } groupTestData { // Templates for messages instances } groupTestSystemConfiguration { // Port and component types and mappings } groupTestCases { // Test case definitions } } IHA præsentation

  43. Message Structure and Test Data type record DnsMsg // simplified message structure! { DnsMsgKind kind, charstring question, charstring answer optional } type enumerated DnsMsgKind {e_query, e_response} templateDnsMsgm_dnsQuestion( charstring p_question ) { kind := e_query, question := p_question, answer := omit // no answer } templateDnsMsgmw_dnsAnswer( charstring p_answer ) { kind := e_answer, question := ?, // any question ok answer := p_answer } IHA præsentation

  44. Test Configuration type port DnsPort message { inout DnsMsg } // Note: port types may also allow multiple different // message types or restrict the direction type componentDnsClient{ port DnsPort serverPort } // Note: component types can also define multiple port // instances of the same or different port type and // declare timers, constants or variables IHA præsentation

  45. Test Behaviour query mtc serverPort response testcase TC_resolveEtsiWww() runs on DnsClient { timer t_ack; serverPort.send(m_dnsQuestion("www.etsi.org")); t_ack.start(1.0); alt { [] serverPort.receive(mw_dnsAnswer("172.26.1.17")) {setverdict(pass); } [] serverPort.receive { // any other message setverdict(fail); } [] t_ack.timeout { setverdict(inconc); } } t_ack.stop;} IHA præsentation

  46. Module Definitions Module Control The Control Part module EtsiDnsTests { // Test definition part modulepar boolean mp_example; testcase TC_resolveEtsiWww() runs on DnsClient { // .. as in previous slide } // Test execution part control { if (mp_example) {execute(TC_resolveEtsiWww()); } } } Module (…) IHA præsentation

  47. TTCN-3 Usage IHA præsentation

  48. TTCN in 3GPP • Lessons from GSM • TTCN ATSscontribute to interoperability • powerful competitive tool for entry to the global GSM mobile market • TTCN ATSs widely used within members’ development processes • 3GPP • TTCN in Protocol Conformance Specs • 34.123-1 Prose description of tests • 34.123-2 Implementation Conformance Statement (ICS) • 34.123-3 TTCN Abstract Test Suite • TTCN will be the normative specification IHA præsentation

  49. Applications TCP/IP HID RFCOMM Control Data Audio L2CAP Link Manager Baseband RF TTCN and Bluetooth • Bluetooth SIG selected TTCN for... • definition of Protocol Requirements tests • interoperability • definition of Profile Requirements tests • application interoperability • end-user expectations IHA præsentation

More Related