1 / 23

Providing testability for ITU Recommendations

ITU-T Workshop on “New challenges for Telecommunication Security Standardizations" Geneva, 9(pm)-10 February 2009. Providing testability for ITU Recommendations. Ostap Monkewich , OMCI. Theme. “What is missing from your Recommendations that is needed for testing?”. Why do we need to test?.

peers
Download Presentation

Providing testability for ITU Recommendations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ITU-T Workshop on“New challenges for Telecommunication Security Standardizations"Geneva, 9(pm)-10 February 2009 Providing testability for ITU Recommendations Ostap Monkewich, OMCI

  2. Theme “What is missing from your Recommendations that is needed for testing?”

  3. Why do we need to test? • To show the customer that the product meets the requirements • To show that the product is likely to interoperate with other vendors • To demonstrate that Recommendations were implemented inside the product • To improve the quality of ITU-T Recommendations

  4. Need high-quality test results • Repeatable by any competent test laboratory • Can be used to compare with test results obtained for similar products • Recognized by all and accepted in all geographical market regions

  5. What kind of testing? Product Test Suite (Gold Standard) • N = 6 products or 15 pairs • Each product is tested 15 times • N = 100 or ~5000 pairs • Each product is tested 5000 times • Conformance Testing • verifies if the implementation does what the Recommendation says it is supposed to do • Interoperability Testing • checks if two implementation communicate at functionalities level • Functionalities • Every pair

  6. Frequently Asked Question • Why not do only interoperability testing? • Answer: We don’t know what to do when two implementations do not interoperate: • If we change something – are we closer to the Recommendation or farther from it? • Non-conforming changes destroy interoperability with other vendors • Serious problems will not be discovered when observing functionalities only • example: turning lights on and off in a new house is a good “interoperability” or functionality test. The house may burn down at a later time because the wiring did not conform to the standard

  7. Sources of Interoperability Problems • Recommendations • Errors in Recommendations • Ambiguities in natural language • Unverified or invalid behaviours described • Implementers’ different interpretations of text • Requirements in text, more than one meaning • No standardized questionnaire for supplier • Incompatible Recommendations/implementations • Different choices of options • Device incompatibilities • Different host system configurations

  8. In addition to Base Recommendation • Requirements Clause • Extracted from text, need no interpretation • Implementation Conformance Statement (ICS) Proforma • Supplier declares what pieces were implemented from Recommendation • Implementation eXtra Information for Testing (IXIT) Proforma • Identifies non-standardized items, how architecture, interfaces are packaged • Test Suite Structure • Logically groups test cases • Test Purposes • one test purpose/verdict per test case • Abstract Test Suite • Set of tests that covers the Recommendation

  9. Recommendations to support testing Requirements Clause ICS Proforma Execution Test Cases IXIT TSS &TP ATS ICS TC1 TC1 R1 TP1 TC2 R2 TP2 ICS Questions R3 TC3 TC3 Base Recommendation ICS Answers IXIT Information TP3 . . . . . . . . . . . . TCn TCn Rn TPn TSS & TP - Test Suite Structure and Test Purposes TP - Test Purpose ATS: - Abstract Test Suite TC - Test Case ICS: - Implementation Conformance Statement R - Requirement

  10. Requirements – NGN Draft Rec. Y.2702 Sections 7.2.3 and 8.2.1 – NGN User authentication for service access and network attachment (R-40) Each user/subscriber associated with an application service subscription is required to be uniquely addressable (R-41) It is required that it be possible for the end user to access a service simultaneously multiple times and/or from multiple devices. (R-42) It is required that it be possible to support multiple subscription profiles for an individual end-user. (O-1) It is an option that network access points supporting NGN TE and TE-BE support capabilities to allow the end user to uniquely identify the NGN provider. (O-2) It is an option that network access points supporting NGN TE and TE-BE support capabilities to allow the user to authenticate and authorize the NGN provider.

  11. Implementation Conformance Statement (ICS) Proforma Index Text Status Ref. Val. Support IV. 2.1.2 1) When User Equipment (UE) sends HTTP Request does Service Provider (SP) return http Response with <lib:AuthnRequest > 3.5.1 M _Yes _No IV. 2.1.2 2) On receipt of HTTP Request from UE does SP send a redirect http Response with <lib:AuthnRequest > in the URL M 3.5.2 _Yes _No IV. 2.1.2 x) When UE sends HTTP Request does SP inform the Identity Provider (IdP) of the receipt O 3.5.3 _Yes _No Appendix IV to NGN Draft Rec. Y.2702 on Identity management (IdM)

  12. Examples of Test Purposes inSIP Security Testing • Ensure that the IUT, after sending the initial REGISTER request to the Registrar, ignores Registrar OK response by sending a second REGISTER request • Ensure that the IUT re-sends the initial REGISTER request on receipt from the Registrar of a 401 Unauthorized response in which WWW-Authenticate header does not contain the nonce= field

  13. Implementation Conformance Statement (ICS) ICS • ICS = ICS Proforma with answers • A list of requirements and options claimed to have been implemented • Used for • Shopping for products to match for interoperability • selecting Test Cases from test suite for test execution

  14. Test Suite Structure Test Suite Test Group Test Group Test Group Test Group Test Case Test Case Test Case Test Case

  15. Test Case Structure Test Case Test Step Test Step Test Step Test Step Test Event Test Event Test Event Test Event

  16. Conclusion • First, Conformance testing then Interoperability testing • High-quality Recommendations are needed • Base Recommendations require additional parts to produce widely accepted test results • Methodology Recommendations for testing are available – attached charts

  17. Additional Slides All you need to develop high-quality Recommendations and Test Specifications

  18. Conformance and Interoperability Testing Methodology Recommendations • X.290 - General Concepts • X.291 - Abstract Test Suite Specification • X.292 - (Superseded by Z.160/170 series) • X.293 - Test Realization • X.294 - Requirements on Test Laboratories and Clients • X.295 - Protocol Profile Test Specification • X.296 - Implementation Conformance Statements • Supplement 1 to X.290 series - Generic approach to interoperability testing • Supplement 2 to X.290 series - Interoperability testing framework and methodology

  19. Testing and Test Control Notation (TTCN-3) • Z.161 (Z.140 Revised): Core Language • Z.162 (Z.141 Revised): Tabular Format • Z.163 (Z.142 Revised): Graphical Format • Z.164 (Z.143 Revised): Operational semantics • Z.165 (Z.144 Revised): Runtime interface • Z.166 (Z.145 Revised): Control interface • Z.167 (New): Using ASN.1 with TTCN-3 • Z.168 (New): The IDL to TTCN-3 mapping • Z.169 (New): Using XML Schema with TTCN-3 • Z.170 (New): TTCN-3 documentation tags

  20. Specification and Description Language • Z.100 Overview of SDL-2008 • Z.101 Basic SDL-2008 • Z.102 Comprehensive SDL-2008 • Z.103 Shorthand notation and annotation in SDL-2008 • Z.104 Data and action language in SDL-2008 • Z.105 SDL-2008 combined with ASN.1 modules • Z.106 Common Interchange Format for SDL-2008

  21. Abstract Syntax Notation One (ASN.1) • X.680 (07/02)  Information technology - Abstract Syntax Notation One (ASN.1): Specification of basic notation • X.680-X.693 (07/02) Information Technology - Abstract Syntax Notation One (ASN.1) & ASN.1 encoding rules • X.681 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Information object specification • X.682 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Constraintspecification • X.683 (07/02) Information technology - Abstract Syntax Notation One (ASN.1): Parameterization of ASN.1 specifications

  22. Abstract Syntax Notation One (ASN.1) • X.690 (07/02)Information technology - ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER) • X.691 (07/02) Information technology - ASN.1 encoding rules: Specification of Packed Encoding Rules (PER) • X.692 (03/02)  Information technology - ASN.1 encoding rules: Specification of Encoding Control Notation (ECN)   • X.693 (12/01) Information technology - ASN.1 encoding rules: XML encoding rules

  23. Supporting Recommendations Z.120 Message Sequence Chart (MSC) Z.150 User Requirements Notation - Language Requirements and Framework Z.151 User Requirements Notation - Language Definition Z.110 Application of Formal Description Techniques Z.450 Quality aspects of protocol-related Recommendations

More Related