190 likes | 203 Views
This article discusses the importance of interoperability in ICT standardization and how well-defined standards and systematic testing can increase the likelihood of interoperability. It highlights the unique resources available to ETSI, such as TC MTS, ETSI PTCC, and ETSI Plugtests Service, which support the development of technology standards and test specifications. The article also explains how PTCC assists ETSI Technical Bodies in specification, validation, and testing, and provides examples of its areas of activity, including GSM, Wi-Fi, and VoIP. Additionally, the article emphasizes the need for designing for interoperability, specifying external interfaces and software APIs, and using profiling and interworking to achieve interoperability.
E N D
Interoperability in the DNA of standards: Ensuring interoperability between building blocks Anthony Wiles, ETSI Protocol and Testing Competence Centre (PTCC) @metis Kick-off meeting 30-31 March 2005
Test Specs Specification Testing Development In Pursuit of Interoperability • Ultimate aim of ICT standardisation is interoperability • Likelihood of interoperability is increased with • Well-defined, accurate and unambiguous standards • Systematic testing of products based on those standards • ETSI produces Base Standards and Test Specifications
Unique Resources Available to ETSI Technical Bodies • TC MTS (Methods for Testing and Specification) • Develops specification methodologies and techniques • http://portal.etsi.org • ETSI PTCC (Protocol and Testing Competence Centre) • Supports ETSI Technical Bodies with the development of: • technology standards: protocols, services, APIs etc. • test specifications • http://www.etsi.org/ptcc • ETSI Plugtests Service • Organises interoperability events • http://www.etsi.plugtests
How Does the PTCC Help? • Assist ETSI Technical Bodies on the use of state of the art techniques for • Specification, validation and testing • Good working practices (Standards Engineering) • Pragmatic and flexible approach • Based on experience • Help to develop usable methodologies • For ETSI’s current and emerging needs • Knowledge transfer • Quality through Continuity
Who PTCC Supports • Technical Bodies (TB) • Technical Committees • ETSI Projects • Partnership Projects etc. • Chairmen, Rapporteurs, Individuals • Working Groups (WG) • STFs (Specialist Task Forces) • PTCC budget for test specifications (15-20 STFs per yr) • ETSI Secretariat
Industry Products mature from prototypes to commercial products Interoperability Testing (Unit) Conformance Testing Interoperability Test Specifications Conformance Test Specifications ETSI TB: Development of base standards ETSI PTCC: Support on Protocol Specification and Testing Good practice, pragmatic application of methodology, text complemented by the use of specification languages: UML, SDL MSC, ASN.1, IDL, XML etc. Planning, methodology, test specifications, TTCN, ISO-9646, TS 102 237 PTCC Support in Standards Development Certification Specification, validation and simulation
Typical PTCC Areas of Activity • GSM and 3G (UMTS) terminals • Wi-Fi: HiperLAN/2, HiperMAN, HiperACCESS • Cordless phones: DECT • Access terminals: FSK, SMS • DSL Splitters • ISDN, Broadband ISDN • OSA (API, IDL) – web services • NGN • VoIP: H.225, H.248, H.245 (ITU), SIP (IETF) • SIGTRAN • Smartcards • DECT • IPv6: Core, IPSEC, Mobility, v4 -> v6 • DSRC (ITS) • DMR • ... future: More telematics/ITS, more security?
Design for Interoperability • Interoperability takes place on external interfaces • Normative (mandatory) features • Options • Consequences must be clear • Abnormal behaviour should be well-defined • Robustness • Never specify how product is to be implemented internally • Different levels of abstraction e.g., 3-stage approach • Methods like SDL, ASN.1, XML, MSC and UML used to specify these interfaces • Data transferred on protocol interfaces • syntax and encodings (ASN.1, XML) • Behaviour of modules as seen from these interfaces • TTCN and test methods used for black box testing • No need to know how product is implemented • External interface is tested according to specification
Software Interfaces: APIs • APIs are external interfaces to software • specification of external interface: • Protocol used at interface • Data exchanged on interface • Behaviour displayed by interface • API can be specified in high-level language like Java, C++ etc. • No specification of software system as a whole • Interoperability is difficult without well-defined external interfaces or connectors • Standardised test procedures required to demonstrate compliance with interface definition • Use of TTCN (TTCN-3)
Profiling and Interworking • Many standards are ‘open’ • Include many options • Interoperability requires profiling • e.g. ETSI TISPAN NGN • Options are screwed down (dropped) • Works with ‘similar’ technologies • Implement and test the profile • Significantly different technologies • Interworking functions • e.g., SIP – H.323 • General reluctance to specify ‘profiles’ • Indications that situation is changing
Conformance Network Integration Robustness Performance Interoperability RF/EMC Different Kinds of ETSI Test Specifications
Characteristics of Conformance Testing • Is unit testing • Tests a single ‘part’ of a product (e.g., a protocol layer) for conformity to the requirements specified in the base standard (specification) • Tests over standardised interfaces • May not be available to ‘normal’ user • Works at a 'low' level • Tests at the protocol (message/behaviour) level (bits) • High control and observability • Means we can explicitly test error behaviour and other 'difficult' scenarios • Repeatable • Requires a test system (and executable test cases) • Can be expensive (e.g., radio-based test system) • Conformance Testing is DEEP and NARROW
Limitations of Conformance Testing • Does not prove end-to-end functionality (interoperability) between communicating systems • Conformance tested implementations may still not interoperate • this is often a specification problem rather than a testing problem! • need minimum requirements or profiles • Does not test a complete product • Tests the parts, not the whole • often a product is more than the sum of its parts • Does not test proprietary features, functions etc. • but this may well be done by a manufacturer with own conformance tests for proprietary item
Characteristics of Interoperability Testing • Do not confuse interoperability testing with interoperability demos • Shows that two products interoperate • Within what may be a limited scenario • Tests Functionality (of an ‘entire’ product) • Tests the ‘whole’, not the parts • e.g, protocol stacks ... • or protocol stacks + applications • High-level (as perceived by users) • Shows function is accomplished (but not how) • Does not necessarily require a test system • Uses existing interfaces (standard/proprietary) • Usually lower-cost than conformance • But can (should be) automated => test drivers • Interoperability Testing is BROAD and SHALLOW
Limitations of Interoperability Testing • Does not prove interoperability with other implementations with which no testing has been done • Does not prove that a product conforms • Interoperable products may still be non-conformant • Cannot explicitly test error behaviour or unusual scenarios • Or other conditions that may need to be forced (lack of controllability) • Has limited coverage • May not be repeatable
Interop Testing Conformance Testing Interoperability Conformance Testing and Interoperability Testing complement each other • Not always a case of either ... or ... • Though sometimes it has to be for practical or economic reasons • But for the best probability of interoperability between products – do both!
Summary • Standards can be designed for interoperability • Standards should be engineered • Plan for testing (early) • Do the right kind of testing and test in parallel
Interoperability in the DNA of standards: Ensuring interoperability between building blocks The End Thank you for your attention