120 likes | 465 Views
Conformance Testing. Lynne S. Rosenthal Software Diagnostic and Conformance Testing Division (301)975-3353 lynne.rosenthal@nist.gov. Outline. Software Standards and Testing Conformity assessment Conformance Testing what is it why bother how it works. Software Standards.
E N D
Conformance Testing Lynne S. Rosenthal Software Diagnostic and Conformance Testing Division (301)975-3353 lynne.rosenthal@nist.gov
Outline • Software Standards and Testing • Conformity assessment • Conformance Testing • what is it • why bother • how it works
Software Standards • Promoteportability & interoperability • Success depends on: • Efficiency of standard development process • Quality of resulting standard • Ability to ensure implementations conform to the standard
Software Testing • Testing - process of trying to find errors by means of experimentation • Testing of realistic system not exhaustive • can show presence of errors not their absence • Testing is either • structural testing (white box) • functional testing (black box) • conformance testing
Conformance Testing • Determines which areas of the specification* are implemented correctly. • tests capture the technical description of the specification • testing is bound in scope by the specification • internal structure of product-under-test not accessible *publicly available standard
Why: conformity assessment • ISO Guide 2:“any activity concerned with determining directly or indirectly that relevant requirements are fulfilled.” • communication between buyer and seller • buyers increased confidence • sellers substantiate claims
Why: • Improves quality of software • Objective method for evaluating products • ensures presence of required characteristics • Promote portability and interoperability
How:building tests • Falsification Testing • find errors by means of experimentation • show presence of errors not their absence • prove non-conformance • Tests based on: • required functionality • logical errors by misunderstanding requirements • errors from boundary conditions and divergence • common programming errors
Requirements Software Conformance Tests Test programs
When: • Develop in parallel with specification • feedback to specification for clarification, correction, etc. • Develop in partnership with product developers
Summary: Conformance tests • Determine if a product has required characteristics • Increases product confidence • Communication between buyer and seller • Improve quality of software