ATML Test Description

1. ATML Test Description Orlando, FL January 2007

2. Agenda • Status • Feedback from Candidate evaluation • Review standard draft • Discuss examples to be included in Annex A • Schedule for review and ballot ATML Test Description

3. Status • Created Draft 1 and Draft 2 (internal review) • Completed review • At ATML Meeting • Discuss document contents & structure • Discuss contents of Annex A • Remaining work • Incorporate changes in Draft 3, post for public review • Create Annex A, include in Draft 4, post for public review • Continue Candidate review, collect feedback, update schemas & standard ATML Test Description

4. Feedback from Candidate evaluation • Web Services using elements from Test Description & Test Results (prototyping) • No problem • Using Test Description schema for functional testing (analysis) • A Test may verify multiple UUT characteristics this may result in multiple Outcomes; all the Outcomes may be used for a diagnostic decision • Possible solutions: • Support for multiple outcomes for tests, steps, etc. • Use “combination” outcomes • Use schema extensions (would occur in multiple places) • Current design is consistent with Test Results and 1232. Test Results schema supports recording of individual measurements & limits; “partial” outcomes can be inferred if needed. Test Description schema supports the description of Test Results and limits. This appears to be sufficient. No change. ATML Test Description

5. Standard Draft - Structure • Overview • Normative references • Definitions, acronyms and abbreviations • Test Description schema • Conformance • Extensibility • Annex A (informative)– Users information and examples • Annex B (informative) – Glossary • Annex C (informative) - Bibliography ATML Test Description

6. Standard Draft – Structure… • Test Description schema • Elements • Root element • Child elements (expanded down to XML schema types, Common types and schema-defined types) • Attribute Groups • Simple Types • Complex Types (organized by functionality); Sort alphabetically? Or describe functional classification? Add class diagram? • Type • Child elements (expanded down to XML schema types, Common types and schema-defined types) • Sort types alphabetically • Include classification in introductory subsection; for each type, provide reference to the section describing it (should be converted into PDF hyperlink!) ATML Test Description

7. Standard Draft – Structure… • For each element and complex type (as applicable): • Description • Attributes • Name • Type • Description • Child elements • Name • Type • Description • Use – make sure cardinality is recorded where applicable (ex. max 2 elements) • Postfix names of abstract types with “(abstract)” • Description of “choice” situations • Use typographic convention for elements that belong to a choice; is there a quasi-standard convention? • References to Common types • Description of constraints not enforced by schema (in descriptive text of attributes or elements) • Describe uniqueness and referential integrity constraints • In specialized table (for the element where they are defined), or • In descriptive text of attributes or elements • Both ATML Test Description

8. Standard Draft - Contents • Power requirements – can be described in two formats • 1. Under GeneralData • TRD-inspired format (DC Power Requirements and AC Power Requirements) • Ripple under DCPowerRequirement – is this amplitude or RMS? • What is PhaseReference under ACPowerRequirement? • Adequate for documentation • 2. Under SignalRequirements (along with stimuli & measurements) • IEEE 1641 signal format • Adequate for programmatic use • Not all characteristics of Format 1 are supported (ex. current capability of voltage source) • If both descriptions are used, they must be consistent; this is not enforced by the schema • Keep both formats? • Remove Format 1 if format 2 can accommodate all data (Matt will investigate) ATML Test Description

9. Standard Draft – Contents… • PerformanceCharacteristics • Descriptive text includes “The description shall contain sufficient detail that any "black box" with the characteristics described could be inserted in the system without degrading the system performance.” Keep this? YES • Descriptive text includes “Also used to assure consistency at all levels of maintenance as well as compatibility between maintenance and QA test results.” Not sure what this means. • Inputs/Input/Characteristics – these are acceptable values; different word? • Outputs/Output/Characteristics – these are expected values; or guaranteed? ATML Test Description

10. Standard Draft – Contents… • Initialization • Descriptive text includes “A “Failed” outcome returned by the referenced test indicates that initialization has failed. When this situation occurs, the execution of the test group should not continue.” OK? Yes • Termination • Descriptive text includes “A “Failed” outcome returned by the referenced test indicates that termination has failed. When this situation occurs, the behavior of the test program may be application-specific. For example, the test program may reset the instrumentation, to bring it in a known state.” OK? Yes ATML Test Description

11. Standard Draft – Annex A • Available sample instance documents – may be used to build examples: • Basic example: sequence, tests, outcomes, parameters, test results • SequenceCall: calling a sequence from another sequence • FaultExample: fault detection and isolation • FailureExample: failure detection and isolation • Params: passing data between tests via parameters • PrePostConditions: usage of Pre-conditions and Post-conditions • StdSignals: use of IEEE 1641 signals as test parameters • PortExample: use of signal ports as test parameters • GlobalSignals: signals that exist for the duration of multiple tests • Actions: use of actions to describe test behavior • ActionsRepeatAndConditional: use of ActionRepeat and ActionConditional ATML Test Description

12. Standard Draft – Annex A… • Proposed examples: • One or more use cases for extensions (José) • Examples showing all data types (José) • Sequence call example with balanced tree (José) • Test group with different values from parameters; using test group parameters in tests and actions (José) • Extend Basic Example to include more UUT Description, General Data, Signal Requirements & Performance Characteristics data (Ion) • Would be nice to use a real(istic) UUT. • Do we have to include the complete XML in the specification? Or just make it avilable on the IEEE web site? This determines the complexity of the examples. • Jose will look into this. • Can have sample instance documents on IEEE web site; not required to have them in spec (they are pretty useless as the XML cannot be transferred in a file, for use w/ tools). Will probably have short snippets where appropriate. • Look at 1671 Annex A for a disclaimer applicable to examples! ATML Test Description

13. Basic example • Test, Outcomes • Parameters • Using data types (may have a few types in XML example; should have all exemplified through XML snippets in document) • Test Results & Limits • Using data types • TestGroup – sequence • Faults • Calling sequence from another sequence • Example 1: Steps reference Test Groups; 3 levels (Root Test Group, called Test Groups; called Tests • Example 2 (maybe in Actions instance document): “TestGroupCall” Behavior; parameterized; 2 calls w/ different actual parameters • Conditions • Passing data between Tests via parameters • Passing parametric (numeric) data between Test Group and Test • Extensibility – using <Extension> • Possibly: a custom format for describing Test Behavior ATML Test Description

14. IEEE 1641 signals example • Signals as Test Parameters • BSC & TSF (from 1641 example TSF Library) • Ports as Test Parameters • Measurements as Test Results • Global Signals • Extensibility – custom TSF library • Actions example • Signal Actions • Repeat & Conditional • MessageIn & MessageOut • Passing parametric data (signal definitions) between Test Group, Test and Actions • Extensibility – type derivation ATML Test Description

15. Schedule • Remaining work • Incorporate changes in Draft 3, post for public review • Create Annex A, include in Draft 4, post for public review (for Madrid) • Continue Candidate review, collect feedback, update schemas & standard • Schedule • Finalize candidate evaluation • Use of capabilities in conjunction with Test Equipment capabilities • Finalize standard review • Ballot • Tentative schedule for balloting other ATML components • .3 and .4 by April '07 • .2 by May '07 • .5 and .6 by July '07 ATML Test Description