1 / 27

Certification Test Tool Update

Certification Test Tool Update. Toby Nixon Program Manager Microsoft Corporation. Assumptions. Familiar with UPnP™ v1 architecture and terminology Interested in evaluating a device’s compliance against the UPnP™ standard Pursuing the UPnP™ Logo. Overview. Certification Test Tool purpose

dareh
Download Presentation

Certification Test Tool Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Certification Test Tool Update Toby Nixon Program Manager Microsoft Corporation

  2. Assumptions • Familiar with UPnP™ v1 architecture and terminology • Interested in evaluating a device’s compliance against the UPnP™ standard • Pursuing the UPnP™ Logo

  3. Overview • Certification Test Tool purpose • What is tested • Test capabilities • Using the tool and interpreting results • Update on the next release

  4. Purpose of Cert Tool • Drive the certification of UPnP™ devices • Execute device independent protocol tests against the UPnP™ v1 standard • Support device dependent syntax and semantic testing

  5. What is Tested • Device-independent protocol tests • Device-specific syntax tests • Security Device tests • Device-specific semantic tests

  6. Protocol Tests • Validate SSDP and HTTP packets and their sequence • Test the following stages: Addressing, Description, Discovery, Security, Control, Eventing • Execute regardless of device type

  7. Syntax Tests • Actions to be tested for each device type are specified in an XML configuration file • Syntax tests are created by editing the <ServiceId>.SyntaxTests.XML file • Working groups provide the XML syntax test configuration files • Tests are run during the Control and Eventing test stages • Actions are invoked in the order in which they appear in the XML file

  8. Device Security Tests • Available in test tool versions 1.4 and above • Device Security tests as part of the service syntax tests (3.*) • The test configuration file is ‘DeviceSecurity1.SyntaxTests.xml’

  9. Semantic Tests • Verify specified output parameters, error conditions and events • Create semantic tests by editing the <ServiceId>.SemanticTests.XML file • Working groups provide the XML syntax test configuration files • Actions must be within the same service • Results of one action cannot be used as input parameters in another action

  10. What is Not Tested • Does not test performance specifications • Does not certify device extensions external to the UPnP™ standard • Does not test incorrect inputs, boundary conditions • Does not test product capabilities Tests on-the-wire interoperability

  11. Test Environment • Test tool requires Windows XP • Recommend XP SP2 (for IPv6 and HTTP/1.1 testing) • Run only one UPnP™ device and a single instance of the test tool on an isolated network • Turn off LAN connection settings (i.e. proxy settings) in Internet Options control panel on Windows XP

  12. Test Environment Diagram Device Under Test Cert Tool running on WinXP Isolated Network Hub DHCP Server for Addressing tests Network Sniffer (optional)

  13. XML Configuration Files • <DeviceId>.xml • Lists the implemented services and embedded devices • <ServiceId>.xml • Lists the implemented Actions and State Variables • <ServiceId>.SyntaxTests.xml • Lists the Action names and the ‘In’ arguments for each Action • <ServiceId>.SemanticTests.xml • Lists the Action names, ‘In’ arguments and the expected ‘out’ arguments or error code.

  14. Using XML Config Files • Working Committees • Write the XML config files for standardized DCPs • Distribute XML config files on the UIC web site • The test tool uses the XML files to verify • Devices have all required services • Services implement required Actions and State Variables • Devices respond correctly for each Action invocation • Proprietary devices can be tested but require the respective XML config files

  15. Device Description <device> <deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType> </device> <device> <Optional/> <deviceType>urn:schemas-upnp-org:device:LANDevice:1</deviceType> </device>

  16. Service Description <serviceStateTable> <stateVariable> <name>EthernetLinkStatus</name> <dataType>string</dataType> <allowedValueList> <allowedValue>Up</allowedValue> <allowedValue>Down</allowedValue> </allowedValueList> </stateVariable> </serviceStateTable>

  17. Syntax Tests XML File <TestCase> <Id>1</Id> <Category>Valid Action And Valid InArgs</Category> <ActionName>GetEthernetLinkStatus</ActionName> <ExpectedReturnCode>ACTION_AND_INARGS_ARE_VALID</ExpectedReturnCode> </TestCase>

  18. Semantic Tests XML File <ManualAction> <ActionInstruction> Remove cable. </ActionInstruction> <VerificationInstruction> Verify link light is off. </VerificationInstruction> <ExpectedSuccessResponse/> </ManualAction>

  19. Test Tool UI

  20. Select Test Options

  21. Step Mode / Run Until Failure • Step Mode • Pauses between each test • Click the “Next” button to continue • Useful when debugging a failure • Run Until Failure • Pauses the test execution at the first failure

  22. Interpreting the Test Log Test Case 2.0.0.1 Passed Test Case 2.1.0 Failed to Run Failure Reason :Precondition Failure - Unable to Construct Device Discovery Message List. Reason : Unable to Acquire Root Device Description Url.Badly formed Notify Alive Packets Header Value. Reason : NT Header Value = TestDevice is Not Compliant to UDA V1.0

  23. Helpful Hints • Review the test log for failure details • Use a network sniffer • UPnP™ parser available for Microsoft Network Monitor • Useful for reporting problems • Use Step Mode to help isolate an issue • Refer to the Test Tool User documentation for descriptions of each test case

  24. Helpful Hints Continued • A full certification pass takes about 25 minutes to complete • For testing purposes, set the CACHE-CONTROL in the device to be less than 60 for SSDP alive notifications • In your final device implementation, set the CACHE-CONTROL according to UDA recommendations

  25. Update on next release v1.5 • Planned for 2nd Quarter 2005 • New features • Using MSEARCH to initiate tests instead of NOTIFY for faster test runs • Improved logging messages for better diagnostics • Testing on non isolated networks and presence of multiple root devices • Embedded devices testing • Comparing arguments and passing parameters across different actions • Presence of extra header information in Discovery, Description, Control and Eventing tests

  26. Q&A • Certification Test Tool • Is made available by UIC • Tool is posted on www.upnp-ic.org • Documentation is included with the tool • For debugging, Cert Tool in UPnP™ Forum website has sources • Support • UIC members receive one free support call • Newsgroup: microsoft.public.upnp • Questions • Email: upnpcert@microsoft.com

  27. Foundation of the Connected Home

More Related