1 / 28

Certification Test Tool

Certification Test Tool. Jon Wheeler Test Lead Microsoft Corporation. Assumptions. Familiar with UPnP v1 architecture and terminology Interested in evaluating a device’s compliance against the UPnP standard Pursuing the UPnP Logo. Overview. Certification Test Tool purpose What is tested

kennethv
Download Presentation

Certification Test Tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Certification Test Tool Jon Wheeler Test Lead Microsoft Corporation

  2. Assumptions • Familiar with UPnP v1 architecture and terminology • Interested in evaluating a device’s compliance against the UPnP standard • Pursuing the UPnP Logo

  3. Overview • Certification Test Tool purpose • What is tested • Test capabilities • Using the tool and interpreting results • Future versions • Tool is posted on www.upnp-ic.org

  4. Purpose of Cert Tool • Drive the certification of UPnP devices • Execute device independent syntax tests against the UPnP v1 standard • Support device dependent syntax and semantic testing

  5. Certification Tests Govind Ramanathan Software Development Engineer Microsoft Corporation

  6. What is Tested • Device independent protocol tests • Device specific syntax tests • Device specific semantic tests

  7. Protocol Tests • Validates SSDP and HTTP packets and their sequence • Tests the following stages: Addressing, Discovery, Description, and Eventing • Protocol tests are executed regardless of device type

  8. Syntax Tests • Test actions can be specified for services that are specific to the device undergoing test • Syntax tests are created by editing the <ServiceId>.SyntaxTests.XML file • Tests are run during the Control and Eventing test stages • Actions are invoked in the order in which they appear in the XML file

  9. Semantic Tests • Verifies specified output parameters, error conditions or events • Create semantic tests by editing the <ServiceId>.SemanticTests.XML file • Actions must be within the same service • Results of one action cannot be used as input parameters in another action

  10. What is Not Tested • Does not test performance specifications • Does not certify device extensions external to the UPnP standard

  11. Test Environment • Test tool requires Windows XP • Run only one UPnP device and a single instance of the test tool on an isolated network • Turn off LAN connection settings (i.e. proxy settings) in Internet Options control panel on Windows XP

  12. Test Environment Diagram UPnP Cert Tool running Device under test on Windows XP Isolated network DHCP server Sniffer for for AutoIP and debugging DHCP tests (optional)

  13. Configuring XML Files • Tool relies on XML configuration files • <DeviceId>.xml • Gives the Services that this device implements and the embedded devices. • <ServiceId>.xml • Gives the Actions and State Variables implemented. • <ServiceId>.SyntaxTests.xml • Gives the Action names and the ‘In’ arguments for each Action. • <ServiceId>.SemanticTests.xml • Gives the Action names, ‘In’ arguments and the expected ‘out’ arguments or error code.

  14. Using XML Config Files • Working Committees write the XML config files • Standardized config files are distributed with the tool • From the XML files the tool verifies if • Devices have all required service. • Services implement required Actions and State Variables. • Responds correctly for each Action invocation. • Proprietary devices can be tested but require the respective XML config files

  15. Device Description <device> <deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType> </device> <device> <Optional/> <deviceType>urn:schemas-upnp-org:device:LANDevice:1</deviceType> </device>

  16. Service Description <serviceStateTable> <stateVariable> <name>EthernetLinkStatus</name> <dataType>string</dataType> <allowedValueList> <allowedValue>Up</allowedValue> <allowedValue>Down</allowedValue> </allowedValueList> </stateVariable> </serviceStateTable>

  17. Syntax Tests XML File <TestCase> <Id>1</Id> <Category>Valid Action And Valid InArgs</Category> <ActionName>GetEthernetLinkStatus</ActionName> <ExpectedReturnCode>ACTION_AND_INARGS_ARE_VALID</ExpectedReturnCode> </TestCase>

  18. Semantic Tests XML File <ManualAction> <ActionInstruction> Remove cable. </ActionInstruction> <VerificationInstruction> Verify link light is off. </VerificationInstruction> <ExpectedSuccessResponse /> </ManualAction>

  19. Launch the Application

  20. Select Test Options

  21. Step Mode / Run Until Failure • Step Mode • Pauses between each test • Click the “Next” button to continue • Useful when debugging a failure • Run Until Failure • Pauses the test execution at the first failure

  22. Interpreting the Test Log Test Case 2.0.0.1 Passed Test Case 2.1.0 Failed to Run Failure Reason :Precondition Failure - Unable to Construct Device Discovery Message List. Reason : Unable to Acquire Root Device Description Url.Badly formed Notify Alive Packets Header Value. Reason : NT Header Value = SSDPBVT is Not Compliant to UDA V1.0

  23. Tips and Future Versions Jon Wheeler Test Lead Microsoft Corporation

  24. Helpful Hints • Review the test log for failure details • Use a network sniffer (e.g. MS NetMon) to monitor network activity • Use Step Mode to help isolate an issue • Read the Test Tool User doc which lists the test cases

  25. More Tips • A full certification pass takes about 25 minutes to complete • For testing purposes, set the CACHE-CONTROL in the device to be less than 60 for SSDP alive notifications. • In your final device implementation, set the CACHE-CONTROL value to be > 1800

  26. Future Version Plans • Support testing against the v1 security standard • Secure logs to prevent tampering • Include more extensive logging • Expose interfaces for developing test scripts • Support future UPnP standards (v2)

  27. Q&A • Tool is posted on www.upnp-ic.org • User documentation is included with the tool • Developer support • Questions?

  28. For the interconnected lifestyle

More Related