1 / 31

NIST HL7 V2 IHE-PCD Pre- and Connectathon Test Tools (2011-2012 Cycle 6)

NIST HL7 V2 IHE-PCD Pre- and Connectathon Test Tools (2011-2012 Cycle 6). John J. Garguilo and Sandra Martinez National Institute of Standards and Technology October 11 th 2011 (WebEx 2 – 3 PM) Contact: john.garguilo@nist.gov. NIST IHE-PCD V2 Test Effort NIST Team Members.

ryanf
Download Presentation

NIST HL7 V2 IHE-PCD Pre- and Connectathon Test Tools (2011-2012 Cycle 6)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NIST HL7 V2 IHE-PCD Pre- and Connectathon Test Tools (2011-2012 Cycle 6) John J. Garguilo and Sandra Martinez National Institute of Standards and Technology October 11th 2011 (WebEx 2 – 3 PM) Contact: john.garguilo@nist.gov

  2. NIST IHE-PCD V2 Test EffortNIST Team Members • John Garguilo (john.garguilo@nist.gov, 301-975-5248) • Sandra Martinez (sandra.martinez@nist.gov, 301-975-3579) • Julien DeShayes (julien.deshayes@nist.gov Guest Researcher) • Jing Gao (jing.gao@nist.gov Guest Researcher) NIST HL7 V2 Tool Web sites Instance Test Environment: http://hit-testing.nist.gov:13100/PCD-HL7WebCon/ New – Isolated Test Environment: http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/

  3. IHE-PCD Testing IHE-PCD Testing – Key Objectives • Increase test comprehensiveness & quality • Support both conformance & interoperability testing • Support for Pre- & Virtual-connectathons, actual Connectathon & enable year round testing • Remain in alignment with IHE-PCD integration profile development road map • Establish single framework for PCD covering increasing complexity and technologies over next 5 years • Coordinate with IHE “Gazelle Project” • Generate work products that companies can use in their regulatory submissions

  4. Advantages of NIST Tooling • Simplicity of use, quick feedback (via reports) • 24 hour x 365 day availability • Test Management Capability – easily record your test results • Viewable by IHE-PCD Test Manager (aka Manny Furst) • Tooling focused on IHE-PCD – including: • Current profiles, including recently adopted CPs, • Framework, Supplement, and Trial Implementation Documentation • PCD Test cases – pre-loaded to match/be synchronized with Gazelle • Level of rigor matches specifications (reference standards and TF docs) • No Conformance Profile needed • Already Integrated into the NIST tooling… • No stand-alone application installation needed (NIST web site interface) • MDC nomenclature/terminology included [hRTM] • Support of NIST team and synergy w/ IHE-PCD group

  5. Tooling Status • HL7 V2 Validation (IHE-PCD) (Required for Cycle 5, 2010-11) • Instance-type Environment (at message level) / Pre and Connectathon Setting • http://hit-testing.nist.gov:13100/PCD-HL7WebCon/ • http://hit-testing.nist.gov:8080/HL7Web/ (general use site – domain agnostic) • New for Cycle 6 (2011-12) • Isolated-type Environment / Pre-Connectathon Setting • Scenario based • Actor centric • One System Under Test (SUT) • http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/

  6. NIST IHE-PCD Test Tool: Actors / Transactions Supported • Test Tool required to fulfill 2011-12 V2 IHE-PCD Pre-Connectathon testing requirements • Results are stored in the NIST IHE-PCD tool; The IHE project manager will review the results

  7. Things to know • If port or IP is wrong, the test system times out > 2 mins • If NIST Test System tool acts as a reporter and sends to an incorrect port or IP – the message will go into cyber-space… • If NIST Test System tool acts as consumer, the parsed message will determine IP and port from reporter known issues/problems/ect… • When on vendor is registering multiple applications, the user must create multiple accounts (e.g., Co_1_Device, Co_2_Device) • The test system infuses “bad, but not too bad” messages so processing can continue; same when TS is consumer(e.g., if MSH.9 - Message Type, MSH.11 - HL7 Version, MSH.12 – are incorrect a ack with “AR’ with no furthering processing by the consumer)

  8. Issues • IPEC, WCM actors are not yet in Test System -Coming soon! • IPEC, WCM related tests have been identified but not completed – NIST is working w/ IHE-PCD Integration Profile groups to finalize (more discussion at F2F – agenda item) • In process of adding descriptions to test case documentation in NIST Test System • In the process of reconciling the test cases between Gazelle and NIST (Tool Index) • User’s Guide not completed - will be available on the site under “Documentation” (within ‘Documentation’ tab)

  9. NIST PCD Test Tool: Support • IHE-PCD Google Group • http://groups.google.com/group/ihe_pcd_testing • Preferred method of communication for discussing common issues • Go to the site and request membership • Administrative (only) issues can be sent directly to john.garguilo@nist.govsandra.martinez@nist.gov or julien.deshayes@nist.gov • Issues Tracker (suggested, not yet created) • http://code.google.com/pcdtool/issues/list • Google utility to report issues, bugs, and enhancement suggestions

  10. Troubleshooting • Site is unavailable • Typically means we are redeploying the application to make updates and the tool • We will announce redeployments • The downtime will be in the neighborhood of 10-15 minutes • Please refresh the browser following the redeployment • Server is controlled by central computing facility and may be unavailable • Site appears to be down • Restart in a new browser window • Site has timed out • This will happen if the tool is open in an inactive browser window • To correct, refresh the browser

  11. Troubleshooting (continued) • No response from console log • You may have sent to a destination that we could not process (i.e., an incorrect receiving name) • We will send back an acknowledge/error over the network • On the console log nothing will happen • You need to check the destination on your end • If you determined that this was the problem you can resend a properly addressed message; we’ll still be listening up to the session timeout • Completed report is missing • When we discover an error in the test case (i.e., the test case is wrong) we fix it • Tests that passed using an incorrect test are no longer valid • This should not occur often

  12. NIST Testing Strategy

  13. NIST Testing Strategy Test Environments • Instance Testing • Conformance (e.g., against HL7 2.x or CDA) • Implementation conforms to Spec. on which it is based • IHE Model: ~Virtual and Pre-Connectathon • Isolated System Testing • Includes Instance Testing Activities • Protocol Conformance • Functional Behavior Conformance • Features and Operational behavior correspond to Specs. • IHE Model: ~Virtual and Pre-Connectathon • Peer-to-Peer System Testing • Includes Isolated System Testing Activities • Interoperability Testing • Testing complete application environment • May include interacting w/ Database, using Network Communications, or interacting w/ other hardware, apps, or systems if appropriate • IHE Model: ~Connectathon

  14. IHE-PCD 2011 Pre-Connectathon NIST Instance Testing Support NIST V2 Testing Tool is available for message validation using the ‘instance testing’ environment: User HL7 V2 Message Services Test Management Web Application Client HL7 V2 Message Validation Report HL7 V2 Message Validation Test Case Results HL7 V2 Message Validation Report http://hit-testing.nist.gov:13100/PCD-HL7WebCon/ • Test Artifacts • Conformance Profile • HL7 Tables • ‘Device’ Test Agents • ISO/IEEE 11073/Rosetta Terminology Test Harness (Java Code) Test Execution Registry/ Repository

  15. NIST Testing Strategy Test Environments • Instance Testing • Conformance (e.g., against HL7 2.x or CDA) • Implementation conforms to Spec. on which it is based • IHE Model: ~Virtual and Pre-Connectathon • Isolated System Testing • Includes Instance Testing Activities • Protocol Conformance • Functional Behavior Conformance • Features and Operational behavior correspond to Specs. • IHE Model: ~Virtual and Pre-Connectathon • Peer-to-Peer System Testing • Includes Isolated System Testing Activities • Interoperability Testing • Testing complete application environment • May include interacting w/ Database, using Network Communications, or interacting w/ other hardware, apps, or systems if appropriate • IHE Model: ~Connectathon

  16. IHE-PCD 2011 Pre-Connectathon NIST Isolated Testing Support NIST V2 Testing Tool is available for message validation using the ‘isolated testing’ environment: Vendor Services Test Management HL7 V2 Message Validation HL7 V2 Message Generation Web Application Client IHE-PCD Client Test Scenario IHE-PCD DOC Test Agent IHE-PCD DOR/DOF Test Agent Results HL7 V2 Message Validation Reports IHE-PCD IOC Test Agent IHE-PCD IOR Test Agent Test Harness (Java Code) Test Execution IHE-PCD AM Test Agent IHE-PCD AR Test Agent IHE-PCD IDCC Test Agent IHE-PCD IDCR Test Agent IHE PCD Device Application Report Router/Logger/Proxy System Under Test http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/

  17. Test Environment Message ValidationNIST V2 Testing Tools: IHE-PCD Validation of IHE-PCD message(s) and corresponding HL7 Profile(s) Syntax and Semantic Content Validation Against HL7 V2 message (e.g., PCD-01) Message structure (e.g., MSH,PID,PV1,OBR,NTE,{{OBX},OBX,OBX,OBX,…}) Against HL7 profile (Msg_type^Event_type^ e.g., ORU^R01^…) Against HL7 and/or user provided tables (value sets) Example of user provided table is RTM for Ref_IDs, Units, etc. Against ‘validation context’, including specific values Defined in XML (e.g., specific test case values)

  18. NIST V2 Testing Tools and Services Testing Validation Types Validation against ‘failure types’: VERSION*: The version in the message and in the profile should match. MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the profile and in the message should match. MESSAGE_STRUCTURE: The message should have a valid message structure (correct usage, correct cardinality, and correct element name). USAGE: R elements should be present; X elements should not be present in the message. CARDINALITY: Elements should be present at least the minimum times and at most the maximum times specified in the profile. It should also take into account the usage of the element (X element with a minimum of 4 should not be present in the message). LENGTH: The value of the element should have a length equal or less than the value specified in the profile. DATATYPE: For the datatype NM, DT, DTM, SI and TM, the value of the element should match the regular expression defined in the standard. DATA: The value of the element should match a constant specified in the profile, a value set specified in a table, a value or a regular expression specified in the message validation context. MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the location specified in the message validation context can't be found in the message. TABLE_NOT_FOUND*: This is a user input when a table can't be found in the table files (TableProfileDocument). AMBIGUOUS_PROFILE*: The profile should not be ambiguous.

  19. Overview of the NIST HL7 V2 IHE-PCDPre-Connectathon Test Tools

  20. NIST HL7 V2 IHE-PCD Test Tool: Access • Web-based application (no downloads or installations needed) • ‘Isolated’ Site:http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/ • ‘Instance’ Site:http://hit-testing.nist.gov:13100/PCD-HL7WebCon/ • Tool may be used in Anonymous Mode or Registered Mode • Anonymous Mode (“Guest” Users) • Does not require user registration and may be used to conduct ad-hoc system testing • Registered Mode • Required for completing the Pre-Connectathon • -- Could be used for participating in the Connectathon • Required to save Pre-Connectathon test results • Test reports are made available to the IHE project manager

  21. END-USER (VENDOR) NIST IHE-PCD HL7 v2/v3 TEST TOOL SYSTEM UNDER TEST (SUT) SPECIFICATIONS (test material that defines test assertions) NIST V2 HL7 IHE-PCD Test Tool: Operational Process INTERACTION/REPORTS MANUAL OR AUTOMATED SUT Web Application Interface MESSAGES (TEST OBJECTS) STIMULUS OR RESPONSE (MESSAGES) (via the communication protocolcurrently only MLLP) V3 – Future Work

  22. Registration, Login, and Steps for using the NIST Test System

  23. Steps for Completing Test Cases • Register/log-in • Select HL7 Version (v2 only, V3 Future add on) and Actor • Select Test Case • Set Configuration Information • Run Test • View/Save/Submit Reports

  24. Registration/Log-in • To register, click-on Register in the upper-right corner of the tool’s home page • Name • Organization Name • Email Address • Login Information • Application Name (MSH 3.1) • Facility Name (MSH 4.1) • IP Address(es) and Port Number(s) • NIST Administrator will confirm log-in credentials • Log-in is required to save test results, submit results to the IHE project manager, and gain credit for test completion NIST IHE-PCD Test Tool IHE Gazelle *Push/Pull to Gazelle IHE Project Manager can view database of results * not yet implemented

  25. Registration/Log-in Fill out the User Information section completely. Users unable to be verified by the NIST administrators will have accounts deleted. Document v2 information as appropriate for your case studies. Fields exist for multiple IP Addresses and Port Numbers if needed, based on your system configuration (future place holder). V3 Actor Information not used currently (future place holder).

  26. Viewing Account Information To view information about the currently logged in user, go to the User Account page. The Actor Information section will display the server configuration fields documented during registration. The Responder Information section will display as a reference the unique client configuration information, which the test tool dynamically generates.

  27. System Configuration Information (Dialog Box) NIST Test System Configuration User must supply the “Vendor Configuration and include all four of these fields.

  28. NIST PCD Test Tool: Support • IHE-PCD Google Group (suggested, not yet created) • http://groups.google.com/group/ihe_pcd_testing • Preferred method of communication for discussing common issues • Go to the site and request membership • Administrative (only) issues can be sent directly to john.garguilo@nist.govsandra.martinez@nist.gov or julien.deshayes@nist.gov • Issues Tracker (suggested, not yet created) • http://code.google.com/pcdtool/issues/list • Google utility to report issues, bugs, and enhancement suggestions

  29. Summary • New ‘isolated’ system available and required for IHE-PCD cycle 6 (Fall Pre-Connectathon 2011) • Potential use in 2012 January Connectathon (simulate actors) • Isolated-type Environment • Scenario based • Actor centric • One System Under Test (SUT) • System will be “official” upon Steve Moore’s announcement of Gazelle ready (registration and test cases) • Start testing after this week’s F2F • Let’s work collaboratively to advance rigor… • Establish Google group for sharing conversations/questions/issues… • Contact Manny Furst for IHE-PCD Procedural issues… • Contact NIST for test tooling issues… • Record and document any findings/inconsistencies/errors and work with PC/TC and Integration Profile sub-groups - Develop CPs as needed…

  30. URLs and References • Additional Tutorial Reference • Last year’s ‘instance’ test system tutorial recording (Nov 2010) • https://ihe.webex.com/ihe/lsr.php?AT=pb&SP=MC&rID=45696462&rKey=120ce82d2491d896 • This year’s tutorial recording (Oct 2011) • Current NIST V2 HL7 IHE-PCD test sites‘Instance’ Test Environment: • http://hit-testing.nist.gov:13100/PCD-HL7WebCon/ • ‘Isolated’ Test Environment(New for Pre-Connectathon this Fall): • http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/ • NIST Healthcare & Medical Device Communication Testing Web sites • www.nist.gov/healthcare • www.nist.gov/medicaldevices

  31. Thank You! • Questions • Discussion • NIST Looks forward to advancing device testing rigor… … with your help!… 

More Related