Work Plan for Testing the LIS and EHR Systems

1. Work Plan for Testing the LIS and EHR Systems • Define Test Flow based from Work Flow • Define a testing methodology • Develop high-level requirements for LIS Test Harness and LIS Test Tool • Identify conformance requirements external to machine-processable XML message profile (on-going) • To include conditionals and variations in profiles (e.g., OIDs/no OIDs) • Include challenging cases (e.g., parent/child) • Determine an initial set of use cases for testing (on-going) • Select from in-scope tests • Vocabulary subgroup identified 137 tests that accounts for 99% of lab results • Select core set of 5-10 (Start with one to establish template) • Develop test cases • Test Data (Elincs/vendor test plans?; vendor data; NIST data for ELR IG (2007) and PH IG (2010) • Create Test Messages • Variation to core messages • Create coverage analysis mapping test case coverage to conformance requirements • Continue until all conformance requirements handled • Organize into a test plans (start with the EHR test plan)

2. Proposed Approach for Testing the EHR System Inspection testing LIS Test Harness LRI Sender ORU R01 Message LRI Receiver EHR RI (System under Test) ACK R01 Message automated testing The EHR system is required to consume lab results messages that conform to the referenced standards and “incorporate” the data into their workflow. The LAB system is a test tool and in this layout is a test harness. The test harness will maintain a collection of pre-defined test cases that will include a range of conformant and non-conformant messages covering various lab results and scenarios. The tester will also have the option to edit the test messages. The tool sends LRI (sender profile) messages to the EHR (SUT). The EHR is expected to consume the message (as specified by the LRI receiver profile specification) and “incorporate” into their system The EHR can be tested using a combination of inspection testing and automated testing. The tests are designed to verify to a reasonable degree of certainty that the EHR correctly “incorporated” the lab results. For the most part this will be black box testing. The Lab test tool will generate a juror document for inspection testing and a validation context for automated testing (of the acknowledgement message).

3. EHR Validation Test Workflow I * • Select from in-scope tests (small core set) • Typical and high priority • For each create relevant lab results data Develop Specific Use Case Scenarios • Document (e.g., in Excel) • Export to word document (resembles lab data sheet) • Add messaging information and map to HL7 (e.g., Excel) Create Test Data • Create Message Profiles (use MWB) • Generate Messages (Message Profile + Test Data) Create/Generate Message • Verify that the message is correct according to the test case (e.g., contains correct LONIC code; missing element) • This is more than validation of the message Verify Message • Generate a juror document for inspection testing • Generate validation context file for automated testing Create Test Validation Artifacts • Configure message (e.g., Sending Application ID) • Configure test system for routing information • Send message to EHR SUT using MLLP Send Test Message to EHR SUT * The core set of messages can be modified to target specific test points

4. EHR Validation Test Workflow II • Receive acknowledgement message • Validate using message profile and validation context file • Generate Report Receive/Validation Acknowledge Message • Tester will instruct vendor to demonstrate incorporate of lab results into EHR system (compare w/ juror document) • Tester will use tool to manually add test outcome Perform Inspection Testing • Combine automated and inspection testing results • Display/Save/Print reports Combine Validation Reports End

5. Questions/Concerns (for testing the EHR) • What assumptions can we make about the EHR system? • What is the EHR required to display? • Is it sufficient to adequately assess the system? • What does CLIA buy us? • How do we make the link between what we sent and how they have incorporated it into their system? • E.g., the EHR will probably not display a LONIC code; it may map the code to its internal code (representation) and display only the internal description as a text string—what is reasonable to accept? • What are other things to look out for? • Could leverage other MU requirements (e.g., create CDA or send Lab results message to public health) • What are the elements that need to be inspected/verified? • i.e., what needs to be in the juror document • How can we ensure that every element specified in the IG is handled correctly in the EHR? • Is it practical to assess each required element without automated testing? If so how? • What can be validated automatically? • How do we test the incorporation of vocabulary? • Inspection testing • View configuration files?

6. Next Steps • Develop Test Plans • LIS • EHR (initial focus) • Test Plans to include • Work Flow • Test Flow and methodology • Test Cases • Test Data (creation of a subgroup) • Conformance requirements and coverage analysis appendix • Dimensions (OIDs, no OIDs, minimally populated, maximally populated, invalid, etc) • Develop high-level design architecture/requirements • LIS Test Tool • LIS Test Harness • Note: tool development discussion will not be part of this subgroup • We’ll have periodic tool demonstration and review