1 / 15

LRI Validation Suite

LRI Validation Suite. LRI Prototype Tool Demonstration Rob Snelick—NIST January 31st, 2011. Overview. Validation Tool for LRI (Prototype for trial use) http://lri.sipilotdevelopment.org/lri-phase-2/ Web Application Firefox; Chrome; IE9 (in compatibility view enabled)

cheri
Download Presentation

LRI Validation Suite

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LRI Validation Suite LRI Prototype Tool Demonstration Rob Snelick—NIST January 31st, 2011

  2. Overview • Validation Tool for LRI (Prototype for trial use) • http://lri.sipilotdevelopment.org/lri-phase-2/ • Web Application • Firefox; Chrome; IE9 (in compatibility view enabled) • Source code will be public domain • Joint effort of NIST and LMCO (ONC) • Wiki Page—Validation Suite Resources • Google Group • Discussion forum for LRI IG, Pilots, Validation Tool, Tool and Resources Updates, Release Notes, etc • http://groups.google.com/group/hl7v2_lri_testing

  3. Timeline • Demonstration of “proof of concept” validation tool • Tuesday December 20th, 2011 • Not for use (review in validation suite meetings) • Based on Ballot 1+ IG version • Prototype for trial use • Goal is early February (~2 weeks before HIMSS)—Week of Feb. 13th • Based on IG version 0.15 (locked for Pilots) • Not all functionality and IG requirements included • On-going updates • development will continue with periodic releases • Will include errata to IG Version 0.15 • Version 1 (“Production Ready”) • 3 months after publication of IG (after ballot 1—i.e., stable IG)** • Not complete number of test cases (5-7 core messages with variation) • Version for MU testing—if in ONC rule?? • Fall 2012 (Dependent on final publication of IG)** • 3 Months after final publication **assuming current level of resources

  4. LRI Validation Suite Meetings • Agenda • Tool updates and status • User Questions/Comments • Test case/message reviews (Users can request test cases to add) • Other topics as needed • Current Schedule • February 28th • March 27th • April 24th • May 22nd • Tuesdays, 1:00 PM EST • Additional meetings to be added if necessary • HL7V2_LRI_Testing Google Group will be used for notification

  5. Validation Suite Products (Drafts) • Test Case Management Spreadsheet • Test Messages • Test Plans • LIS Test Plan, EHR Test Plan • Testing Methodologies (approach to testing) • Vocabulary Spreadsheets (value sets for the IG), XML, and GUI • Including HL7, UCUM, in-scope LOINC, in-scope SNOMED to come • Categorized in-scope LOINC to support testing • XML Conformance Profile • Message Template capturing most message constraints (requirements) • LRI IG  MWBHL7 V2.5.1 XML + CP + CS + Len HL7 V2.7.1 LRI IG XML • Validation Tool (Focus on LRI—i.e., not a general HL7 V2 tool) • LIS Mode • Context-Free (test any LRI message) • Context-Based (Generated Data Sheets from Test Cases – Scenarios) • Associated with test cases (i.e., test data) • EHR Mode • Generated Data Sheets and Juror Documents from Test Cases – Scenarios • Support for NG and GU Profiles (NG initially) • Provide feedback to implementation guide analysis group

  6. LRI Validation Suite WG Charter Overview NIST LRI Test Tool Ambulatory EHR ONC S&I Framework Test Data LRI ORU R01 S & I Framework EHR Pilots LIS LRI Validation Tool EHR LRI Test Harness LRI ACK R01 Vendor EHR Products • NIST LRI Test Tool • Suite scope limited to the requirements specified in the LRI IG • Develop test data • provided & verified by the S&I Framework community • LIS LRI Validation Tool • Used to validate vendor LIS systems & Test Harness • Develop a test plan • Capture and validate LIS LRI messages • EHR LRI Test Harness • Used to validate EHR systems • Develop a test plan • Simulation of an LIS system • Manage, Send, Receive LRI Messages • Inherently provides CLIA Testing • Targets • S & I Framework EHR Pilots • Vendor EHR Products • MU Certification • Ambulatory EHR Systems • EHR Pilots • Draft standard trial use (implementation) • Verifies LRI IG can be implemented—provide feedback • Use LRI test harness to perform conformance testing • Function of the S&I Framework LRI Pilots WG • Vendor Product EHRs • Can use LRI test harness to perform conformance testing • LRI Test Harness expected to be used for MU certification Part of validation suite deliverables Anticipated users of validation suite products

  7. LRI Validation – High Level Overview • LIS Mode • Context-Free • Context-Based (Generated Data Sheets from Test Cases – Scenarios) • EHR Mode • Generated Data Sheets and Juror Documents from Test Cases – Scenarios • Conformance Profile Browser • Vocabulary Browser

  8. LIS Context-Free Testing LIS (or Proxy) • Process: • An LRI message is created by the vendor’s LIS • The message is sent, pasted, or loaded in the test tool • A validation is performed HL7 V2 Lab Results Message • Load • Cut/paste • Send • Context Free Testing: • Context free testing will validate a LRI message created by the LIS • The context (e.g., the type and results of the lab test) is unknown to the validation tool • Therefore not all conformance requirements of the LRI implementation guide can be assessed • However, the validation provides a simple and convenient method for testing message structure and most vocabulary LIS Test Tool Validation Report

  9. Phase 1 Core Test Messages • Erythrocyte sedimentation rate • 30341-2 OBR.4 LN code • Valid Minimally Populated Message – Final Results • Valid Typically Populated Message – Final Results • Valid Typically Populated Message – Corrected Results • Valid Maximally Populated Message – Final Results • CBC W Auto Differential panel in Blood • 57021-8 OBR.4 LN code • Valid Typically Populated Message – Final Results • 27 Results (OBXs) • Lipid 1996 panel in Serum or Plasma • 24331-1 OBR.4 LN code • Valid Typically Populated Message – Final Results • 4 Results (OBXs)

  10. LIS Context-based Testing • Context-based Testing: • Context-based testing will validate a LRI message created by the LIS • The context (e.g., the type and results of the lab test) is known to the validation tool • Therefore all conformance requirements of the LRI implementation guide can be assessed • Supports various scenarios (FC, rejected specimen) • A lab test is ordered for a patient • The specimen is collected, and is received and processed in the lab • The lab result is produced and stored in the LIS database • The lab result message is created • The lab result is transmitted to an ambulatory EHR • The lab result is viewed in the ambulatory EHR Use Case Test Case EHR transmits Hemoglobin blood test order to the LIS for John Doe along with pertinent demographic and order request data Manual entry of test data Lab Results Data Sheet John Doe DOB: 05/23/1959 Gender: M Result: 13.7 g/dL Range 13.2 – 16.2 Status: Final and more… Test Data Sheet LIS HL7 V2 Lab Results Message • Load • Cut/paste • Send • Process: • A technician enters lab results data into LIS based off the data sheet provided • The message is sent, pasted, or loaded in the test tool • A validation is performed LIS Test Tool select test case Validation Report

  11. LRI EHR Testing – Test Harness Use Case • Inspection Testing can be performed by: • On-site inspection • Over a webex like technology • Screen-scraper or screen-capture (include clock) • Printed Reports LRI EHR Test Harness Test Case Inspection Testing Techniques EHR display screens Database access Configuration files Test Data Juror Document LRI Test Message Lab Message EHR Communication ACK ACK Automated Testing Acknowledgement Message Limited Utility Validation Report Validation

  12. Conformance Profile • Machine processable version of the LRI implementation guide • HL7 V2.7.1 Standard Conformance Profile • XML • Chapter 2B • Captures most of the LRI IG requirements • Message Structure, usage, cardinality, condition predicates, conformance statements, lengths, etc. • Key differences from V2.5.1 (and not handled by the MWB) • Min/max length • C(a/b) – configurable conditionals • Conformance Statements • NIST added elements for handling condition predicates and conformance statements (not defined in 2.7.1—NIST will propose for V2.9) • Support for min/max shortly • LRI IG MWBHL7 V2.5.1 XML + CP + CS + Length HL7 V2.7.1 LRI IG XML +

  13. Test Case Management Spreadsheet Processing – Priming the Validation Tool Use Case Generation Context File (XML) Message Generation Engine LRI Test Message Validation Context File (XML) Message Profile (XML) Validation Report Test Cases Message Validation Engine LIS Data Sheet (XML) LIS Test Plan and LRI Test Tool Test Data EHR Data Sheet (XML) EHR Test Plan and LRI Test Tool Regenerate Test Case Management Spreadsheet Juror Document (HTML Tables) Juror Document (XML) EHR Test Plan Validation Context File (XML) LRI Test Tool Test Case Filter LRI Test Message Tab Delimited Text File Message Model User Configuration Modifies Message Model Test Case Management Spreadsheet Processor

  14. A lab test is ordered for a patient • The specimen is collected, and is received and processed in the lab • The lab result is produced and stored in the LIS database • The lab result is transmitted to an ambulatory EHR • The lab result is viewed in the ambulatory EHR • For Discussion: • Does the LAB expect a local code for the order in a electronic order request? • Use Cases to consider: agreement/no agreement between LIS and EHR for local codes Use Case Test Case Manual entry of test data EHR transmits Hemoglobin blood test order to the LIS for John Doe along with pertinent demographic and order request data It is assumed Lab has capabilities for manual entry Local code No code—text LOINC code DB Lab Results Data Sheet John Doe DOB: 05/23/1959 Gender: M Result: 13.7 g/dL Range 13.2 – 16.2 Status: Final • Test tool inspects OBX.3 for a specific LOINC code • Inspect for 718-7 only or inspect for one of 718-7, 20509-6, 30313-1, 30352-9, 30351-1, and 30350-3? Test Data Sheet LIS Data Entry Message No LOINC given for results HL7 V2 Lab Results Message • Load • Cut/paste • Send • For Discussion: • No suggested method given (Should we?) • With no method given we should expect one from a set of valid LOINC codes for this order (Lab dependent?) • If a specific method is given (is this typical/possible?) then should we look for a specific LOINC code in the sent message? LIS Test Tool select test case Validation Report

  15. For Discussion: • What are the requirements on the EHR with regards to LOINC—given the LRI IG, CLIA, and meaningful use? • What are the EHR display (GUI) requirements for the received lab results? • What are the EHR storage requirements for the received lab results? • Given that the LOINC code is likely to be translated into the EHR internal representation, what lab test names need to be displayed in exact form and what lab test names can be displayed in an equivalence text (same concept)? • What qualifies as “equivalence—same concept”? How is that determined? Can it be? Expert inspector only? Should displaying of the LOINC text (only) be the requirement—i.e., no mapping? • What is the impact of sending only the LOINC code (i.e., no local code) on testing? Use Case LRI EHR Test Harness Test Case Test Data Juror Document 718-7 (LN) LRI Test Message 718-7 (LN) Lab Message EHR Communication Displayed as internal display text representation ACK 718-7 (LN) ACK Validation Report Validation Translated into internal representation

More Related