IHE-PCD , HL7 HC Dev WG,
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Medical Device Test Effort NIST Team Members PowerPoint PPT Presentation


  • 47 Views
  • Uploaded on
  • Presentation posted in: General

IHE-PCD , HL7 HC Dev WG, ISO/IEEE 11073, and NIST Medical Device Communication and IHE-PCD Cycle 4 Test Strategy IHE-PCD, HL7, ISO/IEEE Joint WG Meetings (@ Atlanta, GA) 23 September 2009. Medical Device Test Effort NIST Team Members. John Garguilo ( [email protected] ,

Download Presentation

Medical Device Test Effort NIST Team Members

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Medical device test effort nist team members

IHE-PCD , HL7 HC Dev WG,ISO/IEEE 11073, andNISTMedical Device Communication andIHE-PCD Cycle 4 Test StrategyIHE-PCD, HL7, ISO/IEEE Joint WG Meetings (@ Atlanta, GA)23 September 2009


Medical device test effort nist team members

Medical Device Test EffortNIST Team Members

  • John Garguilo ([email protected],

    301-975-5248)

  • Sandra Martinez ([email protected],

    301-975-3579)

  • Maria Cherkaoui ([email protected]

    Guest Researcher)

  • *Jing Gao([email protected] Guest Researcher)

    * Jing joined our project in May ’09

    Project Web site: www.nist.gov/medicaldevices


Nist mdc testing topics

NIST MDC Testing Topics

  • IHE-PCD Test Strategy

    • NIST Framework for Testing Distributed Healthcare Applications

    • IHE-PCD Testing Framework Instance

    • Cycle 4 Pre-connectathon (and virtual-connectathon)

    • Test Plan

    • Support both Conformance & Interoperability

  • RTM Terminology Management System Update

  • NIST ISO/IEEE 11073 Tooling

    • ICSGenerator, ValidatePDU

  • NIST ISO/IEEE 11073 Standards Development Work

    • MDC X73 XML Schema (X73-P10202)


Test environments

Test Environments

  • Instance Testing

    • Conformance (e.g., against HL7 2.x or CDA)

      • Implementation conforms to Spec. on which it is based

      • IHE Model: ~Virtual and Pre-Connectathon

  • Isolated System Testing

    • Includes Instance Testing Activities

    • Protocol Conformance

    • Functional Behavior Conformance

      • Features and Operational behavior correspond to Specs.

      • IHE Model: ~Virtual and Pre-Connectathon

  • Peer-to-Peer System Testing

    • Includes Isolated System Testing Activities

    • Interoperability Testing

      • Testing complete application environment

      • May include interacting w/ Database, using Network Communications, or interacting w/ other hardware, apps, or systems if appropriate

      • IHE Model: ~Connectathon


Test environment message validation nist v2 testing tools ihe pcd

Test Environment Message ValidationNIST V2 Testing Tools: IHE-PCD

Validation of IHE-PCD message(s) and corresponding HL7 Profile(s)

Syntax and Semantic Content Validation

Against HL7 V2 message (e.g., PCD-01)

Message structure (e.g., MSH,PID,PV1,OBR,NTE,{{OBX},OBX,OBX,OBX,…})

Against HL7 profile

(Msg_type^Event_type^ e.g., ORU^R01^…)

Against HL7 and/or user provided tables

Example of user provided table is RTM for Ref_IDs, Units, etc.

Against ‘validation context’, including specific values

Defined in XML (e.g., specific test case values)


Medical device test effort nist team members

NIST V2 Testing Tools and Services

Testing Validation Types

Validation against ‘failure types’:

VERSION*: The version in the message and in the profile should match.

MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the profile and in the message should match.

MESSAGE_STRUCTURE: The message should have a valid message structure (correct usage, correct cardinality, and correct element name).

USAGE: R elements should be present; X elements should not be present in the message.

CARDINALITY: Elements should be present at least the minimum times and at most the maximum times specified in the profile. It should also take into account the usage of the element (X element with a minimum of 4 should not be present in the message).

LENGTH: The value of the element should have a length equal or less than the value specified in the profile.

DATATYPE: For the datatype NM, DT, DTM, SI and TM, the value of the element should match the regular expression defined in the standard.

DATA: The value of the element should match a constant specified in the profile, a value set specified in a table, a value or a regular expression specified in the message validation context.

MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the location specified in the message validation context can't be found in the message.

TABLE_NOT_FOUND*: This is a user input when a table can't be found in the table files (TableProfileDocument).

AMBIGUOUS_PROFILE*: The profile should not be ambiguous.


Instance testing hl7 v2 message validation

Instance Testing HL7 V2Message Validation

User

Test Management

Services

Web

Application

Client

HL7 V2 Message

Validation Criteria

HL7 V2

Message

Validation

Report

HL7 V2

Message

Results

HL7 V2

Message

Validation

Report

  • Test Artifacts

  • Conformance Profiles

  • HL7 Tables

  • Validation Context Files

  • Generation Context Files

Test

Harness

(Java Code)

Test Execution

Conformance Testing: Showing that messages are in

compliance w/ HL7 V2.x and constraints specified in

IHE-PCD framework documents.


Instance system testing ihe pcd rtm validation

Instance System Testing IHE-PCD RTM Validation

User

Test Management

Services

HL7 V2 Message

RTM

Validation Criteria

Web

Application

Client

Report

HL7 V2

Message

Validation

Results

RTM

Message

Validation

Report

HL7 V2

Message

IHE PCD

DOC

Test Agent

RTM

Message

Validation

Test

Harness

(Java Code)

Test Execution

  • Test Artifacts

  • Conformance Profiles

  • HL7 Tables

  • Validation Context Files

  • Generation Context Files

Conformance Testing: Showing that messages are in

compliance w/ Harmonized-Rosetta terminology

Time

Security


Medical device test effort nist team members

Future: Isolated System Testing

IHE-PCD Application Functional Behavior Testing

Vendor

IHE-PCD

DOC

System

Test Management

Services

IHE-PCD DEC

DOC

Test Scenario

Web

Application

Client

HL7 V2

Message

Generation

HL7 V2

Message

Validation

Results

Message

Validation

Reports

IHE-PCD

DOR

Test Agent

IHE-PCD

DOC

Test Agent

Test

Harness

(Java Code)

Test Execution

Report

RTM

Message

Validation

  • Test Artifacts

  • Conformance Profiles

  • HL7 Tables

  • Validation Context Files

  • Generation Context Files

Conformance Testing: May include HL7 message syntax validation, RTM, and/or application functional behavior testing

Time

Security

Router


Future isolated system testing ihe pcd rtm validation

Future: Isolated System Testing IHE-PCD RTM Validation

User

IHE-PCD

DOR

System

Test Management

Services

HL7 V2

Message

HL7 V2 Message

RTM

Validation Criteria

Web

Application

Client

Report

HL7 V2

Message

Validation

Results

RTM

Message

Validation

Report

IHE PCD

DOC

Test Agent

RTM

Message

Validation

Test

Harness

(Java Code)

HL7 V2

Message

Test Execution

  • Test Artifacts

  • Conformance Profiles

  • HL7 Tables

  • Validation Context Files

  • Generation Context Files

Conformance Testing: Showing that messages are in

compliance w/ Harmonized-Rosetta terminology

Time

Security


Medical device test effort nist team members

Future: Peer-to-Peer System Testing

IHE-PCD Application Functional Behavior Testing

Vendor

IHE-PCD

DOC

System

IHE-PCD

DOC

System

Test Management

Services

IHE-PCD DEC

DOC

Test Scenario

Web

Application

Client

HL7 V2

Message

Generation

HL7 V2

Message

Validation

Results

Message

Validation

Reports

IHE-PCD

DOR

Test Agent

IHE-PCD

DOC

Test Agent

Test

Harness

(Java Code)

Test Execution

Report

RTM

Message

Validation

Conformance Testing: May include HL7 message syntax validation, RTM, and/or application functional behavior testingInteroperability Testing: system(s) (functional) behavior

  • Test Artifacts

  • Conformance Profiles

  • HL7 Tables

  • Validation Context Files

  • Generation Context Files

Time

Security

Router/Logger/Proxy


Hl7 message validation study example piv pcd 03

HL7 Message Validation StudyExample PIV (PCD-03)


Piv pcd 03 test case 60101 validation using the ihe tf supplements

PIV-PCD-03 Test Case 60101Validation using the IHE TF/Supplements

  • MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001

  • PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L

  • ORC|RE|12345|||||||||||||||||N60101

  • RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC

  • RXR|IV||IVP

  • OBX|1||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun


Piv pcd 03 test case 60101 validation using hl7 profile tables

PIV-PCD-03 Test Case 60101Validation Using HL7 Profile Tables

  • MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001

  • PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L

  • ORC|RE|12345|||||||||||||||||N60101

  • RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC

  • RXR|IV||IVP

  • OBX|1||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun


Piv pcd 03 test case 60101 hrtm validation

PIV-PCD-03 Test Case 60101hRTM Validation

  • MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001

  • PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L

  • ORC|RE|12345|||||||||||||||||N60101

  • RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC

  • RXR|IV||IVP

  • OBX|1||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun


Piv pcd 03 test case 60101 use test case specific validation

PIV-PCD-03 Test Case 60101Use/Test Case Specific Validation

  • MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001

  • PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L

  • ORC|RE|12345|||||||||||||||||N60101

  • RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC

  • RXR|IV||IVP

  • OBX|1||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun


Piv pcd 03 test case 60101 overall validation

PIV-PCD-03 Test Case 60101Overall Validation

  • MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001

  • PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L

  • ORC|RE|12345|||||||||||||||||N60101

  • RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC

  • RXR|IV||IVP

  • OBX|1||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun


Piv pcd 03 test case 60101 validation coverage

PIV-PCD-03 Test Case 60101Validation Coverage

  • Coverage % = Number of Values in Message / Number of Values Checked

  • IHE Supplement Validation43%

  • Validation using Profile Tables24%

  • Test Case Specific Validation19%

  • Overall Validation74%

  • Overall Validation without profile62%

  • Note: remaining 24% (from overall validation percentage) is validated at the message structure level via HL7; these include message segments that are not of primary interest to the IHE-PCD. If this level of validation is considered, then nearly 100% validation coverage can be expected.


Message validation context tf supplement validation

Message Validation ContextTF/Supplement Validation

  • DEC-PCD-01-ValidationContext.xml

  • SPD-PCD-02-ValidationContext.xml

  • PIV-PCD-03-ValidationContext.xml

  • ACM-PCD-04-ValidationContext.xml

  • IDCO-PCD-09-ValidationContext.xml


Message validation context hrtm validation

Message Validation ContexthRTM Validation

  • DEC-PCD-01-ValidationContext.xml

  • PIV-PCD-03-ValidationContext.xml

  • ACM-PCD-04-ValidationContext.xml

  • Needed: IDCO-PCD-09ValidationContext.xml


Message validation context test case specific validation

Message Validation ContextTest Case Specific Validation

  • PIV(for PCD-03)

    • 60101-ValidationContext.xml

    • 60102-ValidationContext.xml

    • 60103-ValidationContext.xml

    • 60104-ValidationContext.xml

    • 60105-ValidationContext.xml

    • 60106-ValidationContext.xml

    • 60107-ValidationContext.xml

    • 60108-ValidationContext.xml

    • 60109-1-ValidationContext.xml

    • 60109-2-ValidationContext.xml

    • 60109-3-ValidationContext.xml

    • 60109-4-ValidationContext.xml

    • 60110-ValidationContext.xml


Ihe pcd 09 10 pre connectathon

IHE-PCD ’09-’10 Pre-Connectathon

NIST Support (see recommendation document: "NIST Support for IHE-2009v1.docx")

  • Primary Goal:Provide tools to test conformance to HL7 and IHE-PCD Integration Profiles such as DEC, DEC-SPD, ACM, PIV, IDCO, and RTM

  • Instance Testing

    • System Under Test (SUT)  Test Agent (TA)

    • HL7 syntax validation (basic validation)

    • IHE-PCD profile syntax

    • IHE-PCD profile semantic validation

    • RTM validation

    • Test case specific validation

  • Develop Test Plan across IHE-PCD Integration Profiles

    • Test Scenarios

    • Interaction diagrams

    • Pre-conditions

    • Validation criteria – all test-able requirements (assertions)

    • Test cases

    • Validation context file(s)


Ihe pcd 09 10 pre connectathon continued

IHE-PCD ’09-’10 Pre-ConnectathonContinued

Proposed conformance testing for ’09-’10 Pre-Connectathon

  • All DOR messages (PCD-01) sent during DOR testing.

  • All IOP messages (PCD-03) sent during IOP testing.

  • All IDCO-DOR messages (PCD-09) sent during IDCO testing.

  • All DOF messages (PCD-01) sent during DOF testing.

  • All DOC messages (PCD-02) sent during DOF testing.

  • All AR messages (PCD-04) sent during AR testing.


Show nist v2 tools via web

Show NIST V2 Tools (via Web)

Demonstration

NIST HL7 Web

  • Upload HL7 IHE-PCD ORU^RO1 profile

  • Upload HL7 IHE-PCD context validation file (XML)

  • Upload HL7 IHE-PCD message (ER7 format)

  • Validate

  • Generate and review resulting ‘Report’

    • Review errors


Medical device test effort nist team members

RTM Terminology Management System Update


Rtm mgmt system changes from last wg mtgs in may

RTM Mgmt System Changes(from last WG mtgs in May)

Features Completed

  • Enhanced lookup of REFIDs (description of REFID’s) and provided an interface to propose new terms

  • Implemented an interface for browsing terms in X73 Nomenclature


Rtmms in progress and next steps

RTMMS In progress and next steps…

Features based on the Rosetta Supplement

  • Incorporate “Enumerations” management capability(ies)

  • Add ranking capabilities of implementation priority of valid terms in the Rosetta table

  • Implement Rosetta validation against hRTM

  • Add ability to edit hRTM table and save changes

  • Implement hRTM version control and management system 


Rtmms future new enhancements

RTMMSFuture New Enhancements

Features based on the new/recent enhancements to HRTM

  • Assign dimensions to the units

  • Implement Rosetta rules

  • Implement generation of hRTM based on Rosetta rules

  • Generate comment table based on the Recommended Rosetta Comment Submittal Format

  • Implement RCH table

    User oriented features

  • Regular Expression based column filtering

  • Implement “change trailing” capabilities

    • To identify occurred changes, time they were made, users who made them…

  • Enhance registration process

  • Incorporate enhanced X73 Nomenclature database 


Medical device test effort nist team members

NIST ISO/IEEE 11073 Tooling

Update


X73 nist tooling

X73 NIST Tooling…

  • ICSGenerator

    • User Interface Work

      • Met w/ Todd Cooper, captured issues, made changes

      • Updated issues list – used as basis for task list

    • Integrating database into ICSGenerator…

      • Lite-weight database identified

      • To include hRTM…

    • Initial discussions

      • X73-compliant Device Containment

      • Generation of partial OBX segments

  • On horizon: MDC Semantic Database

    • Terminology

    • Nomenclature

    • Medical Device Profiles and other Test Artifacts


Medical device test effort nist team members

ISO/IEEE 11073 Standards Work

Update


Iso ieee 11073 standards work

ISO/IEEE 11073 Standards Work

  • MDC X73 XML Schema (X73-P10202)

  • Verification and validation of XML Schema continues via tool usage

  • Support for Personal Health Device Working Group

    • X73-20601

    • 104xx device specializations

      • Continue to validate via tooling prior to ballot

      • Report errors for correction prior to balloting


Thank you

Thank YOU!

Discussion

Propose “ORU^R01^ORU^Rxx” naming convention for IHE-PCD HL7 profiles

xx denotes profile specific for Integration profile/device class (use of DEC/PCD-01)

Questions?

Thanks for your attention 

Please visit us at:

http://www.nist.gov/medicaldevices


Extra slides

Extra Slides


Nist hit testing infrastructure a framework for building test systems

User

Facilitator

Monitor

IHE-PCD

DOC

System

NIST HIT Testing InfrastructureA Framework for Building Test Systems

Test Management

Services

Test

Description

Validation

Generation

Results

Test Data

Test

Agent

Test

Harness

Test Execution

Evaluation

Agent

Aggregated

Service

Test

Artifacts

Time

Report

Security

Other

Services

Specialized

Services

Router/Logger/Proxy


  • Login