Cardinality Specification and Testing Recommendations

1. Cardinality Specification and Testing Recommendations LOI and LRI MU Certification November 13, 2013 V2.9

2. Wiki Comments Note: definition of vital includes/implies specific segments/elements as unlimited (0/1,*) and others as bounded (0/1,n) Specific comments: C1:1: For limits verify with clinical experts C2:17: Unlimited list OBR, OBX, NTE, NTE-3 C3:36: Notify both sending and receiving parties if supported occurrences exceeded C4:41: Issue regarding partial or complete rejection C5:52: Imperative? C6:58: Process to grow (e.g. 10%) C7:72: Ability to have local agreement that meets base standard but violates IG on cardinality C8:74: Requested examples of Patient Safety C9:75: Multiple levels of ACK C10:77: Option to stop processing on CE C11:78: Is there a “flavor” between “Partial” and “Hard Stop”? C12:79: Partial consumption of results – no this is a CLIA issue C13:80: Error response behavior of client site message replication C14:81: New codes to avoid confusion with usage; suggestion on specific limits and usage

3. Responses to Comments • C1:1: For limits verify with clinical experts • R1:1: Include as part of redefining * to n for limited occurrence items and testing for unlimited items • C2:17: Unlimited list OBR, OBX, NTE, NTE-3 • R2:17: yes, this is the recommended minimum list for unlimited occurrences – update tables • C3:36: Notify both sending and receiving parties if supported occurrences exceeded • R3:36: Yes, new “Notification” slide with recommendations • C4:41: Issue regarding partial or complete rejection • R4:41: Reviewed with CLIA – required to reject all under 493.1291 report all results and associated information in a secure, reliable, accurate manner – see new CLIA slide • C5:52: Imperative? • R5:52: No response required • C6:58: Process to grow (e.g. 10%) • R6:58: Include with R1:1 as part of testing limit evaluation • C7:72: Local agreement that meets base standard; increase upper limit for IG on cardinality • R7:72: Yes, new slide with approach: appropriate adjustments in behavior at upper limit

4. C8:74: Requested examples of Patient Safety • R8:74: Requested from LRW and CDC/FDA • C9:75: Multiple levels of ACK • R9:75: See discussion on new slide – deferred to discussion on Acknowledgements • C10:77: Option to stop processing on SE • R10:77: Option will be added to table • C11:78: Is there a “flavor” between “Partial” and “Hard Stop”? • R11:78: No see R4:41 • C12:79: Partial consumption of results – no, this is a CLIA issue • R12:79: See answer R4:41 • C13:80: Error response behavior of client site message replication • R13:80: Recommendations on new slide – deferred to discussion on Acknowledgements • C14:81: New codes to avoid confusion with usage; suggestion on specific limits and usage • R14:81: Agreed – will update tables

5. Issues related to upper limits • If standard says 0,10 • If guide says 0,5 then • Test can send 0-5 • Test that we can receive and consume 0-5 • Sending • Error if unable to send 0 • Error if unable to send up to 5 • Error if send more than 5 • Receiving • Error if unable to receive 0 • Error if unable to receive and consume less than 5 • Error if receive more then 5 • Can create new derived profile • May not decrease upper limit below IG • May increase upper limit up to base standard • Then may increase the error on upper limit for sending or receiving to be equivalent to the new upper limit

6. Overview

7. Cardinality Definition • Cardinality identifies the minimum and maximum number of occurrences that a message element must have in a conformant message • Some elements shall always be present (e.g., cardinality [1..1], [1..n]) • Other elements shall never be present (e.g., cardinality [0..0]) • Other elements may or may not be present—zero or more occurrences (e.g., cardinality [0..n]) • A conformant message must always contain at least the minimum number of occurrences, and shall not contain more than the maximum number of occurrences

8. How is Cardinality Tested? • Limited case - if cardinality specification is [1..5], per the standard a conformant message must • always contain at least one occurrence • and shall not contain more than five occurrences • Senders must be capable of • sending up to 5 instances of the element and • Receivers must be capable of • “processing” 5 elements. • Receiver behavior defined in associated functional requirements for the element. • Testing the sender • we have a test case where we provide data for 1 instance of the element and validate based on that. • In another test case, we provide data for 5 instances and we would validate to check that 5 instances were in the message. • Another test case we could provide data for 6 instances. Now the application interface may very well allow for this and this is OK (because it could support other use cases), what we would test for here is that only 5 are sent (present in the message). If 6 instances are sent then the application is not conformant. • If we don’t provide any data, the application should recognize that data is needed. • Testing the receiver • we would inspect that the number of elements we sent are correctly processed/associated/stored depending on the element.

9. How is Cardinality Tested? • Unlimited case • a cardinality of [1..*] indicates that implementations must be capable of supporting an unlimited number of instances for that element • Currently, an arbitrary number is selected for testing when the maximum upper limit is indicated with “*” • Testing proceeds exactly as indicated on the previous slide since an arbitrary upper limit is selected, i.e., [1..*]  [1..x] • In the case of “*”, implementers are not excused from supporting unlimitedoccurrences—there is just a practical limit that can be tested • In current MU testing, under-testing is typical because creation of relevant test cases usually is difficult and resource-consuming • NIST welcomes guidance for testing a “minimum upper limit” for unlimited cardinality elements.

10. Recommendations for LOI/LRI Implementation Guides • The specification of “*” for unlimited occurrences of an element should remain as is • That is, systems are required to support unlimited “*” occurrences of elements • This applies for both the Sender and Receiver • There is no need for the proposed [0..x, *] conformance construct; the “x” is described in the Testing Guidance document and is only guidance not a requirement (i.e., unlimited, when specified, remains the requirement)—nothing has effectively changed for implementers • What is new is that guidance for testing is being specified in a separate document and does not affect implementation requirements.

11. Definition of Transaction For the purpose of this analysis and recommendation, the following is the definition of a “transaction” • Limited to a single MSH and associated segments • Does not have an impact on previous or future transactions, e.g., failure to consume a corrected report or a report that completes preliminary or partials does not require removal of an earlier preliminary, partial or final report. • May include one or more orders (ORC/OBR) up to the total number of orders reported under the same MSH • Limited to a single patient • Limited to orders that were placed together on the “requisition” or single order transaction – On reporting, may include any reflex and add-on orders • Note: After reporting a final order (ORC/OBR) (all components are complete), the recommended method is to not report the order again unless there is a change (e.g. corrected result).

12. Conclusion • Failure due to exceeding cardinality limits in a certified EHR expected to be be a very rare occurrence (<<1:1,000,000) for a certified EHR • This will not affect patient care for routine/stat orders • Potential impact on patient safety outweighs any possible benefit from receiving incomplete report (due to EHR inability to consume) • Provider must be notified to call laboratory for results and the laboratory must be informed electronically of error. This will allow other methods of reporting to be pursued in a timely manner.

13. Lab IGs Behavioral Guide • Indicate in this guide the behaviors of a receiving system in error circumstances • Example 1: no element is sent where at least 1 is required • Example 2: Cardinality is [1..5] and the sender (non-conformant) sends more than 5 instances. Receiver is conformant and can only accept 5 instances • Example 3: Sender sends more than receiver can handle (receiver is non-conformance but may implement “practical upper –limit”, but in case of failure it is better to indicate this to the sender then process with missing information—that is, handle in another workflow, potentially manually. • Behavioral options for the receiver • Hard Error -- Reject transaction and ACK with Hard Error • Hard stop, i.e., for an LOI/LRI message, the transaction shall not be processed • Receiver is notified of the hard error (generic requirement defined in functional behaviors guide) • Sender is notified of the hard error via acknowledgement transaction • Soft Error -- Process message and ACK with Warning/Alert Error • Proceed with warning to sender and receiver of exceeding limits (as well as missing/incorrect information), but proceed with processing the lab order/result • Receiver is notified of the soft error (generic requirement defined in functional behaviors guide) • Sender is notified of the soft error via acknowledgement transaction • Note: May also treat as Hard Error • For both LOI and LRI, the appropriate behavioral option is determined based on an element by element basis

14. Processing Requirements – Segments(In Process)

15. Processing Requirements – Elements(In Process)

16. Testing Guidance Document • Will contain guidance for testing elements designated with unlimited “*” cardinality • For each element, a minimum upper limit of instances will be specified to indicate NIST testing for MU certification • The determination of the minimum upper limits is based on a review of the upper limit of practical clinical scenarios • It is important to note that the limits are recommendations and do not replace implementation requirements; vendors are still required to support an unlimited number of occurrences • NIST will follow the recommendations provided; however, at NIST’s discretion an arbitrary higher minimum upper limit could be tested for a few selective elements • Vendor’s EHR technology will be expected to demonstrate that the system supports up to this number of instances • NIST testing can exceed the suggested minimum upper limit of instances, requiring vendor’s EHR technology to demonstrate their system supports a higher number of instances (for example, for [1..*] where 5 is the suggested upper limit, NIST could test for 7) • Test Guide + Behavior Guide ≠ Compliance

17. Minimum Upper-Limit Test Guidance – Segments(In Process) Note: Upper Limits are intended as starting points for discussion only – final determination in Functional Behavior Guide

18. Minimum Upper-Limit Test Guidance – Elements(In Process) Note: Upper Limits are intended as starting points for discussion only – final determination in Functional Behavior Guide

19. CLIA issues with partial report consumption Discussion • 493.1291 requires that a laboratory must report test results and all relevant clinical information in an secure, accurate, and reliable manner to the authorized person • Relevant issues • The lab has chosen to report these orders/tests together.  • The EHR does not have enough information to decide to provide one order if another fails.  • Since this should be a very rare occurrence for a certified EHR and the most common failure will be on a very complex patient, the risk of reporting some but not all of a transaction far outweighs the reward (remember, the provider can call the lab or the lab can send the results via an alternative method).  • Example – complex surgical path report were the failure is in the interpretation of the results (report may be misleading without the interpretation and become the basis for the provider taking an action that is not supported by the full report).  • While one may assume that there should be a parent child relationship, that is not always the caseand therefore cannot be the basis of allowing a partial consumption of a transaction

20. Text in Red deferred to acknowledgement discussion Notification on Error • Receiver (Technical Response) • Must notify sender (Application Level Acknowledgement) • Receiver (Functional Behavior) • Must notify recipient of error • To the extent possible • Associate with Patient/Order • Associate with Intended Recipient • Must be visible to clinical staff, not just technical support • Receiver (Technical Response) • Must be able to receive and consume error response (Application Level Acknowledgement) • Sender (Functional Response) • Must notify appropriate personnel to • Resolve technical problem with receiver • Ensure report / order is handled by exception methods

21. Local Agreement Constraints • Conformant with the IG and certification • Local agreements – must be implemented as documented profiles between trading partners and included in MSH-21 • Cardinality • MAY increase the requirements on both the lower and upper bounds, e.g., • [0,5] to [1,5] • [1,5] to [1,10] if base is [1..>10] • SHALL not exceed the upper limit on the base standard; • SHALL not decrease the upper or lower bound below that of the implementation guide (e.g., [2-5] cannot be constrained to [0-3]). • Conformant behaviors • Any associated conformance or Functional Behaviors will also increase along with the increase in limits (e.g. error notification)

22. Multiple Acknowledgements– defer to Acknowledgement Discussion • Acknowledgements • System • As described in base standard and IG • Application • Multiple levels based on MSH-9(?) and MSA • Sending application must be able to receive multiple acknowledgements: • System is synchronous • Application level messages are asynchronous • Requested Application Level – may be all messages • Error for additional checks is error only

23. Client Side Message Replication– defer to Acknowledgement Discussion • Assumes • Message is split by client using interface engine, middleware, et. • Lab is unaware of, or not responsible for, additional recipients • Replication engine behavior • Replicant copies must have MSH-3 updated to correspond to the replication engine and not the laboratory • If error is received from the replicated recipient and echoed copy of MSH-3 (where should this be placed in the ACK?) is that of the replication engine, it should be intercepted and not reported to the LIS • Message to LIS intended recipient shall not be modified and on application error, the entire message will be passed to the LIS • Certified EHR behavior • Consume message • Error reporting is the same for all messages (replicated and original) • See notification on error • Must include MSH-3 in ACK (where)

24. HL7 V2 Conformance Chapter • Update Chapter 2B to better define cardinality and the requirements it places on implementers • Update as part of V2.8.1 or V2.8.2? • Describe in a table the implementation and operational requirements for cardinality for both sender and receiver (similar to the table created for usage in V2.8)—see next slide • Provide a conformance assessment table for cardinality and appropriate (options) behavior of the receiver in error circumstances