Test EHR Pilot Program Launch

1. Test EHR Pilot Program Launch Avinash Shanbhag Director of Implementation and Testing Division Office of National Coordinator Oct 21, 2013

2. Topics • Pilot Tasks for SITE team • Pilot Contacts and timeline • Summary and Next Steps

3. Pilot Task – For SITE Team Provide feedback on the draft guidance issued by ONC and CMS for designating Test EHR Provide guidance to ONC on any operational issues Test that the Test EHRs are able to meet the requirements of the program Validate the registration process for providers in the “EHR Randomizer Tool” Raise any issues that would impact ability to become a designated “Test EHR”

4. Topics • Pilot Tasks for SITE team • Pilot Contacts and timeline • Summary and Next Steps

5. Pilot Participants • AthenaHealth • POC: Stephen Jacobs • Email: sjacobs@athenahealth.com • McKesson • POC: Sally Connally • email: Sally.Connally@McKesson.com • Meditech • POC: John Valutkevich • email:jmvalutkevich@meditech.com

6. Pilot Support Staff • Pilot Coordinator • POC: Becky Angeles • Email: rebecca.angeles@esacinc.com • EHR Randomizer Tool Lead • POC: Kevin Brady • email: Kevin.Brady@nist.gov • Pilot Lead • POC: Avinash Shanbhag • Email: Avinash.Shanbhag@hhs.gov

7. Pilot Process – Overview New Dates – NOT YET FINALIZED Pilot (Sept 20 – Dec 15) Pilot Participation Finalized Complete Registration of Test EHR ONC Debrief to Pilots Pilot Kick off Pilots Submit Final Feedback to ONC Dec Nov Sep Oct Proposed Milestones • Pilot kickoff: Sept 20th • Complete Registration: Oct 30th, Sept. 30th • Submit feedback to ONC: Nov 22nd, Oct 20th • ONC schedules debrief session with Pilots: Dec 15th, Oct 30th

8. Topics • Pilot Tasks for SITE team • Pilot Contacts and timeline • Summary and Next Steps

9. Testing Process After the kickoff, Pilots will receive email from EHR Randomizer Tool team with steps to create new account and register the Test EHR ONC Implementation and testing team will register as “provider” on the Randomizer tool Send Direct Message (with test CCDA) to each of the pilot Test EHR SITE team will validate receipt of response matches the process described in the registration Submit report (pass/fail) for each Test EHR and feedback on tool usage to ONC/NIST Pilots can submit individual feedback and/or joint report. Pilot Coordinator can assist in logistics

10. Summary and Next Steps Q1: Are there additional requirement for documentation from Test EHR for successful exchange? Q2: Should there be requirements for Test EHR to be member of scalable trust community? Q3: Do you suggest any updates to the Developer Participant and Test Instructions requirements? Q4: Are there any changes that need to be made towards Test EHR registration using the EHR Randomizer Tool?

11. Thanks! • Questions? • Pilot Coordinator • POC: Becky Angeles • Email: rebecca.angeles@esacinc.com

12. Pilot Testing Notes and Feedback from SITE Team • EHR Randomizer Tool URL used by the SITE Team: https://ehr-randomizer.nist.gov:14081/ehr-randomizer-app/#/home • EHR Randomizer Tool Website Feedback: • Not Intuitive to click (because no hand or other icons that are typically used in web pages) and providers may not understand if they should/should not click on something on the page. • Goal Tab: • Not enough information to describe what the goals are clearly. • Architectural Overview Tab • It might be good to combine this with the goal tab and describe the goals more clearly. • Suggest to have Goal Tab to be first and the Provider Workflow tab to be something that a provider can access by clicking the tab. • Provider Workflow: • Steps need to be better described because providers may not know the language and the terms used on the page. Also having the Goals and Overview on the home page may help to educate providers. • Step showing downloading a public certificate which is never done in Direct. Public Certificates are always discovered.

13. Provider Registration Process I think adding a tool tip or something on the purpose of the email, username would help the provider.

14. Provider Registration Process Cont’d 1. No Feedback is provided after accepting the agreement and submit account creation information, don’t know if the account creation succeeded/failed ? 2. It would help to describe some confirmation and say something about What is the next step ? 3. After some time (Within a minute) the Create Account page comes back with blank fields even though the account is created, misleading the provider to potentially recreate the account. 4. Suggest providing feedback upon account creation and ask the user to login and proceed to the next phase. 5. There should be some kind of a work flow or something to tell the user where he is in the process, Use the Provider Workflow and remind the provider what he/she have completed and what remains to be done. 6. I checked my email before I did anything just out of curiosity and I received the following: Dear nbashyam Your account has been successfully created. Your username is: nbashyam Thank you very much The ehr-randomizer Team ----- The above information is not very useful, we should craft some message to tell the provider what to do. Also the Email is from Fred instead of a preferred NIST Organization inbox

15. Provider Registration Process Cont’d 7. I was on the Create Account Page when I logged in and even after I logged in, the screen displays CreateAccount - Why would we create an account once the user is logged in already ? 8. Home Page shows the Provider Workflow - Don’t know what to do or where to enter the information that I need to enter, additional guidance might be useful for providers.

16. Account Management Issues • Logged out of the NIST EHR Website • Tried to login again and was unsuccessful, don’t know why ? (Browsers used originally was Firefox, Browsers used to login second time was Google Chrome and Firefox) • Fixed by NIST, (Kevin indicated it had something to do with Port Forwarding)

17. Other Web Pages: SELECTED My CEHRTs 1. Timezone dropdown should be really simplified (Africa and other places are listed which I don’t think are relevant for MU) --- Should provide information on where to find CHPL ID and how to obtain Id. 2. Developer Dropdown has a list of companies which is very small, Don’t know what this means Because I am in the provider workflow and it does not seem to make sense to me, the name or label should be changed to CEHRT’s maybe using CHPL 3. The label NEW on the left is not intuitive and it seems out of place.. (put some text around it and guide the provider) 4.Request a Match… --- When I request a match, the screen disappears and I don’t know what happens, after sometime the page comes back with a match. -- It would be better to show some kind of progress indicator.

18. Request Match with No CEHRTs in MYCEHRTs Bad Text Shown to provider

19. McKesson Test EHR Pilot • 1. Availability times should include timezones • 2. Add text in the trust anchor section to indicate that it should be downloaded And installed. • 3. McKesson has a .7z.SAFE file extension for the trust anchor which for non-technical people Will be challenging to unzip • 4. There are four certificates that need to be used, how do we know which one to use • Contacted the vendor. • Maybe we should have only one for providers to download and install. • 5. Sent SITE Trust Anchor without zip file that was blocked, so maybe guidance should be provided to vendors to provide their anchor in a zip format so that providers don’t have to decipher these issues. • 6. Discovered that McKesson Trust Anchor was incorrect and notified pilot participants to get it corrected. • - Once it was corrected the four trust anchors were removed and replaced by a single trust anchor which is preferable. • - The replaced trust anchor did not conform to Direct Applicability Statement so it was asked to be reworked by Mckessson. • 7. *** We should test every TEST EHR using an environment like SITE to address these issues so that providers don’t get annoyed during attestation. • 8. After replacing the Trust Anchor once again, I was successfully able to test the Provider Workflow, however the documentation for McKesson does not indicate how the feedback will be provided, so I assumed MDN, but it should be clearly specified.

20. Athena Health Test EHR Pilot • Test Anchor Format was “PEM” which was not showing useful information when you open it, so converted the PEM certificate to DER. • Sent a message and they had a bug in their system related to MDN generation, that they are fixing. They also required only attachments but no text in the email because they were not processing text in their email. • This should be added as a specific note or something in the NIST SITE to communicate with the providers. • Athena’s HISP has been setup to process only MIME content type of XML which is part of a single part message only. • This causes an issue because most implementations create multi-part MIME messages. • The above issue may cause interoperability issues if the sender is sending multi-part MIME bodies but the receiver is only capable of processing single part MIME messages. • We called off further testing on the pilot because there was no easy way to create a message with one part which is of MIME type XML. (Failed Test) • This was surprising given that Athena was already certified.

21. Meditech Test EHR Pilot • Imported the Trust Anchor without issues and sent out the SITE Trust Anchor successfully. • Meditech imported the anchor and we ran the test however there was something wrong in the Meditech configuration which caused an error in their processing and SITE did not receive an MDN. • This was corrected by Meditech later and the processing was completed successfully and mails were exchanged without any issues.

22. Other General Feedback • Before an EHR gets added to the pool of EHR’s for Cross Vendor testing it is better to perform some basic interoperability testing similar to pilot to flush out any DNS / certificate issues. • This is similar to an on-boarding process to prevent providers don’t discover issues when they try to perform the test in the real-world and become skeptics of technology. • MDN’s may or may not be enabled by HISP’s or Direct Systems to be published to the provider, so using that as a way to acknowledge/confirm receipt of a message might not be the most appropriate. • A better way would be to use some kind of email different from the Direct systems to send/receive confirmation similar to McKesson. • Creating a trust bundle for the EHR systems participating in the testing will be a worth-while exercise because it reduces the burden on the EHR vendors willing to participate in the cross vendor testing activities.