1 / 25

Agenda

NVMe- oF ™ Interop 2019 NVMe™ Annual Members Meeting and Developer Day March 19, 2019 Prepared by David Woolf, Senior Engineer, University of New Hampshire InterOperability Lab, UNH-IOL. Agenda. Current State NVMe™ / NVMe- oF ™ Protocol Testing for the Integrators List

tracyl
Download Presentation

Agenda

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NVMe-oF™ Interop2019 NVMe™ Annual Members Meeting and Developer DayMarch 19, 2019 Prepared by David Woolf, Senior Engineer, University of New Hampshire InterOperability Lab, UNH-IOL

  2. Agenda • Current State NVMe™ / NVMe-oF™ Protocol Testing for the Integrators List • Interop for NVMe-oF Transports, Beyond the Integrators List • FC • RoCE • TCP

  3. Test Docs posted to UNH-IOL Site Test Specifications NVMe™ Interop v11.0 NVMe Hosts and SSDs NVMe-MI™ Conformance v11.0 NVMe Conformance v11.0 NVMe-oF Hosts and NVM Subsystems NVMe-oF™Interop v11.0 NVMe-oF Conformance v11.0 RoCE TCP NVMe-oF Binding X Interop (Future) FC NVMe-oF Binding X Conformance (Future)

  4. NVMe Hosts and SSDs Test Tools posted to UNH-IOL Site Test Tools OTS/Free SW NVMe™ Interop v11.0 Quarch (Hotplug) NVMe-MI™ Conformance v11.0 Teledyne LeCroy UNH-IOL INTERACT SW NVMe Conformance v11.0 NVMe-oF Hosts and NVM Subsystems NVMe-oF™Interop v11.0 OTS/Free SW NVMe-oF Conformance v11.0 NVMe-oF Binding X Interop (Future) UNH-IOL INTERACT SW NVMe-oF Binding X Conformance (Future)

  5. NVMe™ SSD Protocol Testing: Mandatory vs. FYI 248 Tests • NVMe protocol tests that apply to all NVMe SSDs, Mandatory vs. FYI • After June 2019 plugfest, mature FYI tests will be made mandatory, and new FYI tests will be introduced to test new NVMe specification features. 80 – FYI, IP 168 - Mandatory

  6. NVMe™ Protocol Testing: PCIe vs. All Transport 248 Tests • NVMe protocol tests that apply to all NVMe SSDs vs NVMe-oF NVM Subsystems 97 Tests apply only to PCIe SSDs 151 apply also to NVMe-oF NVM Subsystems

  7. NVMe™ Protocol Testing for Fabrics: M vs FYI 151 Test Cases apply to NVMe-oF NVM Subsystems • NVMe protocol tests that apply to NVMe-oF NVM Subsystems products Mandatory vs. FYI • After June 2019 plugfest, mature FYI tests will be made mandatory, and new FYI tests will be introduced to test new NVMe specification features. 56 Mandatory 95 FYI

  8. NVMe-oF™ IL Protocol Testing for Fabrics Fall 2018 9 Test Cases were added for NVMe-oF protocol in TP 2007 which are transport agnostic, will be implemented at June 2019 plugfest Future: Add compliance tests for the binding specifications i.e.: Fatal vs. Non-Fatal errors in NVMe/TCP After June 2019 plugfest, mature NVMe-oF FYI tests will be made mandatory, and new FYI tests will be introduced to test new NVMe specification features. • Primary Test Areas (all FYI currently) • Connect to NVM Subsystem • Property Get Command • Identify Command • Discovery Controller • NVMe-oF Test Suite Document

  9. Interop for NVMe-oF™ Transports • Tests deliberately do not test the transport, but use the transport to test NVMe protocols. • Example: Today, NVMe SSDs connect over PCIe. PCI-SIG has a well regarded and robust test program focused on PCIe phy and protocol. • What is the analog for NVMe-oF Transports? It depends…

  10. Interop for NVMe-oF™ Transports • NVMe-oF Plugfest draw many vendors and products across all transports. • Next NVMe-oF event: June 17-21, 2019 • RoCE and TCP have been especially popular • Take advantage of everyone “being in the same room” and run informal ‘Large Build’ test that goes beyond Integrators List requirements. • Days 1-2 focus on formal NVMe IL Tests • Days 3-4 focus on informal Large Build Interop

  11. Interop for NVMe-oF™ Transports • Each transport needs something slightly different • Enable community to find interop issues outside of the scope of NVMe™ protocol, that are critical to deploying NVMe. • Keep separate from NVMe IL testing, but leverage plugfests.

  12. NVMe™/FC - NVMe/RoCE- NVMe/TCP

  13. Interop for NVMe™/FC • Current Work • FC Large Build • Recurring feature of FCIA plugfests for 20+ years • Relatively small # of vendors • FCIA well organized to address this • Future • Checking higher link speeds, check new error recovery features

  14. Tested FC Protocols in parallel (critical use case) FC-NVMe Init. 1 FC-NVMe Init. 2 FC-NVMe Init. 3 FCP Initiator 1 FCP Initiator 2 FC Switch 1 Monitor FC Switch 2 FCP Target 1 FCP Target 2 FC-NVMe™ Target 1 FC-NVMe Target 2

  15. Checked Error Recovery BB Cred FC-NVMe Init. 1 FC-NVMe Init. 2 FC-NVMe Init. 3 BB Cred BB Cred FC Switch 1 Jammer FC Switch 2 Error Recovery needs fed back into FC-NVMe™ spec FC-NVMe Target 1 FC-NVMe Target 2

  16. Interop for NVMe™/RoCE • Current Work: • Verified connectivity/discovery with X targets and Y initiators • Verified IO, Soak test • Injected congestion • Checked Differentiated Services Code Point (DSCP), value match north/south, effect on traffic, Checked Priority Code Point (PCP) values and effect on traffic (802.1 Interop parameters) • Monitor for Pause on congestion • Future: • Many other opportunities around DCB and QoS on congestion, varying packet sizes and pipes, higher link speeds

  17. Applied Congestion Congestion Injector RNIC 1 RNIC 2 RNIC 3 RNIC n Switch 1 Monitor Switch 2 NVMe-oF NVM Subsystem RoCE Target n NVMe-oF™ NVM Subsystem RoCE Target 1 NVMe-oF NVM Subsystem RoCE Target 1

  18. Checked Recovery From Broken Links RNIC 1 RNIC 2 RNIC 3 RNIC n Switch 1 Monitor Switch 2 NVMe-oF NVM Subsystem RoCE Target n NVMe-oF™ NVM Subsystem RoCE Target 1 NVMe-oF NVM Subsystem RoCE Target 2

  19. Interop for NVMe™/TCP • Ubiquity creates many interop vectors • Initial Work: • Check different architectures • 1 to Many • Many to 1 • Causing Retransmission Events • Future Work: • Check discovery with TLS

  20. Cause Retransmit by Jamming ACK ACK NIC 1 NIC 2 NIC 3 NIC n ACK ACK Switch 1 Jammer/Monitor Switch 2 NVMe-oF NVM Subsystem TCP Target n NVMe-oF™ NVM Subsystem TCP Target 1 NVMe-oF NVM Subsystem TCP Target 2

  21. Distributed Reads NIC 1 Switch 1 Jammer/Monitor Switch 2 NVMe-oF NVM Subsystem TCP Target n NVMe-oF™ NVM Subsystem TCP Target 1 NVMe-oF NVM Subsystem TCP Target 2

  22. Shared Storage Pool NIC 1 NIC 2 NIC 3 NIC n Switch 1 Monitor Switch 2 NVMe-oF™ NVM Subsystem TCP Target 1 NS n NS 1 NS 2 NS 3

  23. Next Steps • Plugfest Webinar covering NVMe™ and NVMe-oF™ topics scheduled: • Webinar Registration is open now, May 7, 2019 • https://www.iol.unh.edu/event/2019/05/unh-iol-webinar-nvme-plugfest-11 • NVMe-oF Plugfest Registration is open now June 17-21, 2019 • https://www.iol.unh.edu/event/2019/06/nvme-plugfest-11 • NVMe Plugfest Registration is open now June 24-27, 2019 • https://www.iol.unh.edu/event/2019/06/nvme-pcie-plugfest-11 • After webinar, planning calls for interop tests will begin • All plugfest participants invited to attend

  24. Summary • NVMe™/NVMe-oF™ Integrators List testing is well defined and supported • Ratified test docs available for NVMe, NVMe-oF • Will add testing for binding specs in the future • Test tools already support PCIe, FC, RoCE, TCP transports • Opportunity to leverage interop List testing to aid adoption, especially for NVMe-oF • Plugfests get “everyone in one room”, chance to debug underlying transports in a true NVMe-oF environment

  25. Questions?david@iol.unh.edu

More Related