1 / 7

First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro

First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro. Infiniband in brief. Storage Target. Channel based message passing. TCA. Internet Intranet. Mem Cntlr. CPU. 1. HCA. Link. Link. Link. Link. Switch. Mem Cntlr. CPU. HCA. 2. Link. Link.

ronni
Download Presentation

First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro

  2. Infiniband in brief Storage Target Channel based message passing TCA Internet Intranet MemCntlr CPU 1 HCA Link Link Link Link Switch MemCntlr CPU HCA 2 Link Link Link TCA N/W Target MemCntlr CPU HCA 3 Link Link 1000’s node per subnet Same network to transport low latency ipc, storage I/O and network I/O Link Link xCA Link speed 1x 2.5 Gbps 4x 10.0 Gbps . 12x 30 Gbps MemCntlr Router CPU HCA n 1 x Link 4 x Link 12 x Link

  3. Infiniband Transport Protocols • IBA has been developed with Virtual Interface in mind. VIPL 2.0 includes IBA extensions and RDMA operations. • SCSI RDMA Protocol (SRP). It is a T10 standard. • SRP defines mapping to IBA architecture • it is the transport protocol over IBA • SRP is based on VI • Direct Access Files System (DAFS) • Direct Access Socket (DAS) • TCP/IP functionality over VI/IB fast, low latency network storage fast, low latency TCP/IP sockets File Access Block Access SRP DAFS DAS Virtual Interface over IB IBA Host Channel Adapter

  4. LNL Infiniband Test Bed Leaf Switch 32 1x (2.5 Gbps) ports in 1 U chassis IBA Host Channel Adapter PCI-X (max ~ 380 Mbyte/s) Supermicro P4DL6

  5. Status of Infiniband test bed • All the hardware has been provided by Infiniswitch (1 switch + 4 HCA) • All the hardware is up and running • First p2p measurements have been performed • Software • Virtual Interface Library (VIPL) as provided by Infiniswitch • Send/Receive over reliable connections • RDMA over reliable connections • Sourceforge has a infiniband project over Linux • VIPL source is available. Compiled and works! • Performance as the infiniswitch VIPL (probably they are the same code) • Results • Round trip time small buffers ~ 40 msec (latency 20 msec) • P2p ~ 80% link saturation

  6. First IB p2p tests Link Saturation 220 ~ 40 Mbyte/s Mbyte/s Buffer Size (Bytes)

  7. Work Plan • 2 months IB hardware evaluation (started end of May) • Mainly focus on functionality tests both of the hardware and software available: • SRP – SCSI remote protocol – already got, to be tested • DAS and DAFS if early version will be available • Contacts started with major companies (IBM, Dell, Fujitsu/Siemens) to understand their IB products road map

More Related