kanoj sarcar sr staff engineer sun microsystems presented by bill strahm
Download
Skip this Video
Download Presentation
Solaris IPoIB (ibd) Implementation

Loading in 2 Seconds...

play fullscreen
1 / 8

Solaris IPoIB (ibd) Implementation - PowerPoint PPT Presentation


  • 138 Views
  • Uploaded on

Kanoj Sarcar Sr Staff Engineer Sun Microsystems (presented by Bill Strahm). Solaris IPoIB (ibd) Implementation. Architecture Overview. DLPI driver Only supported for GID0 Interop (TopSpin, openib) Software structure: IP | SDP | uDAPL | ibd | | | ...

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Solaris IPoIB (ibd) Implementation' - chin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
kanoj sarcar sr staff engineer sun microsystems presented by bill strahm
Kanoj Sarcar

Sr Staff Engineer

Sun Microsystems

(presented by Bill Strahm)

Solaris IPoIB (ibd) Implementation
architecture overview
Architecture Overview
  • DLPI driver
    • Only supported for GID0
  • Interop (TopSpin, openib)
  • Software structure:

IP | SDP | uDAPL |

ibd | | | ...

|IBTF / IBMF |

| HCA driver |

  • Performance
tools changes 20 byte mac
Tools changes (20 byte MAC)
  • New SIOC*XARP ioctls and “struct xarpreq”
    • arp(1M) changes
    • PPPD proxyarp etc
  • Undocumented helper function ala ether_ntoa, ether_aton
    • extern char *_link_ntoa(const unsigned char *, char *, int, int);
    • extern unsigned char *_link_aton(const char *, int *);
  • ifconfig(1M), snoop(1M)
  • RARP
    • /etc/ethers, NIS assume 6 bytes!
tools changes ipoib specific
Tools changes (IPoIB specific)
  • DHCP agent
    • Usage of BROADCAST bit
    • Setting IB hardware type in DHCP packet in htype
    • Client id format
  • Code Len Type |<---------------- Client-Identifier -------------->|
  • +-----+-----+-----+-----+-----+-----+-----+-------------------....----+
  • | 61 | 21 | 00 | 00 (4 octets) | 16 octet GID |
  • +-----+-----+-----+-----+-----+-----+-----+-------------------....----+
  • DHCP Server
    • No explicit ARP cache setting with DHCP chaddr
  • Snoop(1M) parses IPoIB, disallows MAC filters
tools changes ipoib specific part ii
Tools changes (IPoIB specific)Part II
  • ifconfig(1M)
  • hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
  • inet 10.6.98.56 netmask ff000000 broadcast 10.255.255.255
  • ether 0:3:ba:24:4:df
  • ibd1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 2044 index 4
  • inet 192.168.100.100 netmask ffffff00 broadcast 192.168.100.255
  • ipib 0:0:4:a:0:0:0:0:0:0:12:34:0:2:c9:1:9:76:57:11
  • SNMP (net-snmp)
    • ifSpeed consistent with draft-ietf-ipoib-ibif-mib-07.txt
    • ifPhysAddress as MAC address (to match ARP)
    • octet/packet counts only for ipoib network interface
ipoib oddities
IPoIB oddities
  • DLPI receiver indicates “src addr unavailable” for unicast packets
    • If transmitter guarantees GID0 usage, SM assisted reverse lookup possible
  • RARP unsupported
    • Non persistant MAC
    • Server requires client\'s MAC address for 3rd party lookup
ipoib configuration
IPoIB Configuration
  • One time discovery
    • Reconfiguration boot or IO discovery command
    • Create possible ipoib instance per port/Pkey
    • SM must be up
  • Interface initialization
    • IP “plumb” triggers MCG probe as validity check
    • SM must be up, fabric admin must create IP MCG
  • Decisions
    • End nodes do not create IP MCG
    • End nodes do not have to know IB parameters
  • Issues
    • IP multicast group membership requests
    • Can not wait, can not spool!
    • Solaris IP stack joins 224.0.0.1 at interface init
interesting driver details
Interesting Driver Details
  • QPN/CQ/WQE/PathRecord
  • Tx path copy vs registration
  • GID/QPN to Pathrecord (hash/cache)
  • Disallow (DLPI) change MAC address
  • Async thread for (blocking) SM communication
  • IB link up/down handling
    • ULP notification for IPMP failover
  • MCG create/delete trap handling
  • 2 MCG membership lists (full/sendonly, non)
  • MCG pathrecord and membership implication
ad