1 / 11

IP over InfiniBand: Connected Mode

This presentation covers the various modes of IP over InfiniBand (IPoIB) including Unreliable Datagram, Reliable Connected, Unreliable Connected, and Reliable Datagram. It also discusses the IPoIB subnet, connection setup, address resolution, encapsulation, and MTU negotiation.

cvines
Download Presentation

IP over InfiniBand: Connected Mode

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IP over InfiniBand: Connected Mode Vivek Kashyap kashyapv@us.ibm.com Title slide

  2. IP over InfiniBand Modes • Unreliable Datagram • AD review comments • IBTA 1.1 updates • Reliable Connected • Unreliable Connected • Reliable Datagram

  3. IPoIB non-UD options • UC • 2^31 bytes • 2-4K links • Yes • No • Message lost • RDMA Write • RD • 2^31 bytes • 2-4K links • Yes • Yes • Yes • One • RC • 2^31 bytes • 2-4K links • Yes • Yes • Yes Large MTU Automatic Path Migration Reliability RDMA No. of messages outstanding Chart Title Table 2

  4. IPoIB over Connected Mode Reliable Connected Unreliable Connected Sectional slide

  5. IPoIB Subnet • IPoIB-Connected subnet is formed over a Non-Broadcast Multiple Access network • IPoIB-UD is over an abstract broadcast link Node A A IPaIPp IPq IPb NodeB NodeD IPd IPs IPr IPc NodeC IB switchednetwork IPoIB UD broadcast domain IPoIB Connected network Basic text slide

  6. Connection Setup: Address Resolution • The Connection can be setup only when the peer’s GID is known • Manually configured (or Permanent Virtual Circuit) • Inverse ARP • Dynamically resolved • ARP Server as in Classical IP over ATM (RFC2225) or • IPoIB address resolution • Use a UD QP • may reuse IPoIB-UD QP • Define ARP server link address same as IPoIB-UD Broadcast GID • FF12:<401B or 601B>:<P_Key>::< 255.255.255.255 or 1>

  7. Connection Setup: Option 1 • SendFrom IPb to IPa • IPb creates ARP/ND packet with local hardware address of : [RC|UC]:FFFFFF:GIDb • A bit indicates RC or UC to let the peer know of the transmitter’s view of the subnet. • IPa receives the packet Determine PathRecord to GIDb Send reply over UD-QP with GIDa • IPb now creates the IB connection to IPa • Service ID of: • 1:Protocol or FF:0:0:0:0:0:0 • Protocol indicates IPv4 or IPv6 • FF indicates both IPv4 and IPv6 packets will use this connection • Private Data includes the IP address that was resolved

  8. Address Resolution: Option 2 • SendFrom IPb to IPa • IPb creates ARP/ND packet with local hardware address of : [RC|UC]:FFFFFF:GIDb • A bit indicates RC or UC to let the peer know of the transmitter’s view of the subnet. • IPa receives the packet Determine PathRecord to GIDb (same as for IPoIB-UD) Creates QPc of type RC/UC as indicated • may reuse existing QP Send reply with QPc and GIDa • IPb now creates the IB connection to the peer • Service ID of: • 1:QPc:0:0:0:0:0:0 • IPa updates ARP/ND entry with QP value received in IB connection

  9. Encapsulation • Connection can carry IPv4 or IPv6 or both packets. Therefore • Type of packet is known • Type of packet is distinguishable from the IP header • Two options: • IP packets directly encapsulated in the IB connection mode payload • IP packets encapsulated with the ‘Ethertype’ indication as in IPoIB-UD • Address resolution over UD QP needs to follow IPoIB-UD encapsulation

  10. MTU • MTU Negotiated per IB connection • Value requested sent in private data of CM REQ message to peer B • If not acceptable counter value in CM REP message to peer A • If not acceptable REJ with desired value in private data to peer B

  11. IPoIB Connected Mode Vivek Kashyap kashyapv@us.ibm.com Closing slide

More Related