iSER on IB

1. iSER on IB draft-hufferd-iser-ib-00.txt/pdf Generalizations to iSER specification John Hufferd Mike Ko Yaron Haviv

2. Abstract • The iSCSI Extensions for RDMA document currently specifies the RDMA data transfer capability for iSCSI over iWARP/TCP • This new document generalizes the iSER document to permit it to be used with other RDMA capable protocols such as • InfiniBand

3. Motivation • Current wordage in iSER is only applicable to iSER on iWARP/TCP • It was felt that the protocol should be made generic for RDMA LLPs, and specifically for: • InfiniBand • Makes an iSCSI based protocol apply across both IB and iWARP Networks thus • Enabling the same basic iSCSI protocol on TCP/IP, iWARP, & IB • Eliminating some of the management and discovery protocols that would otherwise be needed • For example, would not have to fix the short comings of SRP (SCSI RDMA Protocol) on IB

4. Goals • To specify changes/adjustments to the wordage in the iSER document to make it more General and specifically applicable to IB • These changes should not modify the basic operation of iSCSI/iSER when operating on iWARP/TCP

5. Examples of changes (1) • The term “iWARP protocol suite” is generally replaced by “RDMA-Capable Protocol” (RCP) • The protocol or protocol suite that provides the RDMA functionality, e.g., iWARP, InfiniBand • Wherever the “iWARP” term is specific to the TCP implementation, it is replaced with “iWARP Layer” • The term “RNIC” is replaced with “RDMA-Capable Controller” (RCC) • The clause “such as an RNIC” will be used when an iWARP implemented RCC is intended

6. Examples of changes (2) • The Steering Tag (STag) term has its definition extended such that the IB “Local Steering Tag (L-Key)” and the “Remote Steering Tag (R-Key)” are included in the STag definition by way of example • The Definitions for IRD and ORD terms will have their definition extended such that the IB “Responder Resources”, and Initiator Depth” are included in the definitions by way of example • The Term RDMA-capable protocol (RCP) is used when ever any RDMA wire protocol or RDMA protocol stack is applicable • RDMAP is used only when it explicitly refers the iWARP protocol

7. Examples of changes (3) • The Term “RDMAP Stream” is replaced by the term “RCP Stream” and defined as: • RCP Stream - A single bidirectional association between the peer RDMA-capable protocol layers on two Nodes over a single transport-level stream. • For TCP an RCP Stream is also known as an RDMAP Stream • For iSER/TCP, the association is created when the connection transitions to iSER-assisted mode following a successful Login Phase during which iSER support is negotiated • Needed since IB start its RCP stream mode at connection time

8. Examples of changes (4) • The term “RCP Message” is defined and used as a replacement for the term “RDMAP Message” the definition is: • RCP Message – The sequence of packets of the RDMA-capable protocol which represent a single RDMA operation or a part of RDMA Read Operation. For TCP, an RCP Message is also known as an RDMAP Message • When discussing the iSER Hello and HelloReply Messages the term "iSER Message" is used • Instead of “RDMAP Message” • This distinction is needed in order to accommodate LLPs that have native message delivery capability, such as or IB

9. Example of Changes (5) • The following is placed/updated in the Acronym Section • HCA Host Channel Adapter • IB InfiniBand • IPoIB IP over InfiniBand • LLP Lower Layer Protocol • TCA Target Channel Adapter

10. Example of Changes (6) • We permit the iSCSI layer (if appropriate) to use the RCP message mode capability immediately after connection establishment before enabling iSER-assisted mode • Appropriate for IB • In this case the iSER Hello and HelloReply Messages are not the first RCP Messages, but they are the first iSER Messages • Added a discussion of connection establishment along with the use of the “RCP messaging protocol”, for exchanging Login Request and Login Response Messages • Examples are discussed that are appropriate for IB, along with the transitioning of the connection to iSER mode

11. Examples of changes (7) • The discussion of Security specifies that all non IP protocols will define their own requirements for IPsec • However the iSCSI requirements for IPsec are still required: • Wherever an iSER Message enters an IP environment from a non IP one (such as IB) • There is now a SHOULD implement (optional to use) requirement on non IP networks for a packet by packet security facility that is at least as strong as that required by iSCSI Note: “Should” is used instead of “MUST” since currently IB does not have such a capability. Also, since some implementations use IB like an internal PCI-e Bus, some felt it inappropriate to have a “MUST”

12. Generic example of iSCSI/iSER layering in Full Feature Mode

13. IB Informative Appendix (B) • Explains how an InfiniBand RC connection can be used to carry the iSER protocol • Various IB Network topologies are shown • Host Side IB Network, including Gateways • Storage Side that includes iSER/IB • IB iSER discovery process is described • Use of IP over IB (IPoIB) • SendTargets • SLP • iSNS • Conversion of IP address into IB GID via ARP processes • Discussion of what the IBTA are expected to define

14. Recommendations • Accept this Draft as an IPS Draft

15. Backup Charts

16. Host Side iSER/IB Topology

17. Storage Side with iSER/IB

18. Info: What is Expected that IBTA will define • The iSER ServiceID • Means for permitting a Host to establish an iSCSI/iSER connection with a peer InfiniBand end-node • Indicating when that end node does not support iSER • So the Host would be able to fall back to iSCSI/TCP over IPoIB • Means for permitting the Host to establish connections with • IB iSER connections on Storage Controllers in preference to a Gateway path or • IB iSER connected Gateways in preference to IPoIB connected Gateways/Bridges or • Connections to Target Storage Controllers that accept iSCSI via IPoIB • How the ServiceID can be added to the IP port number during the connection process