1 / 8

Sockets over RDMA

Explore SDP Protocol License issues, APIs, and interoperability challenges in the realm of networking appliances. Discuss semantic mapping, memory registration, protocol technicalities, and addressing fundamental SDP operational drawbacks.

charlesvaughn
Download Presentation

Sockets over RDMA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sockets over RDMA Arkady Kanevsky Network Appliance

  2. Three legged stool • Protocol • APIs • Interoperability/Backward compatibility

  3. Protocol License issues • MSFT SDP patents • http://www.microsoft.com/about/legal/intellectualproperty/standards/default.mspx • SDP over IB • each company signs 1-on-1 source code license agreement with MSFT for development of SDP. • SDP source code must include MSFT “license” statement • No license for 3rd party or distributors • kernel.org will not accept SDP code under these conditions.

  4. API • MSFT Windows Socket Direct was extended to exposed memory registration • OpenGroup ICSC Extended Socket API • Need for common simple Socket over RDMA API • proper semantic mapping of socket semantic to RDMA • user API with kernel bypass • kernel API

  5. SDP Protocol Technical • SOCK_STREAM in SDP (RC-based) • SOCK_DGRAM not is SDP • SDP 4 data transfer methods • combined into modes: buffered, pipelined, combined • SDP CM manager is not based on IBTA RDMA IP CM Service Annex (post IBTA 1.2) • assigned SID • Single QP • Flow control • No true zero-copy in OFA SDP implementation

  6. Michael Tsirkin’s Protocol comments Hard to fix issues that seem to be fundamental in SDP mode of operation. • No support for SRQ - forces pre-posting of large number of buffers per-socket. • Low connection rate - a single connection can't be reused for multiple sockets • SW-based credits don't utilize hardware flow control, and break polling-based operation • No support for high availability: failover only with APM • No support for multicast and/or UDP (discard packets on RQ overrun). Issues that might be addressable with extensions to the SDP protocol. • SDP Zcopy mode not a good fit for synchronous operations (Zcopy read needs slow 3 way handshake, zcopy write can not pipeline many writes into a single • advertisement) • mode-based operation - can't mix RDMA (for large buffers) and bcopy (for small buffers) • OOB not supported in zcopy mode • SDP connection setup it too different from TCP socket setup, so SDP termination/routing across a TCP socket quite non trivial. • “Lost RTU" problem - analog of SYN flood (seen when remote is unstable/crashing).

  7. Interoperability • What about interoperability with SDP?

  8. New Protocol needed or not? • What are the requirements? • Let the discussion start…

More Related