1 / 6

SDP Update

SDP Update. Michael Tsirkin, Eitan Zahavi, Mellanox Technologies Nov 2006. OFED 1.1 SDP Perf’ Status. ~10us latency ~960MBs/DDR 64k Msg BW CPU bound and goes down linearly with Msg size Nagle in OFED does not help as ACKs arrive too fast

wendip
Download Presentation

SDP Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SDP Update Michael Tsirkin, Eitan Zahavi, Mellanox Technologies Nov 2006

  2. OFED 1.1 SDP Perf’ Status • ~10us latency • ~960MBs/DDR 64k Msg • BW CPU bound and goes down linearly with Msg size • Nagle in OFED does not help as ACKs arrive too fast • Nagle = When to send = LEN>MSS or all packets were ACKed. • TCP Cork: is a socket option that cancels the second criteria such that only len>MSS counts Mellanox Technologies – Developers Summit @ SC06

  3. OFED 1.1 SDP Perf’ Status • Improve small Msg BW by new module parameter send_poll=0 • Provides 300MB/sec for ½KB Msg • Delaying the send completion by waiting for interrupts (stops polling on sender – thus send completion late) Mellanox Technologies – Developers Summit @ SC06

  4. SDP Future Work • Memory Scalability • Define an adaptive algorithm • When to tighten memory allocation per connection • New posted buffers reclaim protocol • Allow one side to reclaim posted buffers • Possibly require other side to NOP ACK Mellanox Technologies – Developers Summit @ SC06

  5. SDP Future Work (Cont) • Small Message BW • Merge messages waiting on send queue • Send-page • Used by send-file API • Maps to Send-Side-Zero-Copy • Netstat • Provide info on active connections • Track active connections, user, src/dst IP and ports in /proc • Require migrating AF_INET_SDP to PROTO_SDP • Provide as netstat patch Mellanox Technologies – Developers Summit @ SC06

  6. Other topics • ZCopy • Mellanox published Gen1 based ZCopy implementation • SDP protocol limits BW improvements to >4K messages • Development delayed as low ROI (complex implementation) • AIO • Seems like Linux implementation is under retrofit • See mail threads: “kevent: network AIO, socket notifications” • Debate and proposal for new API on OLS 2006 • CMA work • Ongoing: APM • Support IPv6 Mellanox Technologies – Developers Summit @ SC06

More Related