1 / 10

OpenFabrics 2.0 or libibverbs 1.0

OpenFabrics 2.0 or libibverbs 1.0. Sean Hefty Intel Corporation. Pre- OpenIB. Multiple IB vendors each with a proprietary software stack Multiple versions of the ‘verbs’ interface Binary incompatible Formation of OpenIB

komala
Download Presentation

OpenFabrics 2.0 or libibverbs 1.0

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OpenFabrics 2.0or libibverbs 1.0 Sean Hefty Intel Corporation

  2. Pre-OpenIB • Multiple IB vendors each with a proprietary software stack • Multiple versions of the ‘verbs’ interface • Binary incompatible • Formation of OpenIB • Focused on creating a single, open source software stack for InfiniBand • Apps have 1 API to target, distros have 1 solution to support • Greatly increased adoption www.openfabrics.org

  3. OpenIB • Resulting software stack was hardware facing • Verbs is a hardware semantic • Was never intended to be an API • Application developers have always disliked it • OpenIB expanded to include iWarp • Changed name to OpenFabrics • Folded iWarp support under InfiniBand API www.openfabrics.org

  4. OpenFabrics Today • The fundamental stack is largely unchanged • The result: • There are multiple, binary incompatible versions of the ‘verbs’ interface • There are vendor-specific APIs • FCA, MXM, PSM, UCCS • Sounds a lot like the pre-OpenIB days www.openfabrics.org

  5. Why Not Just Extend Verbs? • Author’s prediction: will not succeed • OIB created a single verbs solution, which later split • The split was a direct result of needing extensions • Additional APIs have been already been developed that are NOT part of verbs or extended verbs • Existence proof against • There are fundamental issues with the API that will prohibit scaling to very large systems www.openfabrics.org

  6. Proposed Alternative • Develop application facing APIs and let each vendor determine the best way to support those APIs • There is no intent to disadvantage any hardware solution • Define mechanisms for migrating providers and apps to the new framework www.openfabrics.org

  7. (Scalable) Fabric Interfaces Fabric Interfaces Q: What is implied by incorporating interface sets under a single framework? Objects exist that are usable between the interfaces Isolated interfaces turn the framework into a complex dlopen Interfaces are composable May be used together Control Interface Atomics Message Queue RDMA Collective Operations Active Messaging CM Services Tag Matching www.openfabrics.org

  8. Migrating Providers from Verbs to FI RDMA Message Queue CM Services FI libfabric Verbs ibverbs abstraction layer libibverbs RDMA CM Verbs Provider Verbs Provider www.openfabrics.org

  9. Migrating Apps from Verbs to FI • Expose ‘verbs’ interfaces directly from FI • Use macros to convert ‘libibverbs’ exported calls to FI calls • Or layer libibverbs over libfabric • Applications must recompile • Minimal benefit to app www.openfabrics.org

  10. Migrating Apps from Verbs to FI • Define ‘verbs’ compatibility mode • Allows mapping objects between interfaces • E.g. QP fabric socket CQ  EC • Restricts implementation • Mapping must be documented • Allow software to adopt new interfaces selectively • E.g. send/recv/ec_read SFI verbs libibverbs libfabric Dual-Provider Library Verbs Provider SFI Provider www.openfabrics.org

More Related