1 / 23

OFED - Status and Process

OFED - Status and Process. November 2007 Tziporet Koren. Agenda. OFED 1.3 Status OFED Development Process OFED Future Discussion. OpenFabrics Software Stack. OFED – Update Since Sonoma. OFED 1.2 was released on 22-Jun Kernel base: 2.6.20 OFED 1.2.5 was released on 16-Aug

jerrold
Download Presentation

OFED - Status and Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OFED - Status and Process November 2007 Tziporet Koren

  2. Agenda • OFED 1.3 Status • OFED Development Process • OFED Future Discussion

  3. OpenFabrics Software Stack

  4. OFED – Update Since Sonoma • OFED 1.2 was released on 22-Jun • Kernel base: 2.6.20 • OFED 1.2.5 was released on 16-Aug • Kernel base 2.6.22 • ConnectX support • OFED 1.2.5.3 was released on 14-Nov • RDS in GA for Oracle 11g RAC • OFED 1.3 – in middle of release phase • Kernel base 2.6.24

  5. OFED 1.3 Components OFA development Add on • HCA/NIC Drivers • Mellanox, QLogic, IBM, Chelsio, NetEffect • Core: verbs, mad, SMA, CMA, SA cache • IPoIB • SDP • SRP, iSER • SRP Target • RDS • Qlogic_VNIC • UDAPL • OSM • Diagnostic tools • NFS-RDMA – still open • Bonding module • MPI Components • MVAPICH • Open MPI • MVAPICH2 • Benchmark tests • Proprietary MPIs: Intel, HP, Scali • Proprietary SMs: Cisco, Voltaire, Qlogic Tested with

  6. Main New Features - General • Base kernel: 2.6.24 • Install • Minimize integration effort into OS distribution • User level split into separately-installable RPMs • Based on feedback from Novell and Redhat • Package • Sources arrangement for the end user • Added the make-dist script • Reduce compilation warnings of backports

  7. General Features – Cont. • QoS • OSM, CM & CMA, ULPs: SRP, SDP, iSER, IPoIB • Updated SA cache • Integrated with sa_query module • Use of SA events (GID up/down) to provide additional synchronization • User-space control for dynamic enable/disable and update • Extended RC (XRC) Support • New verbs to enable XRC • New modules • NES (NetEffect driver) • SRP Target: Based on Mellanox IBGD SRPT and interfaces with SCST scsi target middle level driver

  8. Main New Features – ULPs • IPoIB • NAPI • Stateless offloads – for Datagram mode • Small message performance improvements • RDS: RDMA with Zero copy (API version 3) • SDP: • Asynch IO • Send Zero Copy • uDAPL 2.0 • In addition to uDAPL 1.0

  9. Main New Features – Management • OpenSM • Quality of Service manager • QoS implementation of the IBTA annex • QoS policy parser and support for SA (PR, MPR) • Scalability/stability improvements • Performance improvements of the routing algorithms (min-hops, up/down, lash) • Optimized port and switch tables update policy • “Native” daemon mode • Performance manager • Fetches and stores performance counters • OpenSM integrated, work in any OpenSM state (standby, disabled) • Plug-in interface • Diagnostic tools enhancements • IBSIM – IB fabric simulator (new) • Emulates user_mad layer • useful for OpenSM and management tools testing

  10. Main New Features - MPI • MVAPICH 1.0 • Asynchronous progress • Flexible message coalescing • APM support • Enhanced mpirun_rsh for scalable launching • UD-based design • Optimized and tuned collectives • Support for ConnectX • Support for QLogic/PSM • Open MPI 1.2.5 • ConnectX support • MVAPICH2 1.0 • Message Coalescing • Hot-spot avoidance • Application-initiated systems-level checkpointing • APM support • Multi-rail support for iWARP • On-demand connection management for iWARP and uDAPL • RDMA Read • Blocking

  11. OFED 1.3 Release Status • Features freeze: Oct 10, 2007 • Alpha: Oct 15, 2007– kernel base: 2.6.23 • Beta: Nov 20, 2007– kernel base: 2.6.24-rc2 • RC1: Dec 4, 2007 • RC2: Dec 18, 2007 • RC3: Jan 8, 2008 • RC4: Jan 22, 2008 • Release: Jan 29, 2008 Note: The delay mainly caused by the move to kernel 2.6.24

  12. OFED 1.3 OS Matrix • kernel.org: kernel 2.6.23 and 2.6.24 • Novell • SLES 10 • SLES 10 SP1 (up1) • Redhat • RHEL 4 (up4 and up5) • RHEL 5 (up1) • Free distros • Partial support only (no QA) • Open SuSE 10.2 • Fedore C6 • Ubuntu 6.06 (with RPM package installed) • Open: OSVs plans to pick up OFED 1.3, or even 1.2.5

  13. Agenda • OFED 1.3 Status • OFED Development Process • OFED Future Discussion

  14. OFED – Development Process • Start – release definition • Agree on release features • Agree on OS matrix and kernel base • Decide on target schedule • Release definition is placed on the Wiki https://wiki.openfabrics.org/tiki-index.php?page=OFED+Development+Wiki • Note • In all OFED releases there were changes in the above that caused release slips • Tracking meetings • Bi-weekly meetings during the development phase • Weekly meetings on the release phase • Meeting summaries are placed on the Wikihttps://wiki.openfabrics.org/tiki-index.php?page=Meeting+Minutes • Bug tracking in bugzilla • https://bugs.openfabrics.org/

  15. Kernel Development • Have one kernel git tree that is maintained by Vlad • git://git.openfabrics.org/ofed_1_3/linux-2.6.git • Cloned from Linux git tree • Add-on components (e.g., SDP) are added to the tree • New features and bug fixes are added as patches under the fixes directory • Patch convention name: <module>_<number>_<text>.patch (e.g., ipoib_0010_cq_coalescing.patch) • Supporting distros and previous kernels: • Backport directory for each kernel OFED can compile on • Add-on directory for headers from Linux • The make-dist script applies first the “fixes” patches and then the appropriate backport patches

  16. User Space Development • Each maintainer has a separate git tree on the server • Examples git://git.openfabrics.org/~sashak/management.gitgit://git.openfabrics.org/~shefty/libibcm.git • Each package has its own RPM SPEC file • All packages should also be released to the ofa download site • http://www.openfabrics.org/downloads.htm • Approaching a release – a branch with the release name is opened • Userspace component fixes must be committed -- no patching scheme is used

  17. External Packages Development • External packages are provided as source RPMs • Each external package has a directory on the ofa server • Example – the bonding package:/home/monis/public_html/ofed_1_3/ • latest.txt file points to the package that should be used • Ex.: ib-bonding-0.9.0-19.src.rpm

  18. OFED Daily Build • An OFED package is created everyday at 6am PST • http://www.openfabrics.org/builds/ofed-1.3/ • This package can be used by all to test the daily changes • This package includes • Source RPMs for kernel, all user space components, external modules • OFED scripts (install, etc.) • OFED documents • Development phase: the build uses the master branch of all git trees • Release phase: the build is based on the release branch

  19. OFED Release Phase • Starts after the alpha release • Close tracking of components status • Alpha, beta and RC are done according to the package maturity • Bugs severity tracking  which bugs must be fixed for the release • Update of all documents

  20. Agenda • OFED 1.3 Status • OFED Development Process • OFED Future Discussion

  21. OFED Goals • Enterprise Working Group within Open Fabrics Alliance (OFA) • OFED has become a brand name in the industry today • Collaborative effort to test & release OFA software • Broader test participation • Components – Kernel & User Space • Interoperability • Add-on components for vendors to differentiate above OFA Reduces deployment complexity and cost

  22. Reasons to Continue OFED • Industry synchronization point • Set of kernel modules, libraries and applications that are tested together • Comprehensive tests by many vendors • OSVs rely on our qualification work • Faster new feature development and deployment • New drivers for new IHV devices • ULP updates for new ISV software packages, e.g., Oracle 11g • Enables support of new features in existing OSes • Ex.: Bonding to work on Redhat EL4 up5, SLES 10, etc

  23. OFED-based and OS Distro-based • OFED-based and OS Distro-based distribution of OFA software – one or both • Originally, some people intended OFED to be a short-term distribution until mainstream Linux distributions caught up and were able to distribute IB/iWARP support • Today there is synergy between OFED and OS vendors as they rely on the OFED quall and support benefits delivered by commercial IB/iWARP suppliers • Are OSV’s ready to take more quall and support responsibilities at this time? • Discussion …

More Related