1 / 31

Roland Dreier Technical Lead – Cisco Systems, Inc. OpenIB Maintainer Sean Hefty

Stack Development Update, Testing Status and Target Release Goals. Roland Dreier Technical Lead – Cisco Systems, Inc. OpenIB Maintainer Sean Hefty Software Engineer – Intel Corporation OpenIB Maintainer Yaron Haviv CTO – Voltaire. Agenda. www.openib.org Linux Kernel Infrastructure

marymack
Download Presentation

Roland Dreier Technical Lead – Cisco Systems, Inc. OpenIB Maintainer Sean Hefty

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stack Development Update, Testing Status and Target Release Goals Roland Dreier Technical Lead – Cisco Systems, Inc. OpenIB Maintainer Sean Hefty Software Engineer – Intel Corporation OpenIB Maintainer Yaron Haviv CTO – Voltaire

  2. Agenda • www.openib.org • Linux Kernel Infrastructure • Linux Userspace Infrastructure • RDMA Enabled Applications • InfiniBand and Beyond The InfiniBand Software Solution

  3. www.openib.org OpenIB Alliance • Improve interoperability • Reduce the time and cost for deployment Open-Source InfiniBand RDMA Infrastructure

  4. www.openib.org OpenIB Alliance …and the open source community!

  5. www.openib.org Philosophy • Open source development • Dual BSD / GPL License • Integrated with OS infrastructure • Working with companion RDMA standards • Anyone can contribute Code Quality, Stability, Performance

  6. Infiniband Leaders Form OpenIB Alliance “This will enable … a robust, resilient, and manageable infrastructure for data center, Grid, and HPC.” Deliver a high-performance, scalable, and interoperable Infiniband software stack “… accelerate the creation of a single Infiniband software stack” www.openib.org Achievements Jun Nov Mar May Jul ‘04 ‘05 Will support Windows & RDMA enabled Ethernet OpenIB Linux release

  7. www.openib.org Software Architecture

  8. Linux Kernel Infrastructure

  9. Linux Kernel Infrastructure Overview • Verbs API derived from VAPI • Best of breed software solution • Support multiple HCA • Optimized for Linux kernel • Support latest stable kernel release Complete InfiniBand infrastructure

  10. Linux Kernel Infrastructure Architecture

  11. Linux Userspace Infrastructure

  12. Linux Userspace Infrastructure Overview • Library exposes verbs to usermode clients • Secure userspace access • Kernel bypass for speed path operations • RDMA to / from application memory Direct hardware access for applications

  13. Linux Userspace Infrastructure Architecture openib.org 2.6.13

  14. RDMA Enabled Applications

  15. Applications Network (SDP & IPoIB) Storage (File & Block) IPC (MPI & DAPL) IB HCA InfiniBand Host Software Stack RDMA Enabled Applications One Fabric, Many Uses Servers • Low latency IPC for clustering • Variety of high speed file and block protocols, remote boot capable • Bridging to external GbE and FC InfiniBand Multi-Service Switches Management One fabric has it all Parallel FC / IB / IPStorage

  16. RDMA Enabled Applications Networking / IPC APIs Fabric agnostic RDMA API Application APIs unchanged Standard API for scientific clustering Tunnels IP packets over IB Bypasses TCP/IP stack for direct HW access OpenIB software enables applications

  17. RDMA Enabled Applications Networking / IPC Performance How well can apps perform? • 3.4 GHz Dual EM64T Xeon • 4 GB DDR • RedHat EL4.0 • OpenIB SVN 2833 backported • 10 Gb/sec (4X) - 8X PCI Express HCA MTLP25208 Ciara VXB-7520J MTS2400

  18. RDMA Enabled Applications Networking / IPC Performance • Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing. I.e. your results may vary

  19. RDMA Enabled Applications Networking / IPC Performance Small message latency Large message bandwidth 924 MBps 3.46 us IPoIB SDP MPI Verbs

  20. RDMA Enabled Applications InfiniBand Revolutionize Storage A lot of bandwidth(InfiniBand) Switch • High-speed NAS and SAN on the same fabric • Scale out storage through clustering and virtualization Server Racks A Bunch of low cost Storage Arrays Commodity Servers running NAS Gateways or Cluster File System

  21. NIC R-NIC IB HCA RDMA Enabled Applications Storage in OpenIB Storage Switch

  22. High-speed RDMA RPC transport Maps transparently to NFS and applications Reside over kDAPL for fabric independence RDMA Standard NIC RDMA Enabled Applications NFS/RDMA NFS RPC/XDR Switch RDMA TCP UDP kDAPL Direct Access

  23. Xen with IB VM1 VM2 VM3 OS OS OS VM3 VM1 VM2 IPC OS OS OS Direct HW access XEN GbE NIC FC HCA IB HCA RDMA Enabled Applications Xen Virtualization over IB Xen, the traditional way Storage Network Software based Network and Storage switching XEN • Slower I/O • No isolation • Multiple cards and fabrics • Direct HW access for I/O • One 10Gb/s card for Network, Storage, and IPC • Scale out using clustering (IPC)

  24. Infiniband and Beyond What’s left?

  25. InfiniBand and Beyond Future Efforts • Scale out testing • Scalability enhancements • Integrate upper-layer protocol software • Windows software release • Xen support

  26. InfiniBand and Beyond Companion Technologies • Working with OpenRDMA Group • Common RDMA interface for Linux • Support InfiniBand and iWarp

  27. OpenIB – Open-Source Infiniband RDMA Infrastructure Summary OpenIB = InfiniBand software • Solid InfiniBand infrastructure • Available in Linux today, Windows soon • Low latency, high bandwidth applications

  28. InfiniBand and Beyond Call to Action Join OpenIB • Join the open source effort • Request OpenIB software from your Linux distributor • Ask your IB vendor about OpenIB support

  29. Please fill out the Session Evaluation Form.Thank You!

  30. OpenIB – Open-Source Infiniband RDMA Infrastructure Glossary • CM – communication manager • DAPL – direct access provider library • GSI – general services agent • HCA – host channel adapter • IPC – interprocess communication • IPoIB – IP over Infiniband • iSER – iSCSI extensions for RDMA • PMA – performance management agent • SDP – sockets direct protocol • SM – subnet manager • SMI – subnet management interface • SRP – SCSI RDMA procotol

  31. Additional Resources for this Session • Session presentation can be downloaded from the IDF web site – when prompted enter: • Username: idf • Password: fall2005 • More web based info: www.openib.org

More Related