230 likes | 484 Views
Introduction to RDMA Storage Systems. Mr. P. K. Gupta. Sandeep Gupta 051416 Roopak Agarwal 051309. Road Map: Problem Statement Introduction Storage Systems And Storage Devices Brief Introduction To SAN and NAS DMA and RDMA How Does RDMA Works RDMA Application Future Work.
E N D
Introduction to RDMA Storage Systems Mr. P. K. Gupta Sandeep Gupta 051416 Roopak Agarwal 051309
Road Map: Problem Statement Introduction Storage Systems And Storage Devices Brief Introduction To SAN and NAS DMA and RDMA How Does RDMA Works RDMA Application Future Work
Problem Statement • Network Bandwidth: 40%/yr • CPU Power: x2 every 18 months • Storage Capacity x2/yr • Memory and I/O bandwidth?
A Server-to-Storage Bottleneck Source: Brocade
Storage Area Network (SAN) Storage accessed over a Fibre Channel switching fabric, using encapsulated SCSI. • Block level storage system • Fibre-Channel SAN • IP SAN • Implementing SAN over well-known TCP/IP • iSCSI: Cost-effective, SCSI and TCP/IP
NAS • Network Attached Storage(NAS) • Storage accessed over TCP/IP, using industry standard file sharing protocols like NFS, HTTP, Windows Networking • Provide File System Functionality • Take LAN bandwidth of Servers
Motivation for RDMA • Motivation • Take advantage of the RDMA protocol suite & RNIC hardware. • When RDMA technology advances, so does iSCSI. • Zero-copy, direct data placement can be achieved with generic protocols. • High-level goal: Run “most efficiently” on generic RNICs.
Direct Memory Access (DMA) • Direct Memory Access (DMA) is a capability provided that allows data to be sent directly from an attached device to the memory on the computer's motherboard.
Remote Direct Memory Access (RDMA) RDMA is a concept whereby two or more computers communicate via Direct memory Access directly from the main memory of one system to the main memory of another .
DMA RDMA
What Makes RDMA Advantageous ? • There is no CPU cache or context switching overhead needed to perform the transfer. • Transfers can continue in parallel with other system operations.
Some Objections to RDMA • More complex API than TCP/stream interface • A new API is not required (but worthwhile for some applications) • Sockets Direct Protocol (SDP) • Hardware vendors must all agree for approach to succeed in the market • –Extremely valid, but worth the cost • Security concerns about opening memory on the network • Hardware enforces application buffer boundaries. • Makes it no worse than existing security problem with a 3rdparty inserting data into the TCP data stream.
RDMA Applications • Clustered Database , Application Servers : They generate a large number of short messages , typically sent for synchronization and locking among nodes, so latency affects the degree to which they can scale. • High performance Computing • Storage Infrastructure
Future Work • RDMA over WAN • iSCSI extension for RDMA (iSER)
iSCSI Extension for RDMA (iSER) : • Protocol that maps the iSCSI protocol, available from RDMA • Only data transfer in iSCSI protocol is changed • Reduces TCP/IP overhead
iSER Big picture A TCP-based implementation is shown as an example. SCSI • Layered on RDMAP protocol. • iSCSI still requires direct TCP access for bootstrapping (login). • Transition to RDMA • Same iSCSI well-known port iSCSI Datamover Interface 1 2 iSER iWARP Verbs RDMAP DDP Generic RDMA over TCP/IP iWARP protocol suite MPA TCP RNIC