Realizing the Performance Potential of the Virtual Interface Architecture

1. Realizing the Performance Potential of the Virtual Interface Architecture Evan Speight, Hazim Abdel-Shafi, and John K. Bennett Rice University, Dep. Of Electrical and Computer Engineering Presented by Constantin Serban, R.U.

2. VIA Goals • Communication infrastructure for System Area Networks (SANs) • Targets mainly high speed cluster applications • Efficiently harnesses the communication performance of underlying networks

3. Trends • The peak bandwidth increase two order of magnitude over past decade while user latency decreased modestly. • The latency introduced by the protocol is typically several times the latency of the transport layer. • The problem becomes acute especially for small messages

4. Targets VI architecture addresses the following issues: • Decrease the latency especially for small messages (used in synchronization) • Increase the aggregate bandwidth (only a fraction of the peak bandwidth is utilized) • Reduce the CPU processing due to the message overhead

5. Overhead Overhead mainly comes from two sources: • Every network access requires one-two traps into the kernel • user/kernel mode switch is time consuming • Usually two data copies occur: • From the user buffer to the message passing API • From message layer to the kernel buffer

6. VIA approach • Remove the kernel from the critical path • Moving communication code out of the kernel into user space • Provide 0-copy protocol • Data is sent/received directly into the user buffer, no message copy is performed

7. VIA emerged as a standardization effort from Compaq, Intel, and Microsoft It was built on several academic ideas: • The main architecture most similar to U-Net • Essential features derived from VMMC Among current implementations : • GigaNet cLan – VIA implemented in hardware • Tandem ServerNet –VIA software driver emulated • Myricom Myrinet - software emulated in firmware

8. VIA architecture

9. VIA operations Set-Up/Tear-Down : • VIA is point-to-point connection oriented protocol • VI-endpoint : the core concept in VIA • Register/De-Register Memory • Connect/Disconnect • Transmit • Receive • RDMA

10. VIA operations Set-Up/Tear-Down :VIA is point-to-point connection oriented protocol • VI-endpoint : the core concept in VIA • VipCreateVi function creates a VI endpoint in the user space. • The user-level library passes the call to the kernel agent which passes the creation information to the NIC. • OS thus controls the application access to the NIC

11. VIA operations - cont’d Register/De-Register Memory: • All data buffers and descriptors reside in a registered memory • NIC performs DMA I/O operation in this registered memory • Registration pins down the pages into the physical memory and provides a handle to manipulate the pages and transfer the addresses to the NIC • It is performed once, usually at the beginning of the communication session

12. VIA operations - cont’d Connect/Disconnect: • Before communication, each endpoint is connected to a remote endpoint • The connection is passed to the kernel agent and down to the NIC • VIA does not define any addressing scheme, existing schemes can be used in various implementations

13. VIA operations - cont’d Transmit/receive: • The sender builds a descriptor for the message to be sent. The descriptor points to the actual data buffer. Both descriptor and data buffer resides in a registered memory area. • The application then posts a doorbell to signal the availability of the descriptor.The doorbell contains the address of the descriptor. • The doorbells are maintained in an internal queue inside the NIC

14. VIA operations - cont’d Transmit/receive (cont’d): • Meanwhile, the receiver creates a descriptor that points to an empty data buffer and posts a doorbell in the receiver NIC queue • When the doorbell in the sender queue has reached the top of the queue, through a double indirection the data is sent into the network. • The first doorbell/ descriptor is picked up from the receiver queue and the buffer is filled out with data

15. VIA operations - cont’d RDMA: • As a mechanism derived from VMMC, VIA allows Remote DMA operations: RDMA Read and Write • Each node allocates a receive buffer and registers it with the NIC. Additional structures that contain read and write pointers to the receive buffers are exchanged during connection setu • Each node can read and write to the remote node address directly. • These operations posts potential implementation problems.

16. Evaluation Benchmarks • Two VI implementations : • GigaNet cLan B:125MB/sec, Latency 480ns • Tandem ServerNet, 50MB/S, Latency 300ns • Performance measured: • Bandwidth and Latency • Poling vs. Blocking • CPU Utilization

17. Bandwidth

18. Latency

19. Latency Polling/Blocking

20. CPU utilization

21. MPI performance using VIA • The challenge is to deliver performance to distributed application • Software layers such MPI are mostly used between VIA and the application: provide increased usability but they bring additional overhead • How to optimize this layer in order to use it efficiently with VIA ?

22. MPI VIA - performance

23. MPI observations • Difference between MPI-UDP and MPI-VIA-baseline is remarkable • MPI-VIA-baseline is dramatically far from VIA-Native • Several improvements proposed to shift MPI-Via to be closer to VIA native : reduce MPI overhead

24. MPI Improvements • Eliminating unnecessary copies: MPI UDP and VIA use a single set of receiving buffers, thus data should be copied to the application : allow the user to register any buffer • Choosing a synchronization primitive: All synchronization formerly using OS constructs/events. Better implementation using swap processor commands • No Acknowledge: Remove the acknowledge of the message by switching to a reliable VIA mode

25. VIA - Disadvantages • Polling vs. blocking synchronization – a tradeoff between CPU consumption and overhead • Memory registration: locking large amount of memory makes virtual memory mechanisms inefficient. Registering / deregistering on the fly is slow • Point-to-point vs. multicast: VIA lacks multicast primitives. Implementing multicast over the actual mechanism, makes communication inefficient

26. Conclusion • Small latency for small messages. Small messages have a strong impact on application behavior • Significant improvement over UDP communication (still after recent TCP/UDP hardware implementations?) • At the expense of an uncomfortable API