Reduced Communication Protocol for Clusters

1. Reduced Communication Protocol for Clusters Clunix Inc. Donghyun Kim 2000.9

2. Introduction • Communication Sub-system Performance is decided by followings • Transmission speed of physical network • I/O handling capability • Overheads of the communication protocol • Communication using traditional protocols is the bottle-neck of parallel systems • Myrinet with TCP/IP is not FAST. • Small-granularity or communication-dense apps show poor performance Clunix Inc.

3. Introduction – cont’d • A high proportion of apps don’t need very complicated communication functions • By practice and theoretic analysis Clunix Inc.

4. Overheads analysis of traditional protocols • Traditional protocols overheads • Time of context switching • Time of data copying • User space – system space, adjacent protocol layers • Time of data partitioning, re-constructing, data analyzing • Time of transmitting packet headers • Time of routing, connection maintaining, traffic controlling, error detecting, recovering, buffer management Clunix Inc.

5. Overheads analysis of traditional protocols - cont’d • End-to-end latency L, bandwidth W modeling • Assumptions : homogeneous, low network traffic T(n) : n-bytes transmission time nmax : comm. subsystem max packet length m : # of protocol layers Ti(n) : i-th protocol layer processing time (T0(n) : physical network transmission time) Clunix Inc.

6. Overheads analysis of traditional protocols - cont’d • : context switching time • : memory bandwidth • 0 : physical network transmission bandwidth i : max packet length of i-th layer I : packet header length of i-th layer ni : data length of i-th layer i : calling expense (routing,traffic control, error detecting, buffer management, connection maintaining) Clunix Inc.

7. Overheads analysis of traditional protocols - cont’d • Analytical & testing results • Testing conclusions • Very large overhead using above IP protocol layer • Memory-to-memory copying is not neglected • If transmission bandwidth is the same as memory bandwidth, data copying(ni+1/) problem is bigger Clunix Inc.

8. Design Strategies for RPC • Support reliable, synchronous, asynchronous communications • Implement reliale broadcast and multicast basing directly on the physical layer • Lay the protocol below the IP layer • Above physical or datalink layer • Avoid data copying AFAP • If possible, avoid buffer management using hardware buffering • Run the protocol entirely in the user space • In the form of libraries Clunix Inc.

9. Implementation of RCP • OSI-DLPI version • Standard physical-device independent data link layer interface • Can write uniform program on different machines and network devices • Myrinet version • Providing user interface like the TCP-socket Clunix Inc.

10. Implementation of RCP – cont’d • RCP supports unicast, broadcast, multicast • RCP addressing • Unique source/destination using hostname+port# • Static address configuration • Supports heterogeneous machines • No connection maintaining, error detecting • Assuming that underlying network is reliable Clunix Inc.

11. Implementation of RCP – cont’d • Sequencing control, traffic control • Sliding-window algorithm+selective retransmission • Windows size is adjusted accoring to retransmission frequency • Fast-Adapt and Slow-Recover algorithm • Very efficient traffic control • Data partitioning and packaging algorithm • Almost no data-copy, work in user-space Clunix Inc.

12. RCP Tesing results Bandwidth(W) Lantency(L) Clunix Inc.

13. Conclusions and future issues • RCP design considerations • How to reduce the overheads • Over-complicated protocol processing • Context switching • Overhead of data copying • How to use the transmission control functions supported by hardware • To reduce the protocol processing • Future Work • To gurantee the quality of the communication. Clunix Inc.