60 likes | 236 Views
Myrinet. Arthur B. (Barney) Maccabe Computer Science Department The University of New Mexico. MPI Performance (using GM) (Edgar Leon-Borja using 32-bit LANai 7’s). Myricom Bandwidth >140 MB/sec Latency 18 sec. The Good. Programmable NICs Good development environment
E N D
Myrinet Arthur B. (Barney) Maccabe Computer Science Department The University of New Mexico
MPI Performance (using GM)(Edgar Leon-Borja using 32-bit LANai 7’s) • Myricom • Bandwidth >140 MB/sec • Latency 18 sec
The Good • Programmable NICs • Good development environment • Experimental protocols (Portals, GM, FM/HPVM, PM, Trapeze) • Cut through routing • End-to-end flow control • No dropped messages • Stupid Switches • Source routing • Scalable price/performance
The Bad • Stupid switches • End points need to deal with congestion control • Small packets might help • Slow processors on the NICs • Using small packets has a significant cost • OS Bypass: put OS policies on the NIC • Application Bypass: put application policies on the NIC • PCI bus • Limits bandwidth to memory • Poor address mapping (may not be that much of a problem)
The Ugly • InfinaBand • Almost VIA (it’ll be great for systems with 64 nodes) • Sits on memory cross bar • PCI-X? • Fibre Channel • 10 Gig Ethernet • May be a great fabric • We need to look at end-to-end performance