100 likes | 263 Views
HPPNET: A Novel Network for HPC and Its Implication for Communication Software. Panyong Zhang , Can Ma, Jie Ma, Qiang Li, Dan Meng National Research Center for Intelligent Computing Systems , ICT,CAS. Why HPPNET?. P. P. P. …. NIC. NIC. P. P. P. …. Packet. P. P. P. ….
E N D
HPPNET: A Novel Network for HPC and Its Implication for CommunicationSoftware Panyong Zhang, Can Ma, Jie Ma, Qiang Li, Dan Meng National Research Center for Intelligent Computing Systems, ICT,CAS.
Why HPPNET? P P P … NIC NIC P P P … Packet P P P … single-side communication Remote ld/st User-space RDMA Device Driver NIC Multi-link multiplex NIC Device Driver P P P … Core to Core Node to Node • Support HPP Architecture Global Load/store • PGAS Language fine-grained global access load/store support • “Communication Wall” Multi-Link, User-space RDMA • Short Message/Large Message Separated Channel Design
What is HPP architecture? • Focus on peta-scale Challenge • MPP + Cluster • Global Address space • RT+ Single-image OS +Management Software • General Application • Compatible with Cluster hardware, software HPPNET
What HPPNET did? • HyperNIC • HT-enabled Network Interface • Different Channel abstraction for different message sizes • 16 ports HyperSwitch • Hardware support Reliable Barrier, unreliable Multicast • Multiple link • 4 separate links, 2.5Gbps each 10Gbps • Fine-grained Load/Store in global physical address space
HyperNIC • HyperTransport interface to Processor • HT2.0 Compatible • No Embedded Processor • Command window based • User map the Control Page of HyperNIC to operate NIC. • Different core use different channel, =>different control page • Separate Channel Design • NAP (non-address pointer) Channel • MSG (Message) Channel • Global Load/Store Module
Separate Channel Design • NAP Channel • Latency Oriented • Send/Recv Sematics • Determined Routing • MSG Channel • Bandwidth Oriented • RMA Semantics • Adaptive Routing BCL-5 did the Port Abstraction
Global Load/Store support • Global Physical address spaces • Non-cache coherent in global region • HyperNIC Trap the HT global load/store access instruction, issue network operation • OS page table do the address translation • global_address_mmap() • global_address_munmap() • Use 2M Super Page to reduce the Page table footprint • …… • ptr = (int *)global_addr_mmap(…); • *ptr = 0x5a; • global_addr_munmap(ptr…); • ……
Summary • HPPNET • HyperNIC • HT-enabled • Separate Channel • Global Load/store
Acknowledgements • Our Research is supported by the following organizations
Questions? • Contact with Panyong Zhang<zhangpanyong@ncic.ac.cn>