260 likes | 467 Views
IBM RS/6000 SP POWER3 SMP. Jari Jokinen Pekka Laurila. IBM RS/6000 Series. Series of advanced IBM Unix computers. Available from entry level to high-end machines. Allows many different configurations. Current high-end configurations use POWER3 type processors.
E N D
IBM RS/6000 SP POWER3 SMP Jari Jokinen Pekka Laurila
IBM RS/6000 Series • Series of advanced IBM Unix computers. • Available from entry level to high-end machines. • Allows many different configurations. • Current high-end configurations use POWER3 type processors. • Shared memory MIMD (Multiple Instructions, Multiple Data) architecture.
POWER3 processor • 64-bit PowerPC RISC-processor. • Symmetric multi-processor. • 64-bit addressability and double-word integer operations. • Eight execution units. • High bandwidth memory with four floating point operations per cycle.
POWER3 processor • 200MHz processor achieves: • 623 MFLOPS • 1.1Gbps memory bandwidth • The technology allows processor speeds up to 500MHz. • Allows complex switch-based configurations to be built using POWER processor.
Node architecture • SP POWER3 SMP high node architecture. • Up to 16 POWER3 processors per node with up to 64 GB of memory. • Scalable architecture allowing system from one node up to 512 nodes to be built using same technology.
Node architecture • Up to four processor cards. • Each card has up to four processors. • The Node Controller chips for data have 4GB/s bandwidth per processor and 16GB/s bandwidth to the Active Backplane Planar. • Memory and I/O functions have 16GB/s bandwidth to Planar. • Tree topology inside the node.
Connecting nodes • SP Switch2 is used to connect nodes into a supercomputer. • Hardware + software = Communication Subsystem (CSS) • Communication path, monitoring of the switch hardware, controling the network, error detection and recovery action. • Multistage switching technology
SP Switch2 • 32 links, 16 for nodes and 16 for other switches. • For very big networks, switch boards have to be connected together. • Also available is an 8 node switch board for those who do not need more than 8 nodes.
Topology • 2 to 80 nodes can be connected together with maximum of 5 switch boards using star topology. • Data passes through, at most, two switch boards.
80 to 256 nodes • When more than 80 nodes are required, at least six switch boards must be used. Star topology does not allow this. • Instead, additional boards must be used as intermediate switch boards.
257 to 512 nodes • If more than 257 nodes are required, two frames of switch boards have to be used. • 32 NSBs times 16 ISBs equals 512 nodes.
Operating System • IMB’s own AIX operating system is used in the machines based on this architecture. • 64-bit Unix system. • Each node has its own operating system.
NERSC IBM SP RS/6000 • Uses SP POWER3 SMP architecture. • 208 16-processor nodes where 184 are actual computing nodes, 16 for fileserver (GPFS) with 20 Terabytes of disk space, 3 login nodes, 1 network node and 4 service nodes. • 375MHz POWER3 processors.
NERSC IBM SP RS/6000 • Currently fifth on the top500 supercomputer list. • Usage restricted to scientific projects funded by Department of Energy.
NERSC IBM SP RS/6000 • Peak performance of 4.5TFLOPS • Actual benchmarked performance: • With MPI, latency between nodes is 17 microseconds, within nodes it is 9 microseconds. Bandwidth between nodes is 60-400MB/s (16-2 nodes), within nodes 250-500MB/s (16-2 processors).
NERSC IBM SP RS/6000 • MPI programming environment, C/C++, FORTRAN • Other compilers: • OpenMP, Java for AIX, Assembler for AIX • Tools: • Parallel Operating Environment (POE) for AIX, LoadLeveler for job queueing, TotalView for debugging, HPMcount for monitoring hardware and Xprofiler for profiling serial and parallel applications. • Many mathematical libraries.