Recent Trends in the Marketplace of High Performance Computing

1. Recent Trends in the Marketplace of High Performance Computing E. Strohmaier, J.J. Dongarra, H. W. Meuer, H. D. Simon In �Parallel Computing� 2005

2. Summary 1970s Vector computers � raw performance Early 1990s Massively parallel processors (MPP) � price/perf ratio Early 2000s SMP Earth Simulator vector system Renewed interest for new HW architectures and new programming paradigms IBM BlueGene/L � example of shifting design focus

3. Capability computing Capability computing: A type of large-scale computing in which one wants to accommodate very large and time consuming computing tasks. This requires that parallel machines or clusters are managed with the highest priority for this type of computing possibly with the consequence that the computing resources in the system are not always used with the greatest efficiency.

4. Capacity computing Capacity computing: A type of large-scale computing in which one wants to use the system (cluster) with the highest possible throughput capacity using the machine resources as efficient as possible. This may have adverse effects on the performance of individual computing tasks while optimizing the overall usage of the system.

5. MPP vs Cluters There are different definitions of MPP. Some consider clusters MPPs.

6. HPC Clusters, Constellations, MPPs, and Future Directionsby Dongarra et al 2003 �Terminology and taxonomies are subjective. No absolute truths exist, and common usage dictates practical utility, even when self-contradictory or incomplete in meaning.� Bell and Gray�s 2002 paper incorporates an assumption that �corrupts the power of our terminology as a tool to represent and differentiate.� Need to degeneralize the applicability of the term �cluster� to a restricted subset of // computers Top-500 classification: Commodity clusters Cluster:

7. Top-500 classification (by Dongarra et al) Commodity clusters: 2 levels of //ism: 1. # nodes connected by the global comm network, where a node contains all of the processor and memory resources of the cluster 2. # processors in each node, usually configured as an SMP Cluster-NOW: dominant mode of //ism is first level Constellations (SMP Cluster): dominant mode of //ism is second level MPPs

8. Revised framework and taxonomy (by Dongarra et al) 4 dominant dimensions to characterizing // computing systems: Clustering: commodity vs monolithic Name space: distributed vs shared vs cache-coherent //ism: multithreading vs vector vs message passing vs systolic vs VLIW vs producer/consumer vs // processes vs � Latency/locality management: caches vs vectors vs multithreaded vs processor in memory vs parcel or message driven split-transaction vs prefetching vs explicit allocation

9. Examples

10. MPPs and Clusters for Scalable Computing by Zhiwei Xu and Kai Hwang MPP refers to a large-scale computer system that uses commodity microprocessors in processing nodes uses physically distributed memory nodes uses custom-designed interconnect with high communication and low latency can be scaled up to hundreds or more processors Examples: Intel ASCI TeraFLOPS, IBM SP2, Cray T3D and T3E (also DSM machines), Intel Paragon

11. 520.426 // Process Arch notesby R. Jenkins MPP characterized by using commercial microprocessors at each node with physically distributed memory across nodes, a high bandwidth communication network with nearly zero latency, and being able to scalable to hundreds and even thousands of processing nodes Asyn execution (vs syn as in SIMD) Treating dist mem as an unshared (vs shared as in DSM) resource Examples: Intel Paragon, TFLOP

12. EE557 Comp Sys Arch notesby Youngdae Kim 2004? MPP A very large-scale comp sys with commodity processing nodes interconnected with a high-speed low-latency interconnect. Memories are physically distributed. Nodes often run a microkernel Contains 1 host monolithic OS There are overlaps among MPPs, clusters, and SMPs.