520 likes | 743 Views
MS Faculty Days 2010, Rome - Sep 16th 2010. Experience many-core architectures at CASPUR: developing biochemistry application on Microsoft HPC 2008 environment. Nico Sanna Nico.Sanna@caspur.it Co-Chair of the HPC Division & Manager of HPC Support Group in (Bio)Chemical-Physics.
E N D
MS Faculty Days 2010, Rome - Sep 16th 2010 Experience many-core architectures at CASPUR: developing biochemistry application on Microsoft HPC 2008 environment Nico Sanna Nico.Sanna@caspur.it Co-Chair of the HPC Division & Manager of HPC Support Group in (Bio)Chemical-Physics
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR
Numeric Simulation S = f (..,..,T,..,..) MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR HPC @ CASPUR Science Database (experiment) Technology
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR HPC @ CASPUR Computational fields • Computational Physics • Computational Chemistry • Computational Biology • Computational Fluid-Dynamics • Applied Mathematics • Computational Material Science • ………
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR HPC Computational (bio)-chemical-physics Quantum Chemistry • WaveFunction Methods • Density Functional (DFT) Methods Molecular Dynamics • Classical MD • Molecular Modeling Atomic & Molecular Physics • E-mol scattering • DNA Radiation Damage Data mining / warehousing • Genome/Protein databases (GCG/EMBOSS/user-db) • Microarray Experiment databases
CASPUR Matrix cluster 15.0 TFLOPS of sustained performance MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR 2.2 Kcores AMD 23xx @ 2.1 GHz 6 TB RAM 200 TB IB storage sustainable green performance
CASPUR Jazz Fermi GPU cluster 14.3 TFLOPS Peak performance MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR 192 cores Intel X5650 @ 2.67 GHz 14336 cores on 32 Fermi C2050 GPUs 1 TB RAM 200 TB IB storage Peak green performance
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Beyond (… with) multi-core? Many-core computing could be an alternative… Accelerating co-processors become to be viable solutions to intensive computing Exploit GPU (&FPGA) computing at its best
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Beyond (… with) multi-core? Reproduced w/ permission of Prof. E. Stahlberg
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR CASPUR@XLrate project
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR CASPUR@XLrate
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR CASPUR@XLrate
CASPUR: XLRATE environment MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Linux NFS server-HA 192.168.192.2 DNS 192.168.192.3 domain: XLRATE DNS domain: .xlrate.caspur.it network: 193.205.219.x/25 DNS .129 .240 00:17:A4:77:00:08 00:22:19:91:16:23 00:22:19:91:26:1D .246 .247 New HP SE2210 .243 .245 .250 .244 HP BL460c-G1 DELL 2950 DELL M600 DELL M600 HP BL460c-G1 HN LN CN CN LN CN hpchn01 hpcln01 hpccn01 hpccn02 hpcln02 hpcn03 Clear Speed GPU NVIDIA TESLA-S 00:22:19:91:16:25 blade10 00:22:19:91:26:1F 00:17:A4:77:00:0A blade9 .11 .15 .16 .14 .13 .12 network: 192.168.100.x/24 • hpccn01 (DELL 2950) • Quad Core 2,33GHz • HD RAID1: 2x73GB 15RPM • L2 cache: 2x6MB • 2x Broadcom BCM5708C • hpccn02 (HP BL460c-G1) • 2xDual Core Intel Xeon 3.0GHz • 8GB RAM • HD RAID1: 2x73GB 15RPM • 2x Broadcom BCM5708C
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR CASPUR@XLrate
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR CASPUR@XLrate project
e- M+ X rays R• g rays a particles b particles Neutrons Cosmic rays R* Living tissue MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Radiation Damage: effect of the secondary species Ionizing radiation The genotoxic effects, due to various DNA lesions, are not only produced by the direct impact of the initial high energy particles (direct ionization) Role of SECONDARY SPECIES: excited atoms and molecules, radicals, ions, low-energy electrons (LEEs) 1 eV < E < 20 eV (the most abundant, carrying most of the energy of the initial radiation)
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR U
e- SCELib2 + VOLSCAT Cross Sections & Resonances E3 E2 E1 High Intensity Resonances Low Intensity Resonances GTGATAGAG T A G T G A T A G A G
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR The SCELib Package The SCELib package: study of e-/p+ scattering from molecular targets. Properties Library (w/ API) of the molecular target N. Sanna, & F. A. Gianturco, Comp. Phys. Comm. 128(2) (2000) 615 N. Sanna & G. Morelli, Comp. Phys. Comm. 162(1) (2004) 51 N. Sanna, I. Baccarelli & G. Morelli, Comp. Phys. Comm., submitted (2009)
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR The SCE part of SCELib: implementation on the GPUs Multi centered Basis Functions Multi centered Molecular Orbitals Single centered Molecular Orbitals our target in electron-molecule scattering
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR The G80 architecture
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR The G80 architecture
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Measuring intrinsic functions performance Grid 1 1 1 Block 256 2 1 Size = 512 # Iterations = 100000
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Measuring our GTO function performance Grid 1 1 1 Block 256 2 1 Size = 512 # Iterations = 100000 (*) Single process Intel QX7600 quad-core running @ 2.67 GHz DEVICE HOST F1 : (float) CSN[tid] * exp ( -EXP[tid] * RSQ[tid] ) F2 : (float) CSN[tid] * __expf ( -EXP[tid] * RSQ[tid] ) F3 : (float) CSN[tid] fmul __expf ( -EXP[tid] fmul RSQ[tid] )
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Profiling CalcMO() GTO calculation w/ VisualProfiler 0.1 - GPU TIME
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Profiling CalcMO() GTO calculation w/ VisualProfiler 0.1 - SIGNAL
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Current implementation of SCELib 3.0 on GPUs SCE algorithm 1) Primitive GTO mapping 2) MOs Single Center Expansion
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs - optimal parameters (warp size, arrays’ dimension in constant memory). Still work in progress • scalability for CH4 with respect to the number of grid points Execution time (in msec) and speed-up with respect to Host SPP calculation
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs • scalability for CH4 with respect to the number of primitive GTOs G9800 Speed-up with respect to Host SPP calculations
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs • scalability with respect to the type of molecular system
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs Timing (in msec) for individual sections of a SCELib run for SO2 molecule (32 electrons, NOTHRDS=32, NPBSIZE=884), transfer Bandwidth and final Speed-up with respect to Host SPP calculations. AD: Allocated Data (in MB) in the Device Global Memory (A) G9800 GX2; (B) Quadro FX5600; (C) Tesla C1060
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs • scalability with respect to the type of molecular system
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs • scalability with respect to the type of molecular system
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib3.0 performance on GPUs • scalability with respect to the type of molecular system
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib4.0 performance on GPUs • SO2Running on MS HPC 2008 Server. One Tesla C1060 GPU
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib4.0 performance on GPUs • SO2 Running on Linux. One Fermi C2050 GPU
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib4.0 performance on GPUs • CPU scalability. Intel X5650 @ 2.67 GHz
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib4.0 performance on GPUs • CPU scalability. Intel X5650 @ 2.67 GHz
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR SCELib4.0 performance on GPUs • CPU/GPU scalability. Intel X5650 @ 2.67 GHz & C2050 GPU
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Conclusions • GPU computing more than FPGA/FPA • CPU/GPU hybrid architectures are becoming mainstream in HPC • Green Computing & Power Awareness HPC w/ many-core (Matrix = 120 KW / Jazz = 13 KW) • Linux OS is there, but look at MS for low latency & data-intensive applications
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Gaussian 03 performance on CPUs (PGI version) • B3LYP/6-31G** SPE calculation running on AMD/Intel procs
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Gaussian 03 performance on CPUs (PGI version) • B3LYP/6-31G** SPE calculation running on AMD/Intel procs
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Gaussian 03 performance on CPUs (PGI version) • B3LYP/6-31G** SPE calculation running on AMD/Intel procs
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Gaussian 03 performance on CPUs (PGI version) • B3LYP/6-31G** SPE calculation running on AMD/Intel procs
MS Faculty Days 2010 // Rome, Sep 16th 2010 Nico Sanna - CASPUR Gaussian 03 performance on CPUs (PGI V10.8) • B3LYP/6-31G** SPE calculation running on AMD/Intel procs
Amber 11 performance on CPUs/GPU • Nucleosome (25095 atoms) Implicit Solvent
Amber 11 performance on CPUs/GPU • Nucleosome (25095 atoms) Implicit Solvent Nanoseconds/day
Amber 11 performance on CPUs/GPU • FactorIX (90906 atoms) Explicit Solvent
Amber 11 performance on CPUs/GPU • FactorIX (90906 atoms) Explicit Solvent Nanoseconds/day
Amber 11 performance on CPUs/GPU • DHFR (23558 atoms) Explicit Solvent