HPC USER FORUM Stuttgart Germany October 2010 Merle Giles Private Sector Program & Economic Development mgiles@ncs

1. HPC USER FORUMStuttgart Germany October 2010Merle GilesPrivate Sector Program & Economic Developmentmgiles@ncsa.illinois.edu

2. Basic & Applied Research College of Engineering CSL Digital Computer Lab NCSA ECE Biotechnology Center Computer Science Institute for Genomic Biology Beckman Institute Imaginations unbound

3. NCSA Bridges Basic Research and Commercialization with Application Over the horizon development … Phase 2 Design/ Development Phase 3 Prototyping Phase 4 Production/ Deployment Phase 0 Concept/ Vision Phase 1 Feasibility Product Life Cycle Applied Prototyping & Development Optimization & Robustification Commercialization & Production (.com or .org) Theoretical & Basic Research NCSA Bridges the Gap BETWEEN Basic Research & Commercialization Application Universities & Labs Private Industry Imaginations unbound

4. Value Creation and Economic Development • 3D virtual prototyping at NCSA => Caterpillar’s Global Simulation Center in Champaign • Full-scale simulation of cell tower activity • Reduced reliance on wet labs thru computation • Cluster design and architecture • HPC-capable fast-network hard drives • World’s first Internet GUI interface • Prototyping Windows® HPC Operating System Imaginations unbound

5. Value Creation and Economic Development • The HDF Group (pdf-equivalent for data) • Linux OS made standard for HPC • Apache server software • Telnet remote access • R-systems hosts Wolfram Alpha • Music data analytics at One Llama • River Glass spinout – recent Boeing buyout • $1 Trillion per founder Larry Smarr TOTAL $$ Imaginations unbound

6. Industry Partners over time Imaginations unbound

7. CURRENT PARTNERS Imaginations unbound

8. CLASSICAL HPC MISSION • Historically, HPC has focused on science discovery • Economic value has also been achieved in HPC derivatives • Industrial value keys on discovery and optimization • Increasingly, industry brings world-class problems Imaginations unbound

9. Tomorrow’s Industrial Challenge FIDELITY today TIME tomorrow’s HPC Challenge COMPLEXITY Imaginations unbound

10. Discovery is No Longer Sufficient FIDELITY today TIME Industrial Design Cycle tomorrow’s HPC Challenge COMPLEXITY Imaginations unbound

11. Product Lifecycle Management Proves Why HPC||||||| Bills Of Materials CAD/CAE Information Platform PLM Product Specification Product Design Technical Regs and Standards Product Modeling Production Planning Manufacturing Assembly Quality Assurance Shipping Process Platform data integration Supply Chain Mgmt Cost Accounting Imaginations unbound

12. Forbes Magazine: Publisher Rich KarlgaardDigital Rules, September 13, 2010 Smart-aggregation rules: Some forms of content will always need human curating. Dumb-aggregation rules: And some forms of content won’t. The trick is to figure out where algorithms beat humans, and vice versa. Imaginations unbound

13. Competitive Advantage needs Human Expertise Materials, fluids, mechanical, aerothermal, electrical, etc. Petascale, ISV code, data management, architecture, etc. Parallelization, scaling, multiphysics, algorithms, etc. Strategy, process, services, partnerships, etc. Imaginations unbound

14. Headlines • GERMANY Trade & Invest – Partnership is the key to country’s thriving R&D landscape. • IBM Smarter Planet – Thanks to pervasive instrumentation and global interconnection, we are now capturing data in unprecedented volume and variety. • IBM Smarter Planet – World’s network traffic will soon total more than half a zettabyte. • WSJ Steve Conway – “There is growing recognition of the close link between supercomputing and scientific advancement as well as industrial competitiveness.” • Forbes – Consumer technology is now ahead of most industrial technology. Imaginations unbound

15. EXTREMECOMPUTING

16. New Performance Driver NCSA’s Blue Waters is the first open-access system tasked to achieve ≥ 1 petaflop/s on real applications. Imaginations unbound

17. Guess What This Is ? From 1956 . . . Imaginations unbound

18. Guess What This Is ? From 1956 . . . A hard disk drive with 5 MB storage Imaginations unbound

19. Leading-Edge Collaboration Yotta- 1024 Zetta- 1021 Exa- 1018 Peta- 1015 (Bits, Bytes, Flop/s) Tera- 1012 Giga- 109 Mega- 106 Imaginations unbound

20. Blue Waters Expected to Beat 2008’s TOP500® COMBINED! Imaginations unbound

21. U.S. Leadership Computing Programs • U.S. Department of Energy • Oak Ridge National Laboratory: Jaguar + Follow-on • Argonne National Laboratory: Intrepid + Follow-on • Lawrence Livermore National Lab: Dawn + Sequoia • National Science Foundation • University of Illinois/NCSA: Blue Waters • NASA • Ames Research Center: Pleiades Imaginations unbound

22. Petaflop/s Comparison Imaginations unbound

23. Machine Comparison Imaginations unbound

24. Integrated/Scalable System Blue Waters will be the most powerful computer in the world for scientific research when it comes on line in 2011. Blue Waters ~10 PF Peak ~1 PF sustained >300,000 cores ~1.2 PB of memory >18 PB of disk storage 500 PB of archival storage ≥100 Gbps connectivity Blue Waters Building Block 32 IH server nodes 256 TF (peak) 32 TB memory 128 TB/s memory bw 4 Storage systems (>500 TB) 10 Tape drive connections IH Server Node 8 QCM’s (256 cores) 8 TF (peak) 1 TB memory 4 TB/s memory bw 8 Hub chips Power supplies PCIe slots Fully water cooled Quad-chip Module 4 Power7 chips 128 GB memory 512 GB/s memory bw 1 TF (peak) Hub Chip 1.128 TB/s bw Blue Waters is built from components that can be used to build other systems with a wide range of capabilities— from servers to beyond Blue Waters. Power7 Chip 8 cores, 32 threads L1, L2, L3 cache (32 MB) Up to 256 GF (peak) 128 Gb/s memory bw Imaginations unbound

25. IBM P7IH Supernode = 128 CPUs/1024 cores Imaginations unbound

26. Data Center in a Rack • BPA • 200 to 480Vac • 370 to 575Vdc • Redundant Power • Direct Site Power Feed • PDU Elimination • Rack • 990.6w x 1828.8d x 2108.2 • 39”w x 72”d x 83”h • ~2948kg (~6500lbs) • Storage Unit • 4U • 0-6 / Rack • Up To 384 SFF DASD/Unit • File System • Data Center In a Rack • Compute • Storage • Switch • 100% Cooling • PDU Eliminated • Input: 8 Water Lines, 4 Power Cords • Out: ~100TFLOPs / 24.6TB / 153.5TB • 192 PCI-e 16x / 12 PCI-e 8x • CECs • 2U • 1-12 CECs/Rack • 256 Cores • 128 SN DIMM Slots / CEC • 8,16, (32) GB DIMMs • 17 PCI-e Slots • Imbedded Switch • Redundant DCA • NW Fabric • Up to: 3072 cores, 24.6TB • (49.2TB) • WCU • Facility Water Input • 100% Heat to Water • Redundant Cooling • CRAH Eliminated

27. Diverse Large Scale Computational Science Imaginations unbound

28. Programming Environment IO Model: Global, Parallel shared file system (>10 PB) and archival storage (GPFS/HPSS) MPI I/O Environment: Traditional (command line), Eclipse IDE (application development, debugging, performance tuning, job and workflow management) Languages: C/C++, Fortran (77-2008 including CAF), UPC Performance tuning: HPC and HPCS toolkits, open source tools Resource manager: Batch and interactive access Parallel debugging at full scale Full – featured OS(AIX or Linux), Sockets, threads, shared memory, checkpoint/restart Libraries: MASS, ESSL, PESSL, PETSc, visualization… Programming Models: MPI/MP2, OpenMP, PGAS, Charm++, Cactus Low-level communications API supporting active messages (LAPI) Hardware Multicore POWER7 processor with Simultaneous MultiThreading (SMT) and Vector MultiMedia Extensions (VSX) Private L1, L2 cache per core, shared L3 cache per chip High-Performance, low-latency interconnect supporting RDMA Imaginations unbound

29. Blue Waters Benchmark Codes NSF Challenge: ≥1 Sustained petaflop/s Photos courtesy of NERSC, UIUC, IBM Imaginations unbound

30. Path to Petascale Imaginations unbound

31. 2 Paths to Blue Waters Imaginations unbound

32. National Petascale Computing Facility$72.5M, 25MW, LEED Gold+Military-grade securityNon-classified88,000 ft2 Imaginations unbound

33. THANK YOU!http://industry.ncsa.illinois.eduwww.ncsa.illinois.edu/BlueWaters