SC11 Jen-Hsun Huang Keynote

1. Exascale An Innovator’s Dilemma Jen-Hsun Huang, CEO SC11, Seattle, Washington | Nov. 15, 2011 NVIDIA Confidential

2. "Generally, disruptive innovations were technologically straightforward, consisting of off-the-shelf components put together in a product architecture that was often simpler than prior approaches. They offered less of what customers in established markets wanted and so could rarely be initially employed there. They offered a different package of attributes valued only in emerging markets remote from, and unimportant to, the mainstream.”

3. NVIDIA Confidential

4. A History of Amazing Advances ZETTAFLOPS EXAFLOPS PETAFLOPS Cray XT-5 “Jaguar” 1.8 PF, 7 MW TERAFLOPS Cray T3E-1200 0.9 TF GIGAFLOPS Cray Y-MP8 2.7 GF, 0.15 MW 1988 1998 2009

5. Advances with Dennard Scaling ZETTAFLOPS ½ L every 4 yrs 8X – same P 1.68X per/W CAGR EXAFLOPS PETAFLOPS Cray XT-5 “Jaguar” 1.8 PF, 7 MW TERAFLOPS Cray T3E-1200 0.9 TF GIGAFLOPS Cray Y-MP8 2.7 GF, 0.15 MW 1988 1998 2009

6. End of Dennard Scaling ZETTAFLOPS EXAFLOPS PETAFLOPS Cray XT-5 “Jaguar” 1.8 PF, 7 MW ½ L every 4 yrs 8X @ 4X P 1.19X per/W CAGR TERAFLOPS Cray T3E-1200 0.9 TF GIGAFLOPS Cray Y-MP8 2.7 GF, 0.15 MW 1988 1998 2009

7. Supercomputing is Power Limited ZETTAFLOPS EXAFLOPS 1 EF, 20 MW 100 PF, 20 MW 70 PF, 20 MW CPU-only “Titan” 6 PF, 8.6 MW PETAFLOPS Cray XT-5 “Jaguar” 1.8 PF, 7 MW TERAFLOPS Cray T3E-1200 0.9 TF GIGAFLOPS Cray Y-MP8 2.7 GF, 0.15 MW 1988 1998 2009 2012 2019 2022 2035

8. CPUs Fast But Complex Optimized for single-threaded performance ~50X energy to schedule instruction than the operation ~20X energy to move data across chip than the calculation

9. Super Efficient Processors Needed Many simple processors with minimal overhead Locality reduces data movement energy Poor single-threaded performance Innovator’s Dilemma!

10. PRINCIPLE #1 Companies depend on customers and investors for resources. Clayton M. Christensen (1997) The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail

11. PRINCIPLE #2 Small markets don’t solve growth needs of large companies. Clayton M. Christensen (1997) The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail

13. GPU Computing “New Market” Disruption

14. Disruptive technologies underperform established products in mainstream markets. Cheaper, smaller, and frequently more convenient. Clayton M. Christensen (1997) The Innovator’s Dilemma: When New Technologies Cause Great Firms to Fail

15. Nagasaki University Professor Tsuyoshi Hamada’s Homemade supercomputer

16. GPU Computing GPUs with CUDA >350,000,000 Toolkit Downloads >1,000,000 Active Developers >120,000 Universities Teaching GPU Computing >475 100% HPC OEMs offer GPU Clusters

17. World’s First Whole H1N1 Virus Simulation

18. Directives Lifecycles of fish in Australia Stars and galaxies 12.5B years ago Neural networks in a self-learning robot University of Melbourne University of Groningen The University of Plymouth 65xin 2 Days 5.6xin 5 Days 4.7xin 4 Hours

21. The Road to Exascale ZETTAFLOPS 1 EF, 20 MW 1 EF, 20 MW GPU-Accelerated “Titan” 20 PF, 8.6 MW 100 PF, 20 MW 70 PF, 20 MW CPU-only “Titan” 6 PF, 8.6 MW PETAFLOPS 2012 2019 2022 2035

22. The Road to Exascale ZETTAFLOPS 1 EF, 20 MW 1 EF, 20 MW 1 EF, 20 MW GPU-Accelerated “Titan” 20 PF, 8.6 MW 100 PF, 20 MW 70 PF, 20 MW CPU-only “Titan” 6 PF, 8.6 MW PETAFLOPS 2012 2019 2022 2035

23. The Road to Exascale ZETTAFLOPS 100 EF, 20 MW 1 EF, 20 MW 1 EF, 20 MW GPU-Accelerated “Titan” 20 PF, 8.6 MW 100 PF, 20 MW 70 PF, 20 MW CPU-only “Titan” 6 PF, 8.6 MW PETAFLOPS 2012 2019 2022 2035

24. Huge Markets Support GPU Workstation 5M Console 20M PCs 400M Mobile 1,000M

25. “Teraflops” 5 Watts ASCI Red @ Sandia Labs 1997 2019

26. DEMO: BATTLEFIELD 3

27. “Tens” of Teraflops 100 Watts Red Storm @ Sandia Labs 2004 2019

28. DEMO: ASSASSIN’S CREED

29. “Hundreds” of Teraflops 1000 Watts Blue Gene @ LLNL 2006 2019

30. DEMO: MAYA

31. DEMO: FUMEFX

32. One Disruption Enables Another

35. ‘Super’ Computing From Super Computers to Super Phones