1 / 28

U.S. Department of Energy’s Office of Science

U.S. Department of Energy’s Office of Science. Office of Advanced Scientific Computing Research - Status - Petascale Simulations SciDAC. Walter M. Polansky walt.polansky@science.doe.gov. American Competitiveness Initiative.

sitara
Download Presentation

U.S. Department of Energy’s Office of Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. U.S. Department of Energy’s Office of Science Office of Advanced Scientific Computing Research - Status - Petascale Simulations SciDAC Walter M. Polansky walt.polansky@science.doe.gov

  2. American Competitiveness Initiative President Bush in the President’s State of the Union Address on January 31, 2006 “I propose to double the federal commitment to the most critical basic research program in the physical sciences over the next ten years. This funding will support the work of America’s most creative minds as they explore promising areas such as nanotechnology, supercomputing, and alternative energy sources.” Secretary Bodman commenting on President Bush’s statement “Developing revolutionary, science-driven technology is at the heart of the Department of Energy’s mission. To ensure that America remains at the forefront in an increasingly competitive world, our Department is pursuing transformational new technologies in the cutting-edge scientific fields of the 21st century—areas like nanotechnology, material science, biotechnology, and high-speed computing.” CASC- September 6, 2006

  3. Office of Science: Mission Secretary Samuel Bodman, House Committee on Science Hearing, February 15, 2006 “The Office of Science plays a critical role in ensuring America’s scientific leadership and economic dynamism…” The mission of the Science program is to deliver the discoveries and scientific tools that transform our understanding of energy and matter and advance the national, economic, and energy security of the United States. In support of it’s mission, the Science program has responsibilities in three main areas: selection and management of research; operation of world-class, state-of-the-art scientific facilities; and design and construction of new facilities. “Investment in these facilities is much more than bricks and mortar: it is an investment in discovery, and in the future of our nation.” CASC- September 6, 2006

  4. U.S. Department of Energy Office of Science Facilities for the Future of Science: a 20-Year Outlook

  5. Office of Science -- FY2007 Budget Request- $4.1B -- FY2006 Appropriation- $3.6B

  6. First in Computational Science “Best in class in advancing science and technology through modeling and simulation” Facilities Enabling Technologies Computational Partnerships ASCR Vision CASC- September 6, 2006

  7. ASCR Research Strategy • Applied Mathematics and Computer Science deliver the operating systems, programming models, software tools, and mathematical algorithms and libraries needed for scientists to make effective use of petascale computing. • Scientific Discovery through Advanced Computing (SciDAC) program, will strengthen activities at the software centers initiated in FY 2006 for petascale computing. • SciDAC will foster additional research investments in Applied Mathematics and Computer Science to accelerate efforts in modeling and simulation on the petascale computing facilities. CASC- September 6, 2006

  8. Oak Ridge Leadership Computing Facility Argonne Leadership Computing Facility NERSC at Lawrence Berkeley National Lab ESnet at Lawrence Berkeley National Lab ASCR Facilities CASC- September 6, 2006

  9. Applied Mathematics SBIR/STTR $29.4 $6.3 Computer Science LCF $24.3 $53.7 SciDAC $38.1 NERSC $37.5 Distributed Network Research and Environment Research ESnet Evaluation Prototypes $13.6 $19.0 $13.0 FY 2006 ASCR Budget Distribution CASC- September 6, 2006

  10. FY 2007 ACSR Budget Status $319M $319M $319M $235M Dollars in Millions CASC- September 6, 2006

  11. FY 2007 House Energy and Water Development Appropriations Bill passed both the Appropriations Committee and the House Report language: “ The Committee commends the Office of Science and the Office of Advanced Scientific Computing Research for their efforts to provide cutting-edge capabilities to meet current scientific computational needs, and at the same time to extend the boundaries of that cutting edge into the next generation of high-performance scientific computers and supporting software.” FY 2007 - House CASC- September 6, 2006

  12. FY 2007 Senate Energy and Water Development Appropriations Bill passed the Appropriations Committee Report language “ …The Committee is concerned with the relationship between the Office of Science and the NNSA. As an example, the ASCR strategic plan discusses the need to work with other Federal agencies including several defense agencies, but only discusses in general terms three areas of research where NNSA and the Office of Science cooperated. In the area of basic research,the strategic plan states that it is an area that is `not important enough to justify ASCI investment at this time.'The Committee is also aware that the Office of Science has budgeted $13,000,000 for the DARPA to support a petaflop computer deployment by 2010. The Committee believes this funding would be better spent within the Department to support a petaflop initiative. The Department is directed to divide the funds equally between the Office of Science and the NNSA Advanced Simulation and Computing activities to support development of component architecture for high-performance software and storage” FY 2007 - Senate CASC- September 6, 2006

  13. NERSC – delivery of NERSC-5 in FY 2007 LCF at ORNL – Upgrade Cray XT3 upgrade Now: 25Teraflop  50Teraflop By the end of 2007: 50Teraflop  250Teraflop Acquire 1 Petascale Cray Baker system by end of 2008 LCF at Argonne – Acquire 100 Teraflop IBM Blue Gene/P in FY2007 Upgrade to 250-500 Teraflop IBM Blue Gene/P in 2008 On path to a Petascale IBM Blue Gene /Q by end of the decade Facilities Update CASC- September 6, 2006

  14. Innovative and Novel Computational Impact on Theory and Experiment- INCITE • Initiated in 2004 • Provides Office of Science computing resources to a small number of computationally intensive research projects of large scale, that can make high-impact scientific advances through the use of a large allocation of computer time and data storage • Open to national and international researchers, including industry • No requirement of DOE Office of Science funding • Peer-reviewed • 2004 Awards: 4.9 Million processor hours at NERSC awarded to three projects • 2005 Awards: 6.5 Million processor hours at NERSC awarded to three projects CASC- September 6, 2006

  15. Expanded to include SC high end computing resources at PNNL, ORNL and ANL in addition to LBNL and multiple year requests. Received 43 proposals requesting over 95 million processor hours. 60% from Universities 40% had funding from other federal research agencies 15 Awards for over 18.2 million processor hours INCITE 2006 Wired Magazine, August, 2006, pg. 42 CASC- September 6, 2006

  16. Expanded in 2007 to include 80% of resources at Leadership Computing Facilities in addition to 10% of NERSC and 5% of PNNL Call issued July 27, 2006 Proposals due September 15, 2006 Award announcements in mid-December INCITE 2007 http://hpc.science.doe.gov CASC- September 6, 2006

  17. Structure -- Scientific Application Areas - Breakthrough scientific advances through computer modeling and simulation that are impossible using theoretical and experimental studies or improve experimental science. -- Centers for Enabling Technology (CETs)- provide the essential computing and communications infrastructure to support SciDAC applications -- Institutes- University-led centers of excellence; address specific software issues; Engage a broader community of scientists in modeling and simulations. Scientific Discoverythrough Advanced Computing www.SciDAC.gov CASC- September 6, 2006

  18. Path to Petascale Scientific Discovery • Accelerator science and simulation - Astrophysics • Climate modeling and simulation - Computational Biology • Fusion science - High-energy physics • Petabyte high-energy/nuclear physics - Materials science and chemistry • Nuclear physics - QCD • Radiation transport - Turbulence • - Groundwater reactive transport modeling and simulation Applications SciDAC-2 • Centers for Enabling Technology • Scientific Applications Partnerships - Institutes (University-lead) INCITE NERSC Allocation Computing/ Networking CASC- September 6, 2006

  19. … University-led centers of excellence that focus on major software issues through a range of collaborative research interactions. Develop, test, maintain, and support optimal algorithms, programming environments, systems software and tools, and applications software. Focus on a single general method or technique. Focal point for bringing together a critical mass of experts from multiple disciplines to focus on key problems in a particular area of enabling technologies. Forge relationships between experts in software development, scientific application domains, high performance computing, and industrial partners. Reach out to engage a broader community of scientists in the activities of scientific discovery through advanced computation and collaboration. Incorporate training and outreach in high performance computing topics, including for graduate students and postdocs. SciDAC-2 Institutes… CASC- September 6, 2006

  20. … provide the essential computing and communications infrastructure to support SciDAC applications; multi-discipline approach with activities in: Algorithms, methods, and libraries. Program development environments and tools -- terascale and petascale program development and tools provide maximum ease-of-use to scientific end users. Systems software that provides system stability and functionality needed by users for tera- to peta- scale simulations. Visualization and data management systems. CETs work directly with applications on: Development and application of computing systems software that allows scientific simulation codes to take full advantage of the extraordinary capabilities of terascale and petascale computers. Ensuring that the most critical computer science and applied mathematics issues are addressed in a timely and comprehensive fashion. Addressing all aspects of the successful research software lifecycle including transition of a research code into a robust production code and long term software evolution and maintenance and end user support. Centers for Enabling Technologies… (CETs) CASC- September 6, 2006

  21. SciDAC Accomplishments • To better understand combustion which provides 80% of the energy used in the U.S., SciDAC teams created first laboratory-scale flame simulation in three dimensions • Magnetic fusion scientists and applied mathematicians simulated techniques for re-fueling fusion reactors • Teams developed new methods for simulating improvements in future particle accelerators • Partnerships improved effectiveness of scientific applications codes between 275% to over 10,000% Hydroxyl radical in a turbulent jet flame CASC- September 6, 2006

  22. BES BER FES HEP NP Advanced Fuel Cycles NE Neutron Transport EM NNSA Industry The Power and Evolution of PartnershipsToday and Tomorrow Diesel Combustion Ultrafast Science Methane Flames Clouds Simulation of Microbes GCM Performance Full Simulation of Burning Plasma Plasma Core Increased Accuracy Elements beyond Fe LCLS System Design Weak Matrix Elements Performance of Supernova codes Accelerator Design ASCR Groundwater Transport Libraries & Algorithms Science Based Materials Improvements in Mesh Generation Virtual Prototypes CASC- September 6, 2006

  23. Road to Petascale Computing: Combustion Direct Numerical Simulation of Turbulent Combustion, a collaborative effort between Sandia National Laboratories, U. of Wisconsin, U. of Michigan, U. of Maryland, U.C. Davis, and Princeton U., has been focusing on advancing science for improved fuel efficiency of gas turbines and technologies for transportation. Petascale computing will enable fundamental understanding of turbulence-chemistry interactions, e.g. multi-stage auto-ignition and flame stabilization in compression ignition engine environments, in regimes relevant to real devices. This large turbulent CO/H2 jet flame was run on the CrayX1E at ORNL LCF. (Credit for the image is to Kwan-Liu Ma, Hongfeng Yu, and Hiroshi Akiba of UC Davis.) A 50% increase in efficiency of automobiles could save 21% of U.S. oil consumption for transportation CASC- September 6, 2006

  24. Turbulence Volume rendering of scalar dissipation rates in a temporal plane jet flame. High values are shown in red, medium in yellow, and lower values in blue. (Credit: Jackie Chen/SNL) CASC- September 6, 2006

  25. Road to Petascale Computing: Fusion Center for Simulation of Wave Interaction with Magneto-hydrodynamics, a collaboration of PPPL, ORNL, MIT, Columbia U., Indiana U., Wisconsin U., etc., is a project for very large plasma wave simulation for fusion-based research. Using 4,096 processors of Cray XT3 at LCF (ORNL), it is the largest, most-detailed simulation ever done of plasma control waves in a tokamak, the reactor that will eventually form the core of the multinational ITER reactor. CASC- September 6, 2006

  26. Road to Petascale Computing: Astrophysics A multi-institution project for large-scale simulation of Neutrino-driven Supernova moves forward to discover a new instability that is important to core collapse supernovae – the supernova shock wave instability (SASI) and a plausible mechanism for generating the initial neutron star spin required to give birth to pulsars. CASC- September 6, 2006

  27. Road to Petascale Computing:Modeling for Reactor Design Researchers at Argonne National Laboratory have recently developed a Portable, Extensive, Toolkit for Scientific (PETSc) numerical library used in dozen of scientific applications world-wide. PETSc encapsulates the complexity of underlying parallel algorithms and presents them in terms of numerical abstractions familiar to scientists. It has been used to extend fully compressible flow codes, most notably the pressure-oriented implicit continuous Eulerian (PCICE) scheme developed at Idaho National Engineering Laboratory. CASC- September 6, 2006

  28. Road to PetascaleComputing:Modeling the Interaction b/w Fluids & Embedded Solid Particles Researchers at Lawrence Livermore National Laboratory have recently developed a suite of fast and scalable adaptive algorithms for modeling shocks and detonations interacting with movable rigid bodies in a fluid. Accurate 2D simulations tax existing computer resources; extending to well-resolved 3D simulations will require peta-scale resources. This new algorithm combines efficient local representations of the embedded bodies with moving geometry and adaptive mesh refinement. Shock hitting several movable rigid bodies using a Godunov method with adaptive mesh refinement; density and refinement grids are shown The same fluid solver can model detonations using an ignition and growth model and JWL equation of state. This mock Schlieren shows a detonation interacting with solid bodies. (Credit for the images is to Bill Henshaw (PI) and Kyle Chand, LLNL) CASC- September 6, 2006

More Related