1 / 18

Metacomputing Within the Cactus Framework

Metacomputing Within the Cactus Framework. Gabrielle Allen, Thomas Radke, Ed Seidel. Albert-Einstein-Institut MPI-Gravitationsphysik. What and why is Cactus? What has Cactus got to do with Globus?. Cactus 4.0. Why Cactus?.

urbain
Download Presentation

Metacomputing Within the Cactus Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Metacomputing Within the Cactus Framework Gabrielle Allen, Thomas Radke, Ed Seidel. Albert-Einstein-Institut MPI-Gravitationsphysik • What and why is Cactus? • What has Cactus got to do with Globus?

  2. Cactus 4.0

  3. Why Cactus? • Numerical relativity has very high computational requirements. Not every group has the resources or desire to develop a 3D code. (Especially IO, elliptic solvers, AMR) • Previous experiences show that even a few people using one code is problematic. Need a structure that is maintainable and collaborative • Scientists like to program in Fortran • Want the ability to make new computational advances instantly and transparently available, without users modifying code

  4. What Is Cactus? • Cactus was developed as a general, computational framework for solving PDEs (originally in numerical relativity and astrophysics) • Modular … for easy development, maintenance and collaborations. Users supply “thorns” which plug-into compact core “flesh” • Configurable … thorns register parameter, variable and scheduling information with “runtime function registry” (RFR). Object-orientated inspired features • Scientist friendly … thorns written in F77, F90, C or C++ • Accessible parallelism … driver layer (thorn) is hidden from physics thorns by a fixed flesh interface

  5. What Is Cactus? • Standard interfaces … interpolation, reduction, IO, coordinates. Actual routines supplied by thorns • Portable … Cray T3E, Origin, NT/Win9*, Linux, O2, Dec Alpha, Exemplar, SP2 • Free … distributed under the GNU GPL. Uses as much free software as possible • Up-to-date … new computational developments and/or thorns immediately available to users (optimisations, AMR, Globus, IO) • Collaborative … thorn structure makes it possible for large number of people to use and development toolkits • New version … Cactus beta-4.0 released 30th August

  6. Cactus 4.0 Credits • Toni Arbona • Carles Bona • Steve Brandt • Bernd Bruegmann • Thomas Dramlitsch • Ed Evans • Carsten Gundlach • Gerd Lanferman • Lars Nerger • Mark Miller • Hisaaki Shinkai • Ryoji Takahashi • Malcolm Tobias • Vision and Motivation • Bernard Schutz • Ed Seidel "the Evangelist" • Wai-Mo Suen • Cactus flesh and design • Gabrielle allen • Tom goodale • Joan massó • Paul walker • Computational toolkit • Flesh authors • Gerd lanferman • Thomas radke • John shalf • Development toolkit • Bernd bruegmann • Manish parashar • Many others • Relativity and astrophysics • Flesh authors • Miguel alcubierre

  7. Full GR Neutron Star Collision With Cactus

  8. Thorn Arrangements

  9. Cactus 4.0 Boundary CartGrid3D WaveToyF77 WaveToyF90 PUGH FLESH (Parameters, Variables, Scheduling) GrACE IOFlexIO IOHDF5

  10. Cactus: Many Developers DAGH/AMR (UTexas) AEI NCSA FlexIO ZIB Wash. U HDF5 NASA SGI Valencia Petsc (Argonne) Globus (Foster) Panda I/O (UIUC CS)

  11. What Has It Got to Do With Globus? • Easy access to available resources • Access to more resources • Einstein equations require extreme memory, speed • Largest supercomputers too small! • Networks very fast! • DFN gigabit testbed: 622 mbits potsdam-berlin-garching, connect multiple supercomputers • Gigabit networking to US possible • Connect workstations to make supercomputer • Acquire resources dynamically during simulation! • Interactive visualization and steering from anywhere • Metacomputing experiments in progress with Cactus+Globus

  12. TIKSLTele Immersion: Collision of Black Holes German research project aimed to exploit the newly installed gigabit testbed Süd+Berlin Project partners • Albert-Einstein-Institut Potsdam • Konrad-Zuse-Institut Berlin • Rechenzentrum Garching Main project goals • Distributed simulations of black hole collisions with Cactus • Remote visualization and application steering with Amira AEI

  13. Running Cactus in a Distributed Environment Using the Globus services to • Locate computing resources via MDS • Authenticate the cactus users (GSS) • Transfer necessary files to remote sites(executable, parameter files) via GASS • Start the Cactus job via GRAM • Do parallel communication and file I/Ousing Nexus MPI and MPI-IO extensions • Access output data via GASS

  14. Computational Needs for 3D Numerical Relativity • Explicit finite difference codes • ~ 104 flops/zone/time step • ~ 100 3D arrays • Require 10003 zones or more • ~1000 gbytes • Double resolution: 8x memory, 16x flops • Tflop, tbyte machine required • Parallel AMR, I/O essential • Etc t=100 t=0 • InitialData: 4 coupled nonlinear elliptics • Time step update • explicit hyperbolic update • also solve elliptics

  15. (A Single) Such Large Scale Computation Requires Incredible Mix of Varied Technologies and Expertise! • Many scientific/engineering components • Formulation of ee’s, “gauge conditions”, equation of state, astrophysics, hydrodynamics, etc • Many numerical algorithm components • Finite differences? Finite elements? Structured meshes? • Hyperbolic equations: implicit vs implicit, shock treatments, dozens of methods (and presently nothing is fully satisfactory!) • Elliptic equations: multigrid, krylov subspace, spectral, preconditioners (elliptics currently require most of the time…) • Mesh refinement? • Many different computational components • Parallelism (HPF, MPI, PVM, ???) • Architecture efficiency (MPP, DSM, vector, NOW, ???) • I/O bottlenecks (generate gigabytes per simulation, checkpointing…) • Visualization of all that comes out!

  16. Distributing Spacetime: SC’97 Intercontinental Metacomputing at Aei/Argonne/Garching/NCSA Immersadesk 512 Node T3E

  17. Metacomputing the Einstein Equations:Connecting T3e’s in Berlin, Garching, San Diego

  18. The Dream: not far away... Physics Module 1 BH Initial Data Cactus/Einstein solver Budding Einstein in Berlin... MPI, MG, AMR, DAGH, Viz, I/O, ... Mass storage Globus Resource Manager Ultra 3000: Whatever-Wherever Garching T3E NCSA Origin 2000 array

More Related