1 / 21

Cactus Tools for the Grid

Cactus Tools for the Grid. Gabrielle Allen Max Planck Institute for Gravitational Physics, (Albert Einstein Institute). Cactus. CACTUS is a freely available, modular, portable and manageable environment for collaboratively developing parallel, high-

rowdy
Download Presentation

Cactus Tools for the Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cactus Tools for the Grid Gabrielle Allen Max Planck Institute for Gravitational Physics, (Albert Einstein Institute)

  2. Cactus CACTUS is a freely available, modular, portable and manageable environment for collaboratively developing parallel, high- performance multi-dimensional simulations THE GRID: Dependable, consistent, pervasive access to high-end resources www.CactusCode.org

  3. What is Cactus • Flesh (ANSI C) provides code infrastructure (parameter, variable, scheduling databases, error handling, APIs, make, parameter parsing, ) • Thorns (F77/F90/C/C++/Java/Perl/Python) are plug-in and swappable modules or collections of subroutines providing both the computational instructructure and the physical application. Well-defined interface through 3 config files • Just about anything can be implemented as a thorn: Driver layer (MPI, PVM, SHMEM, …), Black Hole evolvers, elliptic solvers, reduction operators, interpolators, web servers, grid tools, IO, … • User driven: easy parallelism, no new paradigms, flexible • Collaborative: thorns borrow concepts from OOP, thorns can be shared, lots of collaborative tools • Computational Toolkit: existing thorns for (Parallel) IO, elliptic, MPI unigrid driver, • Integrate other common packages and tools: HDF5, Globus, PETSc, PAPI, Panda, FlexIO, GrACE, Autopilot, LCAVision, OpenDX, Amira, ... • Trivially Grid enabled!

  4. Current Version Cactus 4.0 • Cactus 4.0 beta 1 released September 1999 • Community code: Distributed under GNU GPL • Currently: Cactus 4.0 beta 8 • Supported Architectures: • SGI Origin • SGI 32/64 • Cray T3E • Dec Alpha • Intel Linux IA32/IA64 • Windows NT • HP Exemplar • IBM SP2 • Sun Solaris • Hitachi SR8000-F • NEC SX-5 • Mac Linux • ...

  5. CactusBase Boundary, IOUtil, IOBasic, CartGrid3D, IOASCII, Time CactusBench BenchADM CactusConnect HTTPD, HTTPDExtra CactusExample WaveToy1DF77, WaveToy2DF77 CactusElliptic EllBase, EllPETSc, EllSOR, EllTest CactusPUGH Interp, PUGH, PUGHSlab, PUGHReduce CactusPUGHIO IOFlexIO, IOHDF5, IsoSurfacer CactusIO IOJpeg CactusTest TestArrays, TestCoordinates, TestInclude1, TestInclude2, TestComplex, TestInterp, TestReduce CactusWave IDScalarWave, IDScalarWaveC, IDScalarWaveCXX, WaveBinarySource, WaveToyC, WaveToyCXX, WaveToyF77, WaveToyF90, WaveToyFreeF90 CactusExternal FlexIO, jpeg6b BetaThorns (In Development) IOStreamedHDF5, Renderer, IOHDF5Util,…, many more Cactus Computational Toolkit: Parallel utilities (thorns) for computational scientist

  6. DLR Astrophysics (Zeus) Numerical Relativity Community AEI Cactus Group (Allen) Cornell Crack prop. San Diego, GMD, Cornell EU Network (Seidel) Berkeley ChemEng (Bishop) Livermore NSF KDI (Suen) Geophysics (Bosl) NASA NS GC SDSS (Szalay) Clemson DFN Gigabit (Seidel) US Grid Forum NCSA, ANL, SDSC “Egrid” Applications “GRADS” (Kennedy, Foster, Dongarra, et al) Microsoft Computational Science Intel Cactus Community

  7. Grid Computing • AEI Numerical Relativity Group has access to high-end resources in over ten centers in Europe/USA • They want: • Bigger simulations, more simulations and faster throughput • Intuitive IO at local workstation • No new systems/techniques to master!! • How to make best use of these resources? • Provide easier access … noone can remember ten usernames, passwords, batch systems, file systems, … great start!!! • Combine resources for larger productions runs (more resolution badly needed!) • Dynamic scenarios … automatically use what is available • Remote/collaborative visualization, steering, monitoring • Many other motivations for Grid computing ...

  8. Grand Picture Viz of data from previous simulations in SF café Remote steering and monitoring from airport Remote Viz in St Louis Remote Viz and steering from Berlin DataGrid/DPSS Downsampling IsoSurfaces http HDF5 T3E: Garching Origin: NCSA Globus Simulations launched from Cactus Portal Grid enabled Cactus runs on distributed machines

  9. Remote Visualization OpenDX OpenDX Amira Contourplots (download) LCAVision IsoSurfaces and Geodesics Grid FunctionsStreaming HDF5 Amira

  10. Remote Steering Any Viz Client HTTP XML Remote Viz data HDF5 Amira Remote Viz data

  11. Viz Client (Amira) HDF5VFD DataGrid (Globus) HTTP DPSS FTP Web Server FTP Server DPSS Server Remote Offline Visualization Viz in Berlin VisualizationClient Downsampling, hyperslabs Only what is needed Remote Data Server 4TB at NCSA

  12. Thorn which allows simulation any to act as its own web server Connect to simulation from any browseranywhere … collaborate Monitor run: parameters, basic visualization, ... Change steerable parameters See running example at www.CactusCode.org Wireless remote viz, monitoring and steering Remote Monitoring: Thorn HTTPD

  13. Open source, (free), multiplatform, large active development community, easy to program Reads HDF5 (Cactus) data from file or remotely streamed from Cactus Simple GUI, select different hyperslabs from 3D data Also support for streamed ASCII data from Cactus Remote Viz: OpenDX

  14. Lightweight, free, isosurface viewer Distributed from Cactus website Displays isosurfaces streamed from Cactus (thorn IsoSurfacer calculates isosurfaces inline with the parallel simulation) Connect to running simulation on any remote machine Steer isosurface value from client Remote Viz: IsoView

  15. Cactus Portal • KDI ASC Project • Technology: Globus, GSI, Java Beans, DHTML, Java CoG, MyProxy, GPDK, TomCat, Stronghold • Locates resources, builds/finds executables, central management of parameter files/job output, submit jobs to local batch queues, tracks active jobs • Submission of distributed runs • Adding more collaborative features (eg group accounting, …) • Accesses the ASC Grid Testbed (SDSC, NCSA, Argonne, ZIB, LRZ, AEI)

  16. New Grid Applications • Dynamic Staging: move to faster/cheaper/bigger machine • “Cactus Worm” • Multiple Universe • create clone to investigate steered parameter (“Cactus Virus”) • Automatic Convergence Testing • from intitial data or initiated during simulation • Look Ahead • spawn off and run coarser resolution to predict likely future • Spawn Independent/Asynchronous Tasks • send to cheaper machine, main simulation carries on • Thorn Profiling • best machine/queue • choose resolution parameters based on queue • ….

  17. New Grid Applications (2) • Dynamic Load Balancing • inhomogeneous loads • multiple grids • Portal • resource choosing • simulation launching • management • Intelligent Parameter Surveys • farm out to different machines • Make use of • Running with management tools such as Condor, Entropia, etc. • Scripting thorns (management, launching new jobs, etc) • Dynamic use of eg MDS for finding available resources

  18. Dynamic Grid Computing Add more resources SDSC Queue time over, find new machine Free CPUs!! RZG SDSC Clone job with steered parameter Calculate/Output Invariants LRZ Archive data Found a horizon, try out excision Calculate/Output Grav. Waves Look for horizon Find best resources Go! NCSA

  19. Users View

  20. Cactus Grid Development • Projects: GrADs, KDI Astrophysics Collaboratory, TIKSL/GriKSL, EGrid/GGF, Globus team, AEI • Grid Application Development Toolkit: Application developer should be able to build simulations with tools that easily enable dynamic grid capabilities e.g.Query information server, Network Monitoring, Simulation instrumenation/Contract specification, Spawning and Migrating, Authentification Server, Data handling, Data transfer, Visualization, Steering, … • Remote visualization, steering, data management • User Portal • Dynamics Applications

  21. More Information ... • Cactus: • Web Site: www.CactusCode.org (Documentation/Tutorials etc) • Cactus Worm: www.CactusCode.org/Development/Egrid.html • Global Grid Forum (Egrid) • www.egrid.org • www.gridforum.org • ASC Portal • www.ascportal.org • TIKSL Gigabit Computing • www.zib.de/Visual/projects/TIKSL/ • Black Holes and Neutron Star: Pictures and Movies • jean-luc.aei.mpg.de • Any questions: cactus@cactuscode.org

More Related