1 / 43

Cactus in GrADS (HFA)

Cactus in GrADS (HFA). Ian Foster Dave Angulo, Matei Ripeanu, Michael Russell. Presentation Outline. Introduction to Cactus Cactus Applications Cactus Architecture Cactus Worm Thorn Tequila Thorn (Cactus in GrADs) Tequila Architectur Issues. What is Cactus?.

peers
Download Presentation

Cactus in GrADS (HFA)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cactus in GrADS (HFA) Ian Foster Dave Angulo, Matei Ripeanu, Michael Russell

  2. Presentation Outline • Introduction to Cactus • Cactus Applications • Cactus Architecture • Cactus Worm Thorn • Tequila Thorn (Cactus in GrADs) • Tequila Architectur • Issues

  3. What is Cactus? Cactus is a freely available, modular, portable and manageable environment for collaboratively developing parallel, high-performance multidimensional simulations

  4. Example Cactus Output Example output from Numerical Relativity Simulations

  5. Cactus Applications • Application Thorns are Astrophysics applications • Calculate Schwartzchild Event Horizons for colliding black holes

  6. Cactus Applications (cont.) • Candidate apps are Elliptical Solver or BenchADM • Abstract Topologies are simple 3D Grid

  7. Cactus Applications (cont.) • Applications can easily be “linked” in with the other thorns used as tools. • Application Thorns are just selected and run with the other selected thorns

  8. Cactus Architecture Cactus Thorns Computational Toolkit Toolkit Toolkit Flesh Make Configure CST Operating Systems Irix SuperUX Linux Unicos HP-UX Solaris OSF NT AIX

  9. Building an executable Cactus Source Configuration Thorns Flesh • Compiler options • Tool options • MPI options • HDF5 options IOBasic IOASCII WaveToy LDAP Worm … Cactus Model (cont.)

  10. Parameter File • Specify which thorns to activate • Specify global parameters • Specify restricted parameters • Specify private parameters Running Cactus

  11. Cactus “flesh” internals Cactus Application Thorn(s) Other Cactus Thorn(s) Cactus “Worm” Thorn Cactus model • This is the currently working Cactus application framework that we will modify

  12. Worm Thorn Functions • Initiates moving to new resource when scheduled time is exhausted • Contacts IS to get a new node to run on • Checkpoints application • Restarts application on the new node • Runs on single node only

  13. Cactus “flesh” internals Cactus Application Thorn(s) Other Cactus Thorn(s) Cactus “Tequila” Thorn GrADS Cactus Model • We will start with “Worm” thorn code to make new “Tequila” thorn (Apotheosized Cactus Worm).

  14. Tequila thorn functions • Receives event (generated by user) to initiate adapting resources. • Contacts ResourceSelector to get new bag of resources • Checkpoints application • Restarts application on the new resources

  15. Events • Events that cause the user to want to adapt resources: • User changes parameters during runtime that requires additional resources • Example: starting an analysis routine • Example: running an event horizon finder • User specifies that performance is not meeting expectations

  16. Future Events • Possible Future Plans for automatic resource adapting: • User changes parameters during runtime that requires additional resources • Contract violations fire similar events • we were wrong first time • resources get overloaded • more (or fewer) (or different) processors appear • distribution changed • resolution changed

  17. Tequila thorn contacts ResourceSelector • ResourceSelector must be set up as service • Tequila thorn sends request for new bag of resources • ResourceSelector responds with the new bag

  18. Request and Response • The Request to the ResourceSelector will be stored in the InformationService • Only the pointer to the data in the IS will be passed to the ResourceSelector • The Response from the ResourceSelector will also be stored in the IS • Only the pointer to the data in the IS will be passed back.

  19. Resource Selector Information Service Cactus Tequila Thorn Tequila communication overview

  20. Grads Communi- cation library Toolkit Cactus Architecture in GrADS Cactus Thorns Computational Toolkit Toolkit Flesh Make Configure CST Operating Systems Irix SuperUX Linux Unicos HP-UX Solaris OSF NT AIX

  21. Open Issues • How does Contract Monitor fit into architecture? • How does PPS fit into architecture? • How does COP and Aplication Launcher fit into architecture (Cactus has its own launcher and compiles its own code)? • How does Pablo fit into architecture (Which thorns are monitored, is flesh monitored)?

  22. End of Presentation

  23. Slides Explaining Communication Details • ********************

  24. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 1 • Event sent to Tequila thorn requesting restart

  25. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 2 • Tequila store AART in IS

  26. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 3 • Tequila sends request to ResourceSelector passing pointer to data in IS

  27. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 4 • ResourceSelector retrieves AART from IS

  28. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 5 • ResourceSelector stores bag of resources (in AART) in IS

  29. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 6 • ResourceSelector responds to Tequila passing pointer to data in IS

  30. Resource Selector Cactus Tqeuila Thorn Information Service Communication Details step 7 • Tequila retrieves AART with new bag of resources from IS

  31. Requirements • Using the IS for communication adds overhead. • Why do this? • GrADS requirement 1: do some things (e.g. compile) at one time and have the results stored in a persistent storage area. Pick these stored results up later and complete other phases.

  32. Requirements (cont.) • GrADS requirement 2: Application people want to be able to allow users to manually interact in any of the "module interfaces." Tequila allows this to be done with a web client.

  33. Slide Explaining Parallelism in Cactus • ***************

  34. Parallelism in Cactus • Cactus is designed around a distributed memory model. Each thorn is passed a section of the global grid. • The actual parallel driver (implemented in a thorn) can use whatever method it likes to decompose the grid across processors and exchange ghost zone information - each thorn is presented with a standard interface, independent of the driver. • Standard driver distributed with Cactus (PUGH) is for a parallel unigrid and uses MPI for the communication layer • PUGH can do custom processor decomposition and static load balancing

  35. Slide with Alternate Tequila Architecture • ***************

  36. Sample Tequila Scenario • User asks to run an ADM simulation 400x400x400 for 1000 timesteps in 10s. • Resource selector contacted to obtain virtual machines • Best virtual machine selected based on performance model. • AM starts Cactus on that virtual machine (and monitors execution Contracts?) • User (or application manager) decides that computation advances too slow and decides to search for a better virtual machine • AM finds a better machine, commands the Cactus run to Checkpoint, transfers files and restart Cactus

  37. Slides Explaining Different Tequila Architectures • *********************

  38. Tequila Architecture Choices • Main presentation explained the short term Tequila Architecture • Open issues covered not-yet-resolved architectural choices for longer term integration

  39. 2. Application Manager instructed to spawn new instance Cactus Flesh Cactus Flesh Worm Thorn Worm Thorn GIIS Application Manager 1. Resources Obtained 3. New instance spawned Worm Spawning

  40. Tequila Spawning • The short-term plan is to simply replace the GIIS with the UCSD Resource Selector • Tequila would make the request for new resources to the RS instead of the GIIS

  41. UCSD Resource Selector 2. Application Manager instructed to spawn new instance Cactus Flesh Cactus Flesh Tequila Thorn Tequila Thorn Application Manager 3. New instance spawned Tequila Spawning 1. Resources Obtained

  42. Tequila Spawning • Longer term plan is not yet resolved. • One possibility is to put all grads pieces into Application Manager

  43. Cactus Flesh Cactus Flesh Tequila Thorn Tequila Thorn 3. New instance spawned Application Manager UCSD Resource Selector 1. Application Manager instructed to spawn new instance 2. Resources Obtained Application Manager

More Related