1 / 42

ABC Code Gridification and Submission: CLI on a UI Machine

This text discusses the gridification and submission process of the ABC code, which requires significant memory and CPU resources. It covers the computational aspects, gridification using P-GRADE Grid Portal, and the CLI interface for execution.

mvillareal
Download Presentation

ABC Code Gridification and Submission: CLI on a UI Machine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Summary ABC reactive scattering code: computational aspects Gridification and submission: CLI on a UI machine Services and web visualization tools for ABC code P-GRADE Grid Portal Conclusions

  2. ABC code • Significant Memory demands (>1GB) • Large request of CPU time (2 days for a set of 30 E) • Each simulation is independent from the other • Must be executed several times for different sets of input parameters (parameter study job) • Executed via the classical command line interface

  3. Prepare for Gridification • Very easy Gridification -Right libraries (Blas, Lapack, libg2c) -Right compiler (ifort, g95) -Right option for compiler (-static, -Bstatic, -static-libcxa) [alex@ui:~/ABC]$ ldd abc not a dynamic executable • Executable: 1.5MB

  4. Prepare JDL file alex@ui:~/ABC$ cat abc.sh #!/bin/bash EXE=abc date echo "*********************************" echo "Executing $EXE" chmod 755 $EXE ls -l time ./$EXE < clhd.d > abc.out echo "*********************************" echo "List directory after Executing" ls -l date exit 0 Type = "Job"; JobType = "Normal"; Executable = "abc.sh"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = {"abc.sh","abc","clhd.d","BW.3p","SW.2p","SW.3p"}; OutputSandbox = {"std.out","std.err","abc.out","clhd.d"};

  5. Step I: VOMS Proxy Create a valid VOMS proxy: alex@ui:~$ glite-voms-proxy-init -voms compchem Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=University of Perugia Dept Maths and CompSci/CN=Alessandro Costantini Cannot find file or dir: /home/alex/.glite/vomses Creating temporary proxy ................................................................... Done Contacting voms.cnaf.infn.it:15003 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=voms.cnaf.infn.it] "compchem" Done Creating proxy ................................................................................... Done Your proxy is valid until Tue Oct 9 00:56:49 2007

  6. Step II: Delegation Delegate your proxy to the WMProxy service: alex@ui:~$ glite-wms-job-delegate-proxy –d delID1234 Connecting to the service https://gliterb00.cnaf.infn.it:7443/glite_wms_wmproxy_server ================== glite-wms-job-delegate-proxy Success ================== Your proxy has been successfully delegated to the WMProxy: https://gliterb00.cnaf.infn.it:7443/glite_wms_wmproxy_server with the delegation identifier: delID1234 ================================================================

  7. Step III: Submission Submit a job to the WMProxy: alex@ui:~$ glite-wms-job-submit –d delID1234 –o idjob test.jdl Connecting to the service https://gliterb00.cnaf.infn.it:7443/glite_wms_wmproxy_server ====================== glite-wms-job-submit Success ====================== The job has been successfully submitted to the WMProxy Your job identifier is: https://gliterb00.cnaf.infn.it:9000/wJbzJtuHdkLXNbIxmLV0Uw The job identifier has been saved in the following file: /home/alex/idjob ==================================================================

  8. Step IV: Monitoring Monitor the status of the job: alex@ui:~$ glite-wms-job-status –i idjob ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://gliterb00.cnaf.infn.it:9000/wJbzJtuHdkLXNbIxmLV0Uw Current Status: Running Status Reason: Job successfully submitted to Globus Destination: ce.grid.unipg.it:2119/jobmanagerlcgpbsshort Submitted: Mon Oct 8 13:03:57 2007 CEST *************************************************************

  9. Step V: Collect the output Retrieve the output: alex@ui:~$ glite-wms-job-output --dir /home/alex/test –i idjob =============================================================== JOB GET OUTPUT OUTCOME Output sandbox files for the job: https://gliterb00.cnaf.infn.it:9000/wJbzJtuHdkLXNbIxmLV0Uw have been successfully retrieved and stored in the directory: /home/alex/test ================================================================

  10. The Gridification of the ABC code • The porting has been carried out using the P-GRADE Grid Portal 2.7 implemented in COMPCHEM • Open source tool (based on GridSphere) • Provides intuitive graphical interface for porting • Collect Grid resources • Observe and supervise the execution

  11. The Gridification of the ABC code • P-GRADE Grid Portal 2.7 • The generic application structure is a workflow • Define parameter study application • Specific graphical web tools can be added: Portlets • Does not require the modification of the original code for execution

  12. P-GRADE Grid Portal 2.7

  13. ABC workflow: explanation of job flow Generator: generates input files with different parameters Executor: executed as many times in parallel as many parameters are generated by “Generator” Collector: collects all output files into a single TAR file

  14. ABC workflow: modify input parameters Double Click on Generator

  15. ABC workflow: modify input parameters Hitting on a key opens the value definition window for that key. Template text with keys. Keys will be replaced with actual numbers by the Generator during the execution of the workflow. All the possible combinations of the replaced template are written into separate files. In the current workflow p_1 parameter defines values for “jmax” and p_2 defines values for “rmax” parameters of ABC. Generator job is a macro processor that generates text files by replacing keys with actual values in a template which is defined by the user.

  16. ABC workflow: modify input parameters In this form you can define actual values for the selected parameter. Using this frame the user can modify the range for each variable in order to define larger parameter sets.

  17. ABC workflow: modify Grid parameters Double Click on directory name

  18. ABC workflow: modify Grid parameters Path syntax is (EGEE): /grid/VONAME/YourUsernameInVO/custom_dir You must change ABC_AGEN_files10 to any other directory name to avoid data-rewrite on the grid

  19. ABC workflow: modify Grid parameters Double Click on Executor

  20. ABC workflow: modify Grid parameters

  21. ABC workflow: input and output files Input files that are the same for every execution of the ABC simulation (ABC binary executable, PESs) The result of the parallel ABC simulation jobs are files that are saved on the Storage Elements. The files are registered in the File Catalog with Logical File Names.

  22. ABC workflow: modify Grid parameters Double Click on file name

  23. ABC workflow: modify Grid parameters Directory path and file name of the output files stored on the SE

  24. ABC job status Status of the submitted workflow The User can download the output files coming from the calculations Status of the Generator Status of each job in the Executor Status of the Collector

  25. Added value on P-GRADE (Prototype) Prototypical visualization tool has been implemented on P-GRADE using Portlet UI components managed and displayed in a web portal 2D-Graph rendering of the Reaction Probabilities of a selected output file for the ABC code Reaction Probabilities Rendering

  26. P-GRADE Portlet: ABC Workflow selection Job selection ABC Reaction Probabilities Graph

  27. Conclusions • Gridification can be a very easy process (it depends on the complexity of your application) • Job submission • CLI (flexible) • Web-Portals (P-GRADE) • -user-friendly interfaces • The ABC workflow can be improved and exported to other applications

  28. ABC workflow: modify Grid parameters Directory path for the STDOUT and STDERROR files

  29. The Molecular Science community and the Grid • The Grid environment allows us to carry out simulation of molecular systems increasing the quality and the quantity of properties investigated • The researcher is able to perform computational campaign: • Massive submission of sequential jobs running on different input datasets • Submission of parallel jobs running on huge pool of nodes • Offers efficient data management facilities • Storage of large amount of data

  30. Technical University of Wien (AT) Johannes Froehlich IChemLab IChemExam Ichem Edu IChemLecture E-Learning Web Services

  31. Molecular Virtual Reality The use of Molecular Virtual Reality (Visualization + Virtual Reality techniques applied at the nanometer level) helps the researchers to better understand the molecular processes We have implemented some virtual monitors of GEMS creating Virtual Worlds based on atoms positions saved during the trajectory calculations (MVR animations) A Virtual World may be created also from a static description of the system in the initial or in the final configuration (mol2,PDB, etc).

  32. CompChem VO in EGEE III In EGEE the Computational Chemistry community is represented by two VO’s: CompChem VO (public domain, shareware software, managed by University of Perugia, Italy) GAUSSIAN VO (commercial software, managed by Cyfronet, PL) A Center of Excellence grouping the labs of both VOs will be established in EGEE III We participated to the MPI working group to promote the use of MPI and parallel software in EGEE (EGEE/int.eu.grid MPI international workshop, Dublin, Dec 11th-12th, 2006) the User Support working group to promote and improve the User Support systems available in EGEE (EGEE Workshop for VO’s, Karlshrue, Germany, March 1st-2nd, 2007 )

  33. http://compchem.unipg.it

  34. The CompChem users

  35. Jobs submitted per week

  36. Hours of CPU per week

  37. ABC workflow: explanation of job flow Represents the input files that differ for the ABC simulations. Represent the input files that are the same for every execution of the ABC simulation (ABC binary executable, PESs) Represent the compressed output files (abc.outs) The result of the parallel ABC simulation jobs are files that are saved on the Storage Elements. The files are registered in the File Catalog with Logical File Names.

  38. ABC code • Quantum mechanical atom-diatom reactive scattering program • Carries out accurate calculations of the quantum S matrix elements • Evaluates reaction probabilities as well as state to state integral and differential cross sections

  39. ABC code II • The program integrates the Schrödinger equation using: • Delves hyperspherical coordinates • A coupled channel method • A time independent technique to integrate the atom-diatom nuclear Schrödinger eq. • [Ĥ-E] ψ = 0

More Related