1 / 13

Grid Services Overview: Certificates, Jobs, File Transfers, MyProxy

This presentation provides an overview of grid services such as obtaining and initializing certificates, submitting fork, serial, and MPI jobs, transferring files with GridFTP, and using MyProxy for certificate storage.

herbertrandall
Download Presentation

Grid Services Overview: Certificates, Jobs, File Transfers, MyProxy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Services @ NERSC demo Shreyas Cholia Open Software and Programming scholia@lbl.gov NERSC User Group Meeting September 19, 2007

  2. Overview • Getting a certificate • Initializing a certificate • Registering certificate with NIM • Submitting a fork job • Submitting a serial job • Submitting an MPI job • Moving files with GridFTP • Third party transfers with GridFTP • Using myproxy to store your certificate NERSC User Group Meeting, September 17, 2007

  3. Getting a Certificate • Instructions at: • http://www.doegrids.org/pages/cert-request.html • Process: • http://pki1.doegrids.org/ca/ • Click on Retrieval->New User. • Fill out form. • You will receive an email once the certificate is ready. Click on the link. • Click on the import button in your browser. • Export/Backup certificate to local file • Convert file to usercert/key.pem using openssl • Renew Annually - repeat above process NERSC User Group Meeting, September 17, 2007

  4. Initializing Your Cert • Load the globus module. One of: module load globus module load osg . /usr/common/osg/0.6.0/setup.sh • Your certificate is in: $HOME/.globus/usercert.pem $HOME/.globus/userkey.pem • Initialize the cert: grid-proxy-init (Creates temporary “proxy” cert valid for 12 hours) • View cert: grid-cert-info grid-proxy-info NERSC User Group Meeting, September 17, 2007

  5. Adding Cert Info to NIM • https://nim.nersc.gov • Click on Grid certificates tab • Click on Add Certificate • Enter information from: grid-cert-info -subject grid-cert-info -issuer NERSC User Group Meeting, September 17, 2007

  6. For More Information • NERSC Website • http://www.nersc.gov/nusers/services/Grid/ • Email • consult@nersc.gov NERSC User Group Meeting, September 17, 2007

  7. Fork jobs • Run a job for immediate execution: # globus-job-run davinci/jobmanager /bin/hostname davinci • Submit a job # globus-job-submit davinci/jobmanager /bin/hostname https://davinci.nersc.gov:60005/6772/1190142432/ • Query submitted job # globus-job-status https://davinci.nersc.gov:60005/6772/1190142432/ DONE • Get output # globus-job-get-outputhttps://davinci.nersc.gov:60005/6772/1190142432/ davinci NERSC User Group Meeting, September 17, 2007

  8. Serial Jobs # globus-job-submit pdsfgrid/jobmanager-sge /bin/date https://pdsfgrid.nersc.gov:60005/6772/1190142432/ # globus-job-get-output https://pdsfgrid.nersc.gov:60005/6772/1190142432/ Tue Sep 18 14:47:06 PDT 2007 NERSC User Group Meeting, September 17, 2007

  9. Parallel Jobs • Create RSL script & (host_xcount=4) (xcount=2) (jobtype=mpi) (directory=/usr/common/homes/s/shreyas/sandbox) (executable=/usr/common/homes/s/shreyas/sandbox/cpi) (stdout=x-gass-cache://$(GLOBUS_GRAM_JOB_CONTACT)stdout anExtraTag) (stderr=x-gass-cache://$(GLOBUS_GRAM_JOB_CONTACT)stderr anExtraTag) NERSC User Group Meeting, September 17, 2007

  10. Parallel Jobs • Submit Job # globusrun -r jacquardgrid.nersc.gov/jobmanager-pbs -f cpi.rsl -b https://jacquardgrid.nersc.gov:60019/7027/1190152305/ • Query Q # globus-job-run jacquardgrid/jobmanager /usr/common/nsg/torque/bin/qstat • Get results # globus-job-get-output https://jacquardgrid.nersc.gov:60019/7027/1190152305/ NERSC User Group Meeting, September 17, 2007

  11. Output PBS Leader node is jaccn203 Job setup time: Tue Sep 18 14:51:47 PDT 2007 Setting up security Job startup at Tue Sep 18 14:51:52 PDT 2007 ---------------------------------------------------------------- pi is approximately 3.1415926535899423, Error is 0.0000000000001492 wall clock time = 0.041841 ---------------------------------------------------------------- Jobs exit status code is Job STDIN/432828.jacin03-m.nersc.gov completed Tue Sep 18 14:51:54 PDT 2007 Submitted by shreyas/shreyas using mpccc Job Limits: ncpus=1,nodes=4:ppn=2,walltime=00:30:00 Job Resources used: cput=00:00:00,mem=5340kb,vmem=33288kb,walltime=00:00:02 Nodes used: jaccn181 jaccn183 jaccn194 jaccn203 Killing any leftover processes... NERSC User Group Meeting, September 17, 2007

  12. File Transfer • Get File from HPSS # globus-url-copy -nodcau \ gsiftp://garchive.nersc.gov//nersc/ccc/shreyas/Animals/gnu file:///home/shreyas/gnu • Third party copy # globus-url-copy gsiftp://pdsfgrid.nersc.gov//home/shreyas/Animals/gnu gsiftp://davinci.nersc.gov//u/s/shreyas/Animals/gnu • Uberftp • Provides interactive access to gridftp server NERSC User Group Meeting, September 17, 2007

  13. MyProxy • If you use several client systems • Copying your .globus directory everywhere can be tedious • Myproxy provides central certificate repository • Storing cert # myproxy-init -s myproxy.nersc.gov (Valid for 1 week. Use -c for non-default lifetime) • Getting cert # myproxy-logon -s myproxy.nersc.gov (Same effect as grid-proxy-init - you have a temporary proxy valid for 12 hours) NERSC User Group Meeting, September 17, 2007

More Related