1 / 23

Getting Started on Emerald

Getting Started on Emerald. Research Computing Group. Course Outline. What is Emerald? Logging into Emerald File manipulation on Emerald Submitting jobs on Emerald Interactive programs on Emerald. Help Documentation. Getting Started on Emerald http://help.unc.edu/6020

mio
Download Presentation

Getting Started on Emerald

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Getting Started on Emerald Research Computing Group

  2. Course Outline • What is Emerald? • Logging into Emerald • File manipulation on Emerald • Submitting jobs on Emerald • Interactive programs on Emerald

  3. Help Documentation • Getting Started on Emerald • http://help.unc.edu/6020 • General overview of Emerald for range of users • Short Course – Getting Started on Emerald • http://help.unc.edu/6479 • Detailed notes for beginning Emerald users

  4. What is Emerald? • 352-processor Linux Cluster • Maintained by Research Computing Group • Appropriate for all users regardless of expertise level • Other Servers: • Cedar/Cypress (128-processor SGI/Altix) • Topsail (4160-processor Dell Linux Cluster) • Mass Storage • Account access

  5. Advantages of Using Emerald • High performance • Large capacity • Parallel processing • Many available software packages • Variety of compiling options

  6. Emerald Linux Cluster

  7. CPU CPU CPU CPU M M M M NETWORK Distributed vs. Shared Memory CPU CPU CPU CPU BUS MEMORY Shared memory - single address space. All processors have access to a pool of shared memory. (examples: Yatta, Cedar/Cypress) Methods of memory access : Bus and Crossbar Distributed memory - each processor has it’s own local memory. Must do message passing to exchange data between processors. (examples: Emerald, Topsail)

  8. Logging Into Emerald • UNIX/Linux/OSX • ssh my_onyen@emerald.unc.edu • Windows: SSH Secure Shell • Setting up a Profile for Emerald • Forwarding X11 packets

  9. Home and Work Directories on Emerald • Home Directory • /afs/isis/home/m/y/my_onyen/ • 250MB quota • ~/private/ • Files backed up daily [ ~/OldFiles] • Space quota/usage in Home Directory: • fslq • Work Directory • /netscr/my_onyen/ • No space limit but periodically cleaned

  10. File Manipulation Commands • SSH Secure File Transfer • Copy files • cp command cp /afs/isis/depts/atn/rcg/example_code/Gaussian/water.com /netscr/my_onyen/. cp /afs/isis/depts/atn/rcg/example_code/sas/test.sas /netscr/my_onyen/. • cp -r • cp -r ~/private/TestDirectory . • Move files • mv commands • mv ~/private/testfile.txt .

  11. File Manipulation Commands • Tar archive • To create a tar file • tar –cvzf TestDirectory.tgz ./ • To see a tar file’s table of content • tar –tvzf TestDirectory.tgz • To untar a tar file • tar –xvzf TestDirectory.tgz

  12. Submitting Jobs: LSF and Packages • LSF (Load Sharing Facility) • Fairly distribute compute nodes among users • 60 processor per user limit • Packages • ipm commands • ipm add (ipm a) • ipm remove (ipm r) • ipm query (ipm q) • Available packages • http://www.renci.org/unc/computing/resources/applications.php

  13. other hosts other hosts Execution host Submission host Master host 3 LIM LIM MLIM Load information 4 2 5 SBD MBD Batch API 11 8 9 Child SBD 1 7 6 queue 12 10 bsub app RES 13 LIM – Load Information Manager MLIM – Master LIM MBD – Master Batch Daemon SBD – Slave Batch Daemon RES – Remote Execution Server User job Details of LSF

  14. Submitting Jobs: bsub Command • bsub command • All files must be in /netscr/my_onyen/ • bsub [- bsub_opts] executable [-exec_opts] • Queues – bqueues command • week • idle • bsub –o • bsub –o out.%J

  15. Submitting Jobs: Following Job Progress • bjobs • bjobs –l JobID • Shows current status of job • bhist • bhist –l JobID • bkill • bkillJobID • Ends job prematurely • bfree

  16. Submitting Jobs: Specialty Scripts • bsas • bsub -q week -R blade sas program.sas • bsas test.sas • bmatlab • bsub -q week -R blade matlab -nodisplay -nojvm -nosplashprogram.m -logfile program.log • bmatlabtest.m

  17. Compiling on Emerald • Compilers • FORTRAN 77/90/95 • C/C++ • Parallel Computing • OpenMP • MPI (MPICH, LAM/MPI, MPICH-GM)

  18. Compiling Details on Emerald

  19. Compiling Details on Emerald • Add a compiler into your working environment • ipm add package_name • Compile a code • command code.f –o executable • Run executable on a compute node using the bsub command • bsub –q week –R blade executable

  20. Submitting Jobs: Job Output • Output sent to email • my_onyen@email.unc.edu • bsub –o • Output saved in working directory • bsub –u email@address • Output sent to specified email address

  21. Interactive Jobs: Setup • X-Windows • Linux/OSX • X11 client • Windows • X-Win32 • Offered on UNC Software Acquisition site • Port forwarding on SSH Secure Shell • Setting up a session on X-Win32

  22. Interactive Jobs: Submission • –Ip • bsub –q int –R blade –Ip sas • bsub –q int –R blade –Ip gv • Specialty Scripts • xsas • xstata

  23. Contacting Research Computing • For assistance with Emerald, please contact the Research Computing Group: • Email: research@unc.edu • Phone: 919-962-HELP • Submit help ticket at http://help.unc.edu

More Related