230 likes | 418 Views
CCPR Workshop Introduction to the Cluster. July 13, 2006. Outline. What is a cluster? How can CCPR’s cluster help you? Using the cluster Connecting from Windows Unix commands Submitting jobs Managing jobs. What is a cluster?. Think of a cluster as having two parts :
E N D
CCPR WorkshopIntroduction to the Cluster July 13, 2006
Outline • What is a cluster? • How can CCPR’s cluster help you? • Using the cluster • Connecting from Windows • Unix commands • Submitting jobs • Managing jobs
What is a cluster? • Think of a cluster as having two parts: • Hardware: a collection of computers • Cluster management system: software for coordinating hardware with a collection of computational tasks (batch jobs)
CCPR’s Cluster: Management System • Jobs are submitted to the cluster via the cluster management system • An advanced batch system • Efficiently allocates resources to jobs • Remembers job requirements • Remembers node specs and availability • Schedules jobs according to requested resources and priority • Users can easily view jobs
How does this help your research? • Easy access to software and data • Faster processors • More RAM • Easy to share data, programs, etc. with colleagues via the cluster • More advanced users: parallel processing, compilers, etc. • Your pc is available for other work when you submit a job to the cluster • Submit a job and forget about it
Using the Cluster • Connecting from Windows • Unix Commands • Submitting jobs • Managing jobs
Secure Shell • Secure Shell software (2 pieces) • SSH Secure Shell Client (Telnet) • SSH Secure File Transfer Client (FTP). • Download software from www.ssc.ucla.edu Computing Security
Secure Shell: Connecting to lexis from Windows • Connect to lexis via SSH Secure Shell Client • Startup SSH Secure Shell/File Transfer • Click Quick Connect • Enter: • Host Name: lexis.ccpr.ucla.edu • User Name: your lexis account name • Port Number: 22 • Click connect and enter password • Can add “profile” for repeated use • Same process for SSH Secure File Transfer Client
Samba: Transferring files from pc to lexis • Analagous to “mapping a drive” • Allows user to view lexis files via Windows Explorer interface • Must use vpn when you’re off campus. Download from • http://www.bol.ucla.edu/services/vpn/ • Step-by-step instructions: • http://lexis.ccpr.ucla.edu/using/samba/
Samba: Transferring files from pc to lexis • Open Windows Explorer on the local machine. • Select Tools->Map Network Drive • In the "Folder" text box enter: \\lexis.ccpr.ucla.edu\your-user-name • Click "Reconnect at login" if you want the drive to be mounted each time the computer starts up. • Click the "different user name" link to enter your user name and password for the connection. • Click "OK" then click the "Finish" button on the Map Network Drive window.
Unix Commands: Using Aliases • Alias • Create an alias for commonly used commands • Example: alias ls=“ls –lah” • Anytime you type ls, you will see the ls command with options lah (l=long, a=all, h=human-readable) • Edit .aliases file to create and remove aliases
Editing Files on Lexis • Pico – easiest editor. Open and type. Commands listed at bottom of editor. • Vi – more advanced. You can move around quickly, do copy/paste, search, etc, but keystrokes required for everything. • Emacs – most advanced
Submitting Stata and SAS jobs • Submit jobs by typing qjob, qstata, qsas, qmatlab at Unix prompt • You will then be prompted for: • filename • memory (number in megs, ex. 1050) • if memory > 1 gig -> Over 2 hours (y/n), • job type (for qjob only) • submit (y/n) • Alternatively, submit all options without being prompted: • qstata sample.do 500 y
Files produced by qstata • filename.log - This is the log file from Stata. The output of your entire Stata job is caputured in this file. • filename.do.cmd - This is the Command file that is submitted to the cluster. • filename.do.joblog - This file will capture the output of your CMD file and any errors that may occur (Stata output is not captured by this log.)
Job Management • qstat or qstatus to view overall cluster usage • qstat –jjobnumber for information particular to your job • qdel jobnumber to kill your job
Working interactively - qinteractive • To work interactively, type “qinteractive” at prompt • Default RAM allotted is 256 • Request x megs by typing “qinteractive x” • Please use interactive nodes responsibly • Limited number of spaces • Testing only • Logoff promptly