CCPR Workshop Lexis Cluster Introduction

1. CCPR WorkshopLexis Cluster Introduction October 19, 2007 David Ash

2. Outline • What is a cluster? ….two components

3. Outline Continued… • How can CCPR’s cluster help you? • Lexis usage • Connecting from Windows • Unix commands • Submitting jobs • Managing jobs

4. Lexis Management System • Jobs are submitted via cluster management system • An advanced batch system • Efficiently allocates resources to jobs • Remembers node specs and availability • Schedules jobs according to requested resources and priority • Users can easily view jobs

5. How does this help your research? • Remote access to software and data from anywhere! • Incredible machines – faster processors and more memory • Flexible system to share data and programs with colleagues • Opens up your home/office PC for other things, instead of grinding away on a Stata job • Advanced users: parallelization, fast compilers

6. Connecting to Lexis • Secure Shell Software • SSH (command line access) • SFTP (file transferring access) • Obtain from: http://computing.sscnet.ucla.edu/public/security/ssh.aspx • Alternative Client Software • PuTTY (SSH) http://the.earth.li/~sgtatham/putty/latest/x86/putty.exe • winSCP (SFTP) http://winscp.net/eng/download.php

7. Connecting to Lexis: Secure Shell • Directions for SSH Secure Shell: • Click Quick Connect • Enter: • Host Name: lexis.ccpr.ucla.edu • User Name: your lexis account name • Port Number: 22 • Click connect and enter password • Can “Add Profile” for repeat usage • Same for SSH Secure File Transfer

8. Mapping Lexis as a Drive using Samba • Creating Lexis as a drive in Windows Explorer • Must use the VPN to do this action off campus! • http://www.bol.ucla.edu/services/vpn/ • Instructions I’m going to show are at: • Step-by-step instructions: • http://lexis.ccpr.ucla.edu/using/samba/

9. Mapping Lexis • Open Windows Explorer • Select Tools -->Map Network Drive • At "Folder" text box enter: \\lexis.ccpr.ucla.edu\your-user-name • Click "Reconnect at login" if you want the drive to be connected each time the computer starts • Click the "different user name" link to enter your user name and password for the connection. • Click "OK" then click the "Finish" button on the Map Network Drive window.

10. Linux Commands

11. Getting Help in Linux

12. Linux File Editors • Pico – easiest editor. Open and type. Commands listed at bottom of editor. • Vi – more advanced. You can move around quickly, do copy/paste, search, etc, but keystrokes required for everything. • Emacs – most advanced • OR edit files over Samba from Windows.

13. Submitting Stata and SAS jobs • Submit jobs to the Cluster using ONE of the following commands: qjob, qstata, qsas, qmatlab • Will be prompted for: • Job type (for qjob only!) • Filename • Amount of Memory (number in megs, ex. 1050) • If memory > 1 gig --> Over 2 hours (y/n) • Confirmation to submit (y/n) • Quick Method: • qstatasample.do 500 y

14. Qstata example output • Two Logs are created • filename.log - This is the log file from Stata. The output of your entire Stata job is captured in this file. • filename.do.joblog - This file will capture the output of your cluster command file and any errors that may occur (Stata output is NOT captured by this log.) • filename.do.cmd - This is the command file that is submitted to the cluster. • Quick TIP: qsub filename.do.cmd

15. Cluster job monitoring • qstat or qstatus to view overall cluster usage • qstat –jjobnumber for information particular to your job • qdel jobnumber to kill your job

16. Interactive Sessions • Interactive Usage • Meant for testing before submitting to cluster • Limited spots • Logoff promptly when finished • Non-GUI Interactive Session • Type “qinteractive”, can also specify memory after command. Default 256mb. • GUI Interactive Session • Login with XliveCD • Type “qlive” – Run sas/matlab/xstata

17. Questions / Comments?