Introduction to Using Supercomputers

1. Introduction to Using Supercomputers Thomas Billman

2. Overview • What is a supercomputer? • Prerequisites • Connecting to a supercomputer • Transferring files • Example

3. What is a Supercomputer? • Lots of RAM and storage • Accessed remotely • Need an active allocation to use Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua

4. Prerequisites • Computer-specific Account • TACC for Stampede2 • PSC for Bridges • XSEDE Account with an allocation • What Dr. Chen uses to add you to her allocation • A computer with an ssh client (PuTTY, etc.) or that can ssh in terminal • How you actually connect to a supercomputer

5. PSC Account Creation • Google “bridges supercomputer user guide” • Click the first link • Scroll down until you see the text from this image -> • Click the underlined link and set up your password

6. XSEDE Account Creation • Google “XSEDE user portal” • Click the first link • Create an account or sign in • Confirm you are on an allocation

7. Connecting to the supercomputer • Open either your terminal or ssh client • Type: • ssh yourusername@stampede2.tacc.utexas.edu for Stampede2 • ssh yourusername@bridges.psc.edu for Bridges • Replace yourusername with your user name • You will see the following dialogue, then enter your password, then the code you are sent

8. Connecting to Stampede2 • You will be met with the following dialogue and be connected to Stampede2

9. What did I just do? • SSH stands for secure shell, and is a connection that allows you to securely send and receive information to the Supercomputer on an unsecure connection • Now that we are connected, we can store files and run scripts on the Supercomputer

10. UNIX commands crash course

11. UNIX commands crash course Using “ls” to list the contents in my home directory Using “cd” to change my directory to the “Howell” folder. Once inside I again list the files.

12. UNIX commands crash course From this folder I want Stampede2 to run the Ratio.R file. I can not do this directly, so I write a shell script with more information and submit that. This allows Stampede2 to run computation on the nodes designed for computation rather than login commands

13. What was in that shell script? Begin script -J Name Job -o Name output file -N Number of nodes -n Number of cores -p Queue -t Time limit -A Allocation Email for updates Mail type Commands

14. Preparing shell scripts for Bridges Information on all options can be found here: https://www.psc.edu/bridges/user-guide/running-jobs#batch

15. Whew...

16. Transferring Files

17. File transfer from the command line (sftp) • SFTP stands for Secure File Transfer Protocol. • Similar to SSH, but you have access to both computers • To begin type: • sftp yourusername@stampede2.tacc.utexas.edu for Stampede2 • sftp yourusername@data.bridges.psc.edu for Bridges • Similar commands to ssh, but not the same • cd and ls control your Supercomputer directory • lcd and lls control your local directory (local change directory and local list) • The “put” command places files from your local directory into your Supercomputer directory • The “get” command places files from your Supercomputer directory into your local directory

18. Example Job

19. Running an R script on Stampede2 • Write and test your script locally • Edit a shell script to run your file • sftp into Stampede2 and upload your R script and shell script • Exit out of your sftp connection • Ssh into Stampede2 and sbatch your shell script • Receive an email telling you when your job finishes • sftp into Stampede2 and download the output onto your computer • Evaluate the output and see if rerunning is necessary

20. More information: Bridges User Guide: https://www.psc.edu/bridges/user-guide

21. Questions?

22. R and Rstudioin Bridges • https://www.psc.edu/user-resources/software/r • You can write CSV file in your HPC directory. • cd $SCRATCH • cd $HOME • id –gn • id -GN