Supermachines Bluefire Frost Ranger Steel

1. Getting started at NCARPetascaleWorkshop July 6, 2010Si Liu and Rory KellyConsulting Group NCAR/CISL

2. SupermachinesBluefire Frost Ranger Steel • Bluefire (NCAR) IBM clustered Symmetric MultiProcessing system https://www2.cisl.ucar.edu/docs/bluefire-user-guide • Frost (NCAR) Four-rack, IBM Blue Gene/L system https://www2.cisl.ucar.edu/docs/frost/frost-user-guide • Ranger (TACC) Sun Constellation system http://services.tacc.utexas.edu/index.php/ranger-user-guide • Steel(Purdue) Dell computer cluster http://www.rcac.purdue.edu/teragrid/resources/

3. Login to Bluefire

4. Login to Frost Your UCAR password is inside the package you get.

5. Login to Rangerssh –X –Y username@ranger.tacc.utexas.eduYour password is inside the package you get. Login to Steel ssh –X –Y username@tg-steele.purdue.teragrid.org Your password is inside the package you get.

6. How to use Yubikey • Enabling your Yubikey token • Your yubikey has been activated and is ready for use. • The yubikey is activated by the warmth of your finger not the pressure in pushing the button. • Using your Yubikey token • When you are logging in to Bluefire or Frost, ssh -X –Y your_logon@bluefire.ucar.edu your screen displays a response similar to: Token_Response: • Enter you PIN number on the screen (do not hit enter) then touch the yubikey button. This will insert a new one-time password(OTP) and a return. • More information of Yubikey: • https://www2.cisl.ucar.edu/docs/enabling-your-yubikey-token

7. Compilers on bluefire • The Fortran and Fortran90 compilers: • xlf, xlf_r, xlf90, xlf90_r • mpxlf, mpxlf_r, mpxlf90, mpxlf90_r • The C and C++ compilers: • xlc, xlc_r, xlC, xlC_r • mpcc, mpcc_r, mpCC, mpCC_r • The _r versions are threadsafe. • We recommend them over the non _r versions. • Compile your source code • xlc_rhello_world.c -o hello_world_c.exe • xlf90_r hello_world.f -o hello_world_f.exe • More information • http://www2.cisl.ucar.edu/docs/bluefire/running-jobs

8. Compilers on Frost • The Fortran and Fortran90 compilers: • xlf, xlf90, xlf95 • mpxlf, mpxlf90 • The C and C++ compilers: • xlc, xlC • mpxlc, mpxlC • Scripts in /contrib/bgl/bin automatically set the include and MPI libraries for you. • mpxlc , mpxlC , mpxlf, mpxlf90 • Compile your source code • xlchello_world.c -o hello_world_c.exe • xlf90 hello_world.f -o hello_world_f.exe • More information • https://www2.cisl.ucar.edu/docs/frost/compiling-and-running-on-frost

9. Script file on Bluefire #!/bin/ksh # LSF batch script to run an MPI application on bluefire #BSUB -P 35141080 # project number for the workshop #BSUB -W 1:00 # wall clock time #BSUB -n 64 # number of MPI tasks #BSUB -R "span[ptile=64]" # run 64 tasks per node #BSUB –J myjob # job name #BSUB –o myjob.%J.out # output filename #BSUB -e myjob.%J.err # error filename #BSUB -q petascale # queue mpirun.lsf /usr/local/bin/launch ./myjob.exe

10. Submitting, monitoring, and deleting Jobs on Bluefire • Submit a job • bsub < script • Check job status • bjobs • bjobs -u all • bjobs -q regular -u all • bhist -n 3 -a • Deleting a job • bkill [jobid]

11. Script file on Frost cqsub –P 35141080 -q default -t 00:10:00 \ -n 28 -c 55 -m vn \ ./myjob.exe • -P: Project number for the workshop • -q: Specifies the queue for you job • -t: Set the wall clock time • -n: Specifies the node count for a job • -c: Specifies the number of processors to use • -m: co (coprocessor mode) or vn (virtual node mode)

12. Submitting, monitoring, and deleting Jobs on Frost • Submit a job • ./script • Check job status • cqstat • Delete a job • cqdel [jobid]

13. Questions • Examples and demos • Bluefire: /usr/local/examples/Petascale2010 • Frost: /home/rory/petascale/exercises • Check the update on our website • https://www2.cisl.ucar.edu/training/Petascale_UCAR_2010 • Send an email to your TAs • siliu@ucar.edu • rory@ucar.edu