1 / 14

Research Computing with Newton

Research Computing with Newton. Gerald Ragghianti SCALE-IT on Newton HPC workshop Dec. 2, 2010. What is the Newton Program?. Research computing support Infrastructure management Consultation Training Research Objectives Effectiveness Efficiency Capability. The Newton cluster.

murphy-nieves
Download Presentation

Research Computing with Newton

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Computing with Newton Gerald Ragghianti SCALE-IT on Newton HPC workshop Dec. 2, 2010

  2. What is the Newton Program? Research computing support Infrastructure management Consultation Training Research Objectives Effectiveness Efficiency Capability

  3. The Newton cluster “Normal” Linux compute cluster 295 computers 2500 processors 5TB RAM 40 Gbit/sec Infiniband 80 TB Storage Head node Lustre storage Storage server Lustre storage Storage server Lustre storage External network Ethernet network Compute node Compute node Compute node Compute node Interactive node Compute node Compute node Interactive node Compute node Compute node Infiniband network

  4. Newton cluster machines

  5. Getting started SSH to login.newton.utk.edu using NetID Transfer files with scp, sftp, or FileZilla Display graphics with X11, xorg, or Xming Requires X11 “tunneling” through SSH client $ ssh gragghia@login.newton.utk.edu Password: *************** [gragghia@newton1 ~]$ ls Test.sge filename.txt • [gragghia@newton1 ~]$ w • 10:36:49 up 32 days, 15:07, 20 users, load average: 1.98, 1.81, 1.88 • USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT • gragghia pts/0 poltth Tue05 1:05 1.39s 1.39s -bash • mkzadd pts/1 bkg.engr.utk.edu Thu18 15:16m 0.06s 0.06s -bash • Krrrccc pts/2 ares.bio.utk.edu 03Aug10 3days 0.03s 0.03s -bash

  6. Environment management Modules utility Manages environment variables and aliases User chooses applications and libraries to use Allows multiple versions to be available Example use: See available modules: “module avail” Load a module: “module add R” Unload a module: “module unload R”

  7. Resource Management: The Grid Engine Accepts job requests Executable to run Execution time Parallelization RAM needed Finds available resources (compute nodes) Reserves and uses resources Returns output

  8. A simple job Create a job request file. $ cp ~gragghia/workshop ~/ Change working directory $ cd ~/workshop/test Submit job $ qsubtest.sge Monitor job $ qstat -g t View result log files #$ -N Test #$ -cwd #$ -q short* uname –a sleep 30

  9. More Sophistication: Array jobs »Run the same job multiple times Start a new working directory $ cd ~/workshop/array Create data files (optional) $ ~gragghia/make_datafiles.sh Create a job request file with “-t” option: Submit job $ qsubarray.sge Monitor job $ qstat -g t View result log files #$ -N Array #$ -cwd #$ -q short* #$ -t 1-10 md5sum data-$SGE_TASK_ID.dat

  10. A parallel job: MPI Download the software: $ wget http://newton.utk.edu/workshop/hello.tar Extract the software: $ tar –vxf hello.tar Verify correct MPI version: $ module list Compile the application: $ cd hello $ make Create a batch submit file Submit the job #$ -N Hello #$ -q short* #$ -cwd #$ -peopenmpi* 16 mpirun hello sleep 30

  11. Using Graphics Useful for: Development using an IDE Data visualization (graphs) Complex GUI programs Steps Start X11 server on your workstation (Xming) Enable “X11 tunneling” in your SSH client Sign in to Newton using SSH Verify that $DISPLAY contains “localhost” $ echo $DISPLAY Start graphical program Optionally using “qrsh”

  12. Compiling and Installing Software Example: Fractal generator Find the software Transfer to Newton Direct: wget http://newton.utk.edu/workshop/gmandel.tgz Indirect: Download to workstation and scp (sftp) Extracting the source code Uncompressed: tar Compressed: gunzip or unzip Configure the software: $ ./configure –prefix=$HOME/gmandel Compile: $ make Install: $ make install $ wgethttp://newton.utk.edu/workshop/gmandel.tgz $ tar –vzxf gmandel.tgz $ ./configure –-prefix=$HOME/gmandel $ make install …

  13. Commercial Applications Matlab Graphical (interactive) Batch mode (parallel): matlab –r <Function> SAS SPSS $ module load matlabt $ matlab $ matlab –r ‘TestFunction’

  14. More Information Newton Program website: http://newton.utk.edu/ Program policies Documentation Meetings / support / consulting schedule Research Computing Mailing List: USG_HPCC@listserv.utk.edu

More Related