1 / 8

The Evolving Environment at NCCS

The Evolving Environment at NCCS. Vickie Lynch. Compute Node Linux. Jaguar will be down November 1-5 to change from Catamount to Compute Node Linux (CNL). CNL is a stripped-down version of Linux on the compute nodes.

glenna
Download Presentation

The Evolving Environment at NCCS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Evolving Environment at NCCS Vickie Lynch

  2. Compute Node Linux • Jaguar will be down November 1-5 to change from Catamount to Compute Node Linux (CNL). • CNL is a stripped-down version of Linux on the compute nodes. • Try the system running CNL that is accessible to all jaguar users: jaguarcnl.ccs.ornl.gov. • Jaguarcnl contains 2,976 dual core XT3 compute nodes each running CNL. • Eight additional cabinets will be removed from jaguar and added to jaguarcnl increasing it from 32 cabinets to 40 cabinets (7440 cores).

  3. Changes for Compute Node Linux • Currently only the /tmp/work/$USER lustre directory is visible to compute nodes. • There are several important differences between Catamount and CNL, and users are encouraged to view the details of those differences at http://info.nccs.gov/resources/jaguar. • Due to the differences between CNL and Catamount, users should recompile on the CNL system prior to running. • The batch option “-lsize” specifies cores under CNL. • aprun replaces yod as the job launcher under CNL. • Each user is currently allowed only one job in a running state. • The maximum walltime limit is 12 hours.

  4. CNL batch example #PBS -A STF006 #PBS -N test #PBS -j oe #PBS -q debug #PBS -l walltime=0:50:00,size=4 cd /tmp/work/vel/cqlptr export PGPLOT_FONT=/tmp/work/vel/pgplot_xt3/grfont.dat aprun -n4 ./cqlp

  5. Pathscale compiler • The pathscale compiler is now available on jaguar and jaguarcnl. • Before compiling: • module swap PrgEnv-pgi PrgEnv-pathscale • Use ftn –O3 instead of ftn –fast for compiler optimization • Use -Wl,--allow-multiple-definition for the ldr flag if you get too many warnings about multiple definitions • Use a SuperLU library compiled with pathscale: • /spin/home/vel/lib/SuperLU_DIST/libsuperlu.a

  6. Quad-core Upgrade • Jaguar shutdown for upgrade: December 4, 2007- ? • During this time users will have access to the remaining 40 XT cabinet transition system (7440 cores). • Jaguar will be upgraded to 250 teraflops. • The dual-core processors will be replaced with quad-core processors. • Memory will be doubled. • This transition will involve significant down time. • Consider using OpenMP across the cores of a processor

  7. Petaflop system • A petaflop system will be installed at the NCCS in 2008. • The LCF Petaflop system will also be quad-core. • It will come with a new interconnect that should enable better scalability than the current XT4. • This new interconnect is called Gemini and it will allow you to effectively perform Co-Array Fortran, UPC and Global Arrays.

  8. Summary

More Related