Software Overview Environment, libraries, debuggers, programming tools and applications

1. Software Overview Environment, libraries, debuggers, programming tools and applications Jonathan Carter NUG Training 3 Oct 2005

2. Overview • Environment • Libraries • Debuggers • Programming Tools • Applications

3. Environment • Suse Linux • Linux Networx HPC software stack • Pathscale compilers • PBS Pro Batch System • GPFS HOME and SCRATCH file systems

4. Environment (cont) • Passwords • NIM password, use NIM to change • Shells • Default shell is tcsh, to change use NIM interface • Startup files • Read-only, use <name>.ext if you want to add customizations • fixdots command will repair files

5. Environment (cont) • Access to libraries, tools and applications is controlled via modules • Number of modules reduced compared with seaborg, since we only have 64-bit ABI and fewer software versions

6. Libraries • Math • ACML, fftw, NAG, parpack, scalapack, superlu, (imsl, petsc) • I/O • hdf, hdf5, netcdf • MPI I/O available via MVAPICH • Visualization • ncar

7. Libraries (cont) • ACML • Optimized for AMD processors • BLAS 1-3, FFT and LAPACK • “fast” math functions (log is ~90 cycles instead of ~120) • “vector” math functions, e.g. vrda_log(n, x, y) is roughly twice as fast as log at n>20.

8. Libraries (cont) • Fortran I/O • Intel/AMD hardware is little-endian, Power is big-endian, so binary data is incompatible • pathf90 offers several features to write binary data in big-endian format

9. Libraries (cont) • pathf90 options • -byteswapio writes all data in format opposite to that of native processor • -conversion [native, little_endian, big_endian] • assign command FILENV=.assign export FILENV assign –N mips u:10

10. Libraries (cont) • Many libraries are linked at runtime, so LD_LIBRARY_PATH must include a path to each library linked against • Modules environment takes care of this provided you load at compile time and at run time

11. Debuggers • Totalview and gdb available for serial applications module load totalview totalview progname [corefile] gdb progname [corefile]

12. Debuggers (cont) • Totalview and gdb are coming for parallel applications • gdb opens xterm per processor module load totalview mpirun –tv –np procsprogname mpirun –debug –np procsprogname

13. Debuggers (cont) • Generating core files • Serial applications ulimit -S -c unlimited • Parallel applications, make shell script progname.sh and run this via mpirun (this problem will be resolved by PBS bugfix) #!/bin/bash ulimit -S -c unlimited exec progname

14. Debuggers (cont) • Useful Pathscale compiler options • Use –g flag to help debugging • Use –trapuv to set uninitialized (local, automatic, alloca();not Fortran allocatable) variables to NAN

15. Debuggers (cont) • Further information on Totalview • Etnus tutorial http://www.etnus.com/TotalView/started/getting_started.html • LLNL tutorial • http://www.llnl.gov/computing/tutorials/totalview/

16. Programming Tools • AMD CodeAnalyst – suite of tools to optimize code for Opteron: coming soon • ipm – mpi overhead and performance report • papi – interface to hardware performance counters • tau – suite of tools enabling performance analysis of application codes • valgrind – multiple tools to check memory use, profile cache use, detect data race conditions in threaded applications

17. Applications • Chemistry & Materials Science • GAMESS, Gaussian, Gromacs, MOLPRO, NWChem, NAMD, VASP • Math • maple, matlab, mathematica • Visualization • idl, vmd • Other • Subversion (client)