MPI

1. MPI • MPI = Message Passing Interface • Specification of message passing libraries for developers and users • Not a library by itself, but specifies what such a library should be • Specifies application programming interface (API) for such libraries • Many libraries implement such APIs on different platforms – MPI libraries • Goal: provide a standard for writing message passing programs • Portable, efficient, flexible • Language binding: C, C++, FORTRAN, Python programs

2. General MPI Programs #include <mpi.h> main( int argc, char** argv ) { MPI_Init( &argc, &argv ); /* main part of the program */ /* Use MPI function call depend on your data partitioning and the parallelization architecture */ MPI_Finalize(); }

3. MPI Basics • MPI’s pre-defined constants, function prototypes, etc., are included in a header file. This file must be included in your code wherever MPI function calls appear (in “main” and in user subroutines/functions) : • #include “mpi.h” for C codes • #include “mpi++.h” * for C++ codes • include “mpif.h” for f77 and f9x codes • MPI_Init must be the first MPI function called • Terminates MPI by calling MPI_Finalize • These two functions must only be called once in user code.

4. MPI Basics • C is case-sensitive language. MPI function names always begin with “MPI_”, followed by specific name with leading character capitalized, e.g., MPI_Comm_rank. MPI pre-defined constant variables are expressed in upper case characters, e.g.,MPI_COMM_WORLD.

5. Basic MPI Datatypes MPI datatypeC datatype MPI_CHAR signed char MPI_SIGNED_CHAR signed char MPI_UNSIGNED_CHAR unsigned char MPI_SHORT signed short MPI_UNSIGNED_SHORT unsigned short MPI_INT signed int MPI_UNSIGNED unsigned int MPI_LONG signed long MPI_UNSIGNED_LONG unsigned long MPI_FLOAT float MPI_DOUBLE double MPI_LONG_DOUBLE long double

6. MPI is Simple • Many parallel programs can be written using just these six functions. • MPI_Init • MPI_Finalize • MPI_Comm_size • MPI_Comm_rank • MPI_Send • MPI_Recv

7. Initialization • Initialization: MPI_Init()initializes MPI environment • Must be called before any other MPI routine (so put it at the beginning of code) • Can be called only once; subsequent calls are erroneous. MPI_Init(int *argc, char **argv)

8. Termination • MPI_Finalize() cleans up MPI environment • Must be called before exits. • No other MPI routines can be called after this call, even MPI_Init()

9. Use Putty to Log in Penzias Click Start, find PuTTy, and open it

10. Use Putty to Log in Penzias Or download it from the following link: https://the.earth.li/~sgtatham/putty/latest/w32/putty.exe, and open it. 1. Type the Host Name: penzias.csi.cuny.edu 2. Click “Open”

11. Use Putty To Login in Penzias Click Yes

12. Use Putty to Log in Penzias 1. Type your account, and press Enter 2. Type your password, and press Enter. Don’t press Backspace. The typed characters will NOT be shown.

13. Use Putty to Log in Penzias

14. Example 1 Compile • Type “cd example1”, you will enter the folder example1 • Compile the MPI program by typing “mpicc -o hello hello.c”, and you will have the executable file hello in the folder example1 • Type “ls”, we can check the files and folders in the current folder example1

15. Example 1 Run vi mpi.sh #!/bin/bash #BATCH --job-name slurm_mpi #SBATCH --nodes 2 #SBATCH --ntasks 4 #SBATCH --ntasks-per-node 16 #SBATCH --mem 20000 #SBATCH --partition partedu cd $SLURM_SUBMIT_DIR echo 0 > cap mpirun -np 4 ./hello Number of used processors

16. Run a Job on HPC using Slurm Common commands: sbatch: submit a job scancel: cancel a job squeue: display state of jobs