1 / 37

Message Passing Interface

Message Passing Interface. Amir Yazdanbakhsh. Outline. Introduction Point to point communication MPI global operations. Introduction. Message passing is a programming paradigm used widely on distributed memory parallel computers

roscoe
Download Presentation

Message Passing Interface

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Message Passing Interface Amir Yazdanbakhsh

  2. Outline • Introduction • Point to point communication • MPI global operations

  3. Introduction • Message passing is a programming paradigm used widely on distributed memory parallel computers • The standard defines the syntax and semantics of a core of library routines useful to a wide range of users writing portable message-passing programs in Fortran, C, or C++. • Version 1 of this standard called MPI-1 was released in summer 1994. • Since its release, the MPI specification has become the leading standard for message-passing libraries for parallel computers. • In July 1997 the MPI Forum completed the definition of MPI-2

  4. What Is Included in MPI? • Version 1 of the MPI standard includes: • Point-to-point communication • Collective operations • Process groups • Communication domains • Process topologies • Profiling interface • To these MPI-2 added: • Dynamic process management • Input/Output • One-sided operations

  5. MPI functions • MPI-1 comprises 129 functions [150 function in MPI-2] • 6 basic functions for point to point communication

  6. Main Functions Many parallel programs can be written using just these six functions: • MPI_INIT: Initiate an MPI computation • MPI_FINALIZE: Terminate a computation • MPI_COMM_SIZE: Determine the number of processes • MPI_COMM_RANK: Determine process identifier • MPI_SEND: Send a message • MPI_RECV: Receive a message

  7. MPI_INIT and MPI_FINALIZE • MPI_INIT() • Initiate an MPI computation • must be called before any other MPI function • must be called exactly once per process • MPI_FINALIZE() • shut down a computation • No further MPI functions can be called after MPI_FINILIZE

  8. MPI_COMM_SIZE MPI_COMM_SIZE(comm, size) • Determine the number of processes in a computation • Communicator specifies a communication domain for this communication • MPI_COMM_WORLD is a default communicator provided upon start-up that defines an initial communication domain for all the processes that participate in the computation.

  9. MPI_COMM_RANK (comm , pid) Determine the integer identifier assigned to the current process MPI_COMM_RANK

  10. MPI Send and Receive • MPI provides a set of send and receive functions that allow the communication of typed data with an associated tag. • Typing of the message contents is necessary for heterogeneous support—the type information is needed so that correct data representation conversions can be performed as data is sent from one architecture to another. • The tag allows selectivity of messages at the receiving end: one can receive on a particular tag, or one can use a ''wild-card" value for this quantity, allowing reception of messages with any tag.

  11. MPI Datatype MPI datatype C datatype MPI_CHAR signed char MPI_INT signed int MPI_FLOAT float MPI_DOUBLE double MPI_SIGNED_CHAR signed char MPI_UNSIGNEDCHAR unsigned char MPI_UNSIGNED unsigned int MPI_LONG signed long MPI_UNSIGNED_LONG unsigned long MPI_LONG_DOUBLE long double MPI_SHORT signed short MPI_UNSIGNED_SHORT unsigned short

  12. MPI Blocking Send MPI_SEND (buf, count, type, dest, tag, comm) • The message buffer is described by (buf, count, type). • The target process is specified by dest, which is the rank of the target process in the communicator specified by comm. • When this function returns, the data has been delivered to the system and the buffer can be reused. The message may not have been received by the target process. • MPI_SEND can return the error codes • MPI_ERRBUFFER (an invalid buf), • MPI_ERR_COUNT (an invalid count), • MPI_ERR_TYPE (an invalid datatype), • MPI_ERR_RANK (an invalid dest), • MPI_ERR_TAG (an invalid tag) • MPI_ERR_COMM (an invalid comm)

  13. MPI_SEND (buf, count, type, dest, tag, comm) MPI_SEND

  14. MPI Blocking Receive • MPI_RECV (buf, count, type, source, tag, comm, status) • Waits until a matching (on source and tag and comm) message is received from the system, and the buffer can be used. • The receiver may specify a rank or wildcard value for source (MPI_ANY_SOURCE), and/or a wildcard value for tag (MPI_ANY_TAG), indicating, respectively, that any source and/or tag are acceptable. • One cannot specify a wildcard value for comm. • Receiving fewer than count occurrences of datatype is OK, but receiving more is an error. • status contains information on the source and tag of the message and how many elements were actually received

  15. MPI_RECV(buf, count, type, source, tag, comm, status) MPI_RECV

  16. MPI_GET_COUNT MPI_GET_COUNT(status, datatype, count) Gets the number of "top level" elements.

  17. Example char msg[20]; int myrank, tag = 99; MPI_Status status; MPI_Comm_rank(MPI_COMM_WORLD, &myrank); /* find my rank */ if (myrank == 0) { strcpy(msg, "Hello there"); MPI_Send(msg, strlen(msg)+1, MPI_CHAR, 1, tag, MPI_COMM_WORLD); } else if (myrank == 1) { MPI_Recv(msg, 20, MPI_CHAR, 0, tag, MPI_COMM_WORLD, &status); }

  18. Description • process 0 sending a message to process 1. • The code executes on both process 0 and process 1. • Process 0 sends a character string using MPI_Send( ). • The first three parameters of the send call specify the data to be sent: the outgoing data is to be taken from msg; it consists of strlen(msg)+1 entries, each of type MPI_CHAR. • The fourth parameter specifies the message destination, which is process 1. • The fifth parameter specifies the message tag. • Finally, the last parameter is a communicator that specifies a communication domain for this communication.

  19. Description • The receiving process specified that the incoming data was to be placed in msg and that it had a maximum size of 20 entries, of type MPI_CHAR. • The variable status, set by MPI_Recv( ), gives information on the source and tag of the message and how many elements were actually received. The receiver can examine this variable to find out the actual length of the character string received.

  20. Blocking & Non-Blocking • The send call blocks until the send buffer can be reclaimed (i.e., after the send, process 0 can safely overwrite the contents of msg). • Similarly, the receive function blocks until the receive buffer actually contains the contents of the message. • MPI also provides nonblocking send and receive functions that allow the possible overlap of message transmittal with computation, or the overlap of multiple message transmittals with one another. • Non-blocking functions always come in two parts: the posting functions, which begin the requested operation; and the test-for-completion functions, which allow the application program to discover whether the requested operation has completed.

  21. Communication Modes • Buffered-mode send operation can be started whether or not a matching receive has been posted. (MPI_BSEND) • Synchronous-mode send operation can be started whether or not a matching receive was posted. However, the send will complete successfully only if a matching receive is posted, and the receive operation has started to receive the message sent by the synchronous send. (MPI_SSEND) • Ready-mode send operation may be started only if the matching receive has already been posted. Otherwise, the operation is erroneous and its outcome is undefined. (MPI_RSEND)

  22. Message order • Message-passing programming models are by default nondeterministic • the arrival order of messages   sent from two processes, A and B, to a third process, C, is not defined • MPI does guarantee that two messages sent from process A to process, B, will arrive in order • Message tags provide a mechanism for distinguishing between different messages

  23. Global Operations • Barrier: Synchronizes all processes • Broadcast: Sends data from one process to all processes • Gather: Gathers data from all processes to one process • Scatter: Scatters data from one process to all processes • Reduction operations: Sums, multiplies, etc., distributed data

  24. used to synchronize execution of a group of   processes No process returns from this function until all processes have called it MPI_BARRIER (comm)

  25. MPI_BCAST (buf, count, type, root, comm) Broadcast data from root to all processes MPI_BCAST

  26. MPI_BCAST • one-to-allbroadcast • a single process ( root) sends the same data to all other processes • After the call, data is replicated in Buf in all processes (including root)

  27. MPI_GATHER(inbuf, incnt, intype, outbuf, outcnt, outtype, root, comm) Gathers data from all processes to one process MPI_GATHER

  28. all-to-one gather All processes (including the root process) send the inbuf data to root the outbuf in the root process must be n times larger than inbuf MPI_GATHER

  29. MPI_SCATER(inbuf, incnt, intype, outbuf, outcnt, outtype, root,comm) Scatters data from one process to all processes MPI_SCATER

  30. MPI_SCATER • one-to-all scatter • sending the i th portion of its inbuf to process i • each process receives data from root in outbuf • the inbuf in the root process must be n times larger than outbuf

  31. MPI_SCATER MPI_Scatter( sendbuf, 100, MPI_INT, rbuf, 100, MPI_INT, root, comm); The root process scatters sets of 100 int to each process in the group.

  32. MPI_REDUCE, MPI_ALLREDUCE • Collective reduction functions • MPI_REDUCE(inbuf,outbuf,count,type,op,root,comm) • MPI_ALLREDUCE(inbuf,outbuf,count,type,op, comm)

  33. MPI_REDUCE, MPI_ALLREDUCE • Combine the values provided in the input buffer of each process and return the combined value either to the output buffer of the single root process (in the case of MPI_REDUCE) or to the output buffer of all processes ( MPI_ALLREDUCE) • Valid operations : • maximum and minimum (MPI_MAX & MPI_MIN) • sum and product ( MPI_SUM & MPI_PROD) • logical and, or, and exclusive or ( MPI_LAND & MPI_LOR & MPI_LXOR) • bitwise and, or, and exclusive or ( MPI_BAND & MPI_BOR & MPI_BXOR )

  34. MPI_REDUCE, MPI_ALLREDUCE An array of integers is distributed among processors. The program performs the reduction operation(summation) of data distributed over processes, and brings the result to the root process.

  35. MPI_REDUCE, MPI_ALLREDUCE

  36. Example int done = 0, n, myid, numprocs, i, rc; double PI25DT = 3.141592653589793238462643; double mypi, pi, h, sum, x, a; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); while (!done){ if (myid == 0) { printf("Enter the number of intervals: (0 quits) "); scanf("%d",&n); } MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD); if (n == 0) break; h = 1.0 / (double) n; sum = 0.0; for (i = myid + 1; i <= n; i += numprocs) { x = h * ((double)i - 0.5); sum += 4.0 / (1.0 + x*x); } mypi = h * sum; MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); if (myid == 0) printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); } MPI_Finalize();

  37. Any Question?

More Related