1 / 85

Chapter 6

Chapter 6. Floyd’s Algorithm. The All-Pairs Shortest-Path Problem. Floyd’s Algorithm. Input n – number of vertices a[0..n-1,0..n-1] – adjacency matrix Output Transformed a that contains the shortest path lengths for k ← 0 to n-1 for i ← 0 to n-1

brent
Download Presentation

Chapter 6

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 6 Floyd’s Algorithm

  2. The All-Pairs Shortest-Path Problem

  3. Floyd’s Algorithm Input n – number of vertices a[0..n-1,0..n-1] – adjacency matrix Output Transformed a that contains the shortest path lengths for k ← 0 to n-1 for i ← 0 to n-1 for j ← 0 to n-1 a[i,j] ← min(a[i,j], a[i,k] + a[k,j]) endfor endfor endfor

  4. Creating Arrays at Run Time For one-dimensional n-element array of integers Int *A … A = (int *) malloc (n * sizeof(int))

  5. For two-dimensional array of integers int **B, *Bstorage, i; … Bstorage = (int *) malloc (m*n* sizeof(int)); B = (int **) malloc (m * sizeof (int *)); for (i=0; i < m; i++) B[i] = &Bstorage[i*n];

  6. Point to Point Communication

  7. Point to Point Communication • Point to Point Communication Instruction • MPI_Send • MPI_Recv • MPI Datatype • MPI Tags • Deadlock • Non-blocking Communication • MPI_Isend • MPI_Irecv • MPI_Wait • MPI_Sendrecv

  8. MPI Send/Receive • Most basic message-passing commands in MPI. • A message is data + envelope. The additional information from the envelope of the message are: • The rank of the receiver • The rank of the sender • A tag • A communicator

  9. Function MPI_Send int MPI_Send ( void *message, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm )

  10. Function MPI_Recv int MPI_Recv ( void *message, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status )

  11. status  MPI_source is the rank of the process sending the message status  MPI_tag is the message’s tag value status  MPI_ERROR is the error condition MPI_ANY_SOURCE can be the fourth argument MPI_ANY_TAG can be the fifth parameter argument

  12. MPI Send/Receive int Sdata[2] = {1,2}, Rdata[2], count = 2, src = 0, dest = 3; if(CPU_id == src) MPI_Send(&Sdata, 2, MPI_INTEGER, dest, 1, MPI_COMM_WORLD); if(CPU_id == dest){ MPI_Status status; MPI_Recv(&Rdata, 2, MPI_INTEGER, src, 1, MPI_COMM_WORLD, &status); } CPU0 CPU1 CPU2 MPI_Send MPI_Recv Sdata Rdata [1,2] [1,2] CPU3

  13. MPI Tags • Messages are sent with an accompanying user-defined integer ”tag”, to assist the receiving process in identifying the message. • Messages can be screened at the receiving end by specifying a specific tag, or not screened by specifying MPI_ANY_TAG as the tag in a receive.

  14. Program example 4.0 0 1.0

  15. Program example: original code #include <stdio.h> #define f(x) (4.0/(1.0+x*x)) void main(int argc, char ** argv){int i, n = 10000000; double w, x, sum, pi; w = 1.0/n; sum = 0.0; for(i=1 ; i<= n ; i++){ x = w*(double)(i – 0.5); sum += f(x);} pi = w*sum; printf(“PI : %.16f”,pi); }

  16. Program example: parallel code (1/2) #include “mpi.h” #include <stdio.h> #define f(x ) (4.0/(1.0+x*x)) void main(int argc, char * argv[]){ int myid, t_process, i, n = 10000000; double mypi, pi, h, sum, x, recv_pi; MPI_Init(&argc, &argv); MPI_Comm_size(MPI_COMM_WORLD, &t_process); MPI_Comm_rank(MPI_COMM_WORLD, &myid); h = 1.0 / (double)n; sum = 0.0; for(i = myid +1 ; i <= n ; i+= t_process){ x = h*(double)(i– 0.5); sum += f(x); }

  17. Program example: parallel code (2/2) mypi = h*sum; if(myid == 0){ pi = myid; MPI_Status status; for(i=1 ; i< t_process ; i++){ MPI_Recv(&recv_pi, 1, MPI_DOUBLE, i, 1, MPI_COMM_WORLD,&status); pi += recv_pi; } printf(“PI : %.16f\n”, pi); } else MPI_Send(&mypi, 1, MPI_DOUBLE, 0, 1, MPI_COMM_WORLD); MPI_Finalize(); }

  18. Communication Definitions • “Completion” of the communication means that memory locations used in the message transfer can be safely accessed • Send: variable sent can be reused after completion. • Receive: variable received can now be used. • MPI communication modes differ in what conditions are needed for completion. • Communication modes can be blocking or non-blocking.

  19. Blocking Communication • Call does not return until the operation has been completed. • Allows you to know when it is safe to use the data received or reuse the data sent.

  20. Blocking Communication • There is a diagram as follow that represents two SPMD tasks. • Both tasks are calling blocking standard sends at the same point of the program. • The diagram demonstrates that the two sends are each waiting on their corresponding receives in order to complete.

  21. Blocking Communication int data = 0, recv; MPI_Status stauts; if(id == 0) { MPI_Send( &data, 1, MPI_INTEGER, 1, 1, MPI_COMM_WORLD); MPI_Recv( &recv, 1, MPI_INTEGER, 1, 1, MPI_COMM_WORLD, &status); } if(id == 1) { MPI_Send( &data, 1, MPI_INTEGER, 0, 1, MPI_COMM_WORLD); MPI_Recv( &recv, 1, MPI_INTEGER, 0, 1, MPI_COMM_WORLD, &status); } CPU0 CPU1 MPI_Send MPI_Send MPI_Recv MPI_Recv time Waiting other process

  22. Deadlock • Deadlock is a phenomenon most common with blocking communication. • The waiting processes will never again change state, because the resources they have requested are held by other waiting processes.

  23. Deadlock Condition • Mutual Exclusion • A resource that cannot be used by more than one process at a time. • Hold and Wait • Processes already holding resources may request new resources held by other processes. • Non-Preemption • No resource can be forcibly removed from a process holding it, resources can be released only by the explicit action of the process. • Circular Wait • Two or more processes form a circular chain where each process waits for a resource that the next process in the chain holds.

  24. Deadlock

  25. Avoid Deadlock • Different ordering of calls between tasks • Arrange for one task to post its receive first and for the other to post its send first. That clearly establishes that the message in one direction will precede the other. • Non-blocking calls • Have each task post a non-blocking receive before it does any other communication. This allows each message to be received, no matter what the task is working on when the message arrives or in what order the sends are posted.

  26. Non-blocking Communication • Call returns immediately without knowing if the operation has been completed. • Allows you to overlap other computation while testing for the operation to be completed. • Less possibility of deadlocking code. • Used with MPI_Wait or MPI_Test.

  27. MPI_Isend int data[], count, dest, tag, err; MPI_Datatype, type; MPI_Comm comm; MPI_Request request; err = MPI_Isend( &data, count, type, dest, tag, comm, &request);

  28. MPI_Irecv int data[], count, src, tag, err; MPI_Datatype type; MPI_Comm comm; MPI_Request request; err = MPI_Irecv( &data, count, type, src, tag, comm, &request);

  29. MPI_Wait • Used for both sender and receiver of non-blocking communications. • On the receive side, receiving process blocks until message is received, under programmer control. • On the sending side, sending process blocks until send operation completes, at which time the message buffer is available for re-use. MPI_Request request; MPI_Status status; int err = MPI_Wait( &request, &status);

  30. Non-blocking Communication int data = 0, recv; MPI_Status stauts; MPI_Request request; if(id == 0) MPI_Isend(&data, 1, MPI_INTEGER, 1, 1, MPI_COMM_WORLD, & request); else if(id == 1) MPI_Irecv(&recv, 1, MPI_INTEGER, 0, 1, MPI_COMM_WORLD, & request); …… MPI_Wait(&request, &statuts); …… CPU0 CPU1 MPI_ISend MPI_IRecv MPI_Wait Computing Computing Computing without data time

  31. Avoid Deadlock • MPI_Sendrecv, MPI_Sendrecv_replace • Use MPI_Sendrecv. This is an elegant solution. In the _replace version, the system allocates some buffer space (not subject to the threshold limit) to deal with the exchange of messages. • Buffered Mode • Use buffered sends so that computation can proceed after copying the message to the user-supplied buffer. This will allow the receives to be executed.

  32. MPI_Sendrecv int Sdata[], Rdata[], Scount, Rcount, dest, src, Stag, Rtag, err; MPI_Comm comm; MPI_Datatype Stype, Rtype; MPI_Status status; err = MPI_Sendrecv( &Sdata, Scount, Stype, dest, Stag, &Rdata, Rcount, Rtype, src, Rtag, comm, &status);

  33. Designing the parallel algorithm Partitioning Communication Agglomeration Matrix input/output

  34. Documenting the Parallel Program

  35. Analysis and Benchmarking

  36. Collective Communication

  37. Collective Communication Instruction • MPI_Bcast • MPI_Scatter • MPI_Gather • MPI_Allgather • MPI_Reduce • MPI Reduction Function • MPI_Allreduce • MPI_Barrier • MPI_Wtime • Vector variant • MPI_Scan

  38. MPI_Bcast • Copy a scalar variable or an array from a CPU id to the other CPU ids which is at same communicator. int Data[], count, src, err; MPI_Datatype type; MPI_Comm COMM; err = MPI_Bcast( Data, count, TYPE, src, COMM);

  39. MPI_Bcast int data[4] = {1,2,3,4}; int count= 4, src= 0; MPI_Bcast(data, count, MPI_INTEGER, src, MPI_COMM_WORLD); CPU0 CPU1 CPU2 CPU3 data data data data MPI_Bcast

More Related