MPI_REDUCE()

1. MPI_REDUCE() Philip Madron Eric Remington

2. Basic Overview • MPI_Reduce() simply applies an MPI operation to select local memory values on each process, with a combined result placed in a memory location on the target process. • For example:

3. Basic Overview • Consider a system of 3 processes, which wants to sum the values of its local variable “int to_sum” and place it is result at process 0. • An abstraction of what occurs with a MPI_Reduce() statement is as follows:

4. Basic Overview • All processes reach the MPI_Reduce() function call. • Every process, including the target, sends their local value to be an operand in computing the desired result. • For the example (with the subscript indicating process number) here is what we in effect achieve:

5. Basic Overview • Here are operations that can be performed with MPI_Reduce(): • MPI_MAX maximum MPI_MIN minimum MPI_SUM sum MPI_PROD product MPI_LAND logical and MPI_BAND bit-wise and MPI_LOR logical or MPI_BOR bit-wise or MPI_LXOR logical xor MPI_BXOR bit-wise xor MPI_MAXLOC max value and location MPI_MINLOC min value and location

6. Basic Overview Notice there are no operations such as one that performs a subtraction. Consider the following:

7. Basic Overview Proc 0 Proc 1 Proc 2 Sum op • For the example, all processes are sending their data values to be summed, then placed in Proc 0. • As we are dealing with a parallel system, the scheduling of the processes is unknown, and the time at which they arrive to be summed is unknown—but as addition is commutative this is not important.

8. Basic Overview • If we had a subtract function it would not be possible to always correctly obtain the correct result, as subtraction is not commutative. A programmer would have to develop their own operation to account for the random process execution in the parallel environment. • All operations provided for MPI_Reduce() are commutative, and it’s implementation gives insight into how parallel program execute. • MPI_Reduce() can be used to perform a large variety of operations, where the programmer wants to apply an operation to a set of numbers with only copy of the operation’s result on a single process.