Computer Science 320

1. Computer Science 320 Introduction to Cluster Computing

2. SMP (shared memory processor) architecture Cluster architecture

3. Cluster Middleware for Parallel Java

4. A Cluster Program in Parallel Java static Comm world; static int size; static int rank; static long x; static long t1, t2, t3; public static void main(String[] args) throws Exception{ t1 = System.currentTimeMillis(); Comm.init(args); world = Comm.world(); size = world.size(); rank = world.rank(); x = Long.parseLon(args[rank]); isPrime(x); // Output running time of rank here. } $ java –Dpj.np=4 Program1Clu 1000000000000037 1000000000000091 / 1000000000000159 1000000000000187

5. Communicators and Messages • Cluster processes communicate by sending messages over the backend network • A communicator is an abstraction of a medium for sending and receiving messages

6. A PJ World Comm.init(); Comm world = Comm.world(); int size = world.size(); int rank = world.rank(); $ java –Dpj.np=4 Program1Clu 1000000000000037 1000000000000091 / 1000000000000159 1000000000000187

7. Types of Communication • Point-to-point: transfer data from one process to another process • Collective: transfer data to all processes

8. Sending and Receiving world.send(toRank, buffer) CommStatusstatus = world.receive(fromRank, buffer) A buffer is an abstraction of a data source It can be a number, an array, a portion of an array, a matrix, a portion of a matrix, or other items

9. Sending and Receiving Master-worker pattern uses send and receive

10. Wildcard Receive CommStatus status = world.receive(null, buffer) Don’t need to know the rank of the processor; just receive from anybody

11. Nonblocking Send CommRequest request = new CommRequest(); world.send(toRank, buffer, request); // Other processing goes here request.waitForFinish(); Starts another thread to send the data, returns to do other work, then can block until send completes

12. Nonblocking Receive CommRequest request = new CommRequest(); world.receive(fromRank, buffer, request); // Other processing goes here CommStatus status = request.waitForFinish(); Starts another thread to receive the data, returns to do other work, then can block until receive completes

13. Send-Receive CommStatus status = world.receive(toRank, srcBuffer, fromRank, dstBuffer); Performs a send and a receive; nonblocking version is similar to the others