Extension of Fortran 95/2003 for parallel processing

1. Chris Sienkiewicz, Iain Flynn & Hemanth John Jose COMP60621 Directed Reading 1 - Co-array Fortran • Extension of Fortran 95/2003 for parallel processing • Published by Robert Numrich and John Reid in 1998 • The Fortran 2008 standard (ISO/IEC 1539-1:2010) includes co-arrays • Uses Single-Program-Multiple-Data (SPMD) programming model • A program is replicated a number of times and all copies were executed asynchronously • The data objects of each copy is termed an image • The number of images is fixed throughout execution • Shared data across images is represented using an extended array syntax with an index called co-array subscript in square brackets • The set of objects on all the images is itself an array, called a co-array

2. Chris Sienkiewicz, Iain Flynn & Hemanth John Jose COMP60621 Directed Reading 1 - Co-array Fortran Memory Model Syntax Examples -- num_images() - fixed number of images during execution -- this_image() - images operate asynchronously – real x(20, 20) - a private 20x20 array in each image – real y(20, 20) [*] - a shared 20x20 array in each image – x(:,j:j+2) = y(:,p:p+2) [r] - copy columns from p:p+2 into local columns -- sync_all

3. Chris Sienkiewicz, Iain Flynn & Hemanth John Jose COMP60621 Directed Reading 1 - Co-array Fortran Code Sample – MPI Vs. Co-array Fortran MPI program main include 'mpif.h' real array(10) integer rank, status(MPI_STATUS_SIZE), error ! Initialize MPI, and get process rank call MPI_INIT(error) call MPI_COMM_RANK(MPI_COMM_WORLD, rank, error) ! Initialize array on process 1 if ( rank == 1 ) array = 0.5 ! Send/recv data if ( rank == 0 ) then call MPI_RECV(array, 10, MPI_REAL, 1, 0, MPI_COMM_WORLD, status, error) else if ( rank == 1 ) then call MPI_SEND(array, 10, MPI_REAL, 0, 0, MPI_COMM_WORLD, error) endif ! Finish up call MPI_FINALIZE(error) end program Co-array Fortran program main real array(10)[*] if ( this_image() == 2 ) array = 0.5 sync all if ( this_image() == 1 ) array(:)[1] = array(:)[2] end program

4. Chris Sienkiewicz, Iain Flynn & Hemanth John Jose COMP60621 Directed Reading 1 - Co-array Fortran Strengths • Syntax is similar Fortran and requires Fortran programmers to learn only a few new rules • Data exchange points between images can be easily identified from syntax • Different size images are possible • Complier level optimisation is relatively easy Weaknesses • Standardised in a way to introduce minimal modifications to Fortran • Co-arrays must be declared as global variables • There is no support for processor subsets • No support for global pointers • No mechanisms to handle latency with data in remote images Recent Developments • Co-array Fortran 2.0, Rice University - CAF 2.0 compiler (pre-alpha)ha prototype CAF 2.0 compiler

5. Chris Sienkiewicz, Iain Flynn & Hemanth John Jose COMP60621 Directed Reading 1 - Co-array Fortran References • [1] R. Numrich, Silicon Graphics, Inc. and J. Reid, Rutherford Appleton Laboratory, Co-Array Fortran for parallel programming, ACM SIGPLAN Fortran Forum Volume 17 Issue 2, Aug. 1998 • [2] C. Coarfa et al., An Evaluation of Global Address Space Languages: Co-Array Fortran and Unified Parallel C, PPoPP’05, June 15–17, 2005 • [3] J.Reid, Coarrays in the next Fortran Standard, April 2010 • [4] R. Numrich, Co-Array Fortran - What is it? Why should you put it on BlueGene/L?, Minnesota Supercomputing Institute, University of Minnesota, July 2003 • [5] MachResearch.org, Is Co-Array Fortran the Future of Parallelism?, Oct 15 2007, Code sample accessed from: http://www.macresearch.org/is_co_array_fortran_the_future_of_parallelism • [6] Co-array Fortran 2.0, Rice University, http://caf.rice.edu/ • [7] ISO/IEC DIS 1539-1 Draft for Ballot, June 2010