Applications of HPJava

1. Applications of HPJava Bryan Carpenter, Geoffrey Fox, Han-Ku Lee and Sang Boem Lim Community Grids Lab Indiana University www.hpjava.org

2. Introduction • HPJava is an environment for data-parallel and SPMD programming on distributed-memory parallel computers (e.g. clusters). • It introduces various Fortran-like features into the Javaprogramminglanguage. • It also introduces a language model for data parallel programming that has some novel features (HPspmd model). • Talk will introduce HPJava, and briefly describe two applications. www.hpjava.org

3. Roots of HPJava • Many projects have worked to exploit C++ features for parallel computing with distributed data • A++/P++, Kelp, HPC++, STAPL,… • These often use templates, operator overloading, iterators, etc, to create “little languages” for parallel computing. • Some years ago we tried to develop one such system, reusing parallel libraries originally developed for HPF. • ad++ programming model seemed to capture much of the spirit of HPF (and MPI). But difficult to see how to make it efficient, without an intelligent preprocessor to optimize the special parallel loop constructs. • Writing a preprocessor for C++ seemed hard. • Java emerged. It looked like a simpler, C++-like language. We thought we could produce a preprocessor for Java extended with desired features quickly: HPJava. www.hpjava.org

4. Rationale II • Since Java emerged, various people have tried to make the Java platform more suitable for scientific and large scale computing—particularly through the Java Grande process. • One focus of the Java Grande Numerics Working Group has been introduction of scientific multidimensional arrays (or multiarrays) into Java. • A natural Java approach is through class libraries for new containers. • E.g. the Multiarray package JSR from IBM. • May again need special treatment by compiler/JVM, for efficiency. • Another approach actively discussed by JGNWG is to add extra syntax to the language for multiarrays, and expand by a preprocessor. • HPJava incorporates an implementation of this approach. www.hpjava.org

5. Example: Multiarray Syntax • Examples of HPJava syntax for multiarrays: int [[ * , * ]] a = new int [[ 5 , 5 ]] ;// Type signature, // creation expression for (int i=0; i<4; i++) a [ i , i+1 ] = 19 ;// Subscripting foo ( a[[ : , 0 ]] ) ;// Section int [[ * ]] b = new int [[ 100 ]] ;// 1d Multiarray int [ ] c = new int [ 100 ] ;// Ordinary Java array // (Note difference between b and c). www.hpjava.org

6. Is It Still Java? • HPJava translated to standard Java by “intelligent” preprocessor. • Does full static semantic check of source program—accepts only programs conforming to JLS 2 + HPJava extensions. • Generally preserves line numbers, so run-time exception messages refer correctly back to HPJava source—easy debugging. • Is it Java? • Pro: • Result is standard Java class file. Execute on standard JVMs (most compiler technology in standard Java goes into JVM, notjavac?) • Any standard Java class libraries can be used in an HPJava program. • Any standard Java source is a valid HPJava program (modulo reserved keywords, class names). • Con: • Extra syntax means cannot use Java IDEs to develop code! www.hpjava.org

7. Statically Compiled, Parallel Java • Very selective list. These compile to C, C++, or native code. • Manta (Amsterdam Vrije Universiteit) • “A fast, parallel Java.” • Titanium (UC Berkeley) • Adds language support for distributed shared address space. • Timber (Delft U of Technology) • Adds multidimensional arrays, tuples, foreach, … • Jade (Urbana-Champaign) • Adds message-driven parallelism, chares, code migration. www.hpjava.org

8. “JVM-based” Parallel Java • Very biased list: select representatives only! • Threads and shared memory • Libraries: java.lang.Thread, … • Syntax extensions: Jomp from Edinburgh, … • RMI-based approaches • Libraries: KaRMI from Karlsruhe, … • Syntax extensions: JavaParty, … • MPI-based approaches • Libraries: mpiJava, MPJava, JMPI, CCJ, … • Syntax extensions: HPJava, … www.hpjava.org

9. 0 1 2 a[0,0] a[0,1] a[0,2] a[1,0] a[1,1] a[1,2] a[2,0] a[2,1] a[2,2] a[3,0] a[3,1] a[3,2] a[0,3] a[0,4] a[0,5] a[1,3] a[1,4] a[1,5] a[2,3] a[2,4] a[2,5] a[3,3] a[3,4] a[3,5] a[0,6] a[0,7] a[1,6] a[1,7] a[2,6] a[2,7] a[3,6] a[3,7] 0 a[4,0] a[4,1] a[4,2] a[5,0] a[5,1] a[5,2] a[6,0] a[6,1] a[6,2] a[7,0] a[7,1] a[7,2] a[4,3] a[4,4] a[4,5] a[5,3] a[5,4] a[5,5] a[6,3] a[6,4] a[6,5] a[7,3] a[7,4] a[7,5] a[4,6] a[4,7] a[5,6] a[5,7] a[6,6] a[6,7] a[7,6] a[7,7] 1 int N = 8 ; Procs2 p = new Procs(2, 3) ; on(p) { Range x = new BlockRange(N, p.dim(0)) ; Range y = new BlockRange(N, p.dim(1)) ; int [[-,-]] a = new int [[x, y]] ; } HPJava Distributed Array Example p.dim(1) p.dim(0) www.hpjava.org

10. Parallel HPJava Syntax Procs p = new Procs2(2, 2); on(p) { Range x = new ExtBlockRange(M, p.dim(0), 1), y = new ExtBlockRange(N, p.dim(1), 1); float [[-,-]] a = new float [[x, y]] ; . . . Initialize edge values in ‘a’ (boundary conditions) … float [[-,-]] b = new float [[x,y]], r = new float [[x,y]]; // r = residuals do { Adlib.writeHalo(a); overall (i = x for 1 : N – 2) overall (j = y for 1 : N – 2) { float newA = 0.25 * (a[i - 1, j] + a[i + 1, j] + a[i, j - 1] + a[i, j + 1] ); r [i, j] = Math.abs(newA – a [i, j]); b [i, j] = newA; } HPutil.copy(a, b); // Jacobi relaxation. } while(Adlib.maxval(r) > EPS); } www.hpjava.org

11. Remarks • Ghost regions are used for algorithms with some kind of local stencil update involving neighbor elements in an array. • This certainly isn’t the only kind of application of HPJava, but it is a significant one. • By using ExtBlockRange distribution format, arrays are allocated with extensions at the edge of the local block. • One must explicitly call Adlib.writeHalo() to update these extra cells with corresponding values from neighbor processes, whenever those have been changed. • The example also illustrates a different kind of Adlib communication function called Adlib.maxval(). • Bindings to other libraries (besides Adlib) would be very desirable! www.hpjava.org

12. HPJava = HPF in Java? • Arguably the differences between the HPF programming model and HPJava programming are as interesting as the similarities: • HPF maintains a semantic equivalence with a sequential Fortran program. Preserving this equivalence causes various headaches. • HPJava is supposed to directly abstract conventional, MPI-like, SPMD programming practices—overall is just supposed to formalize the local, “data-parallel” loops in multiprocess, explicitly SPMD-parallel programs. • HPspmd approach proposed independently of Java. www.hpjava.org

13. overall • There is a strong restriction on how distributed dimensions of multiarrays may be subscripted: float [[-,-]] a = new float [[x, y]] ; … overall (i = x for 1 : N – 2) overall (j = y for 1 : N – 2) { … a [i, j] … } • A distributed array dimension may only be subscripted with a (possibly shifted) distributed index like i—typically the index of an overall construct. • This index must be distributed with the same range as the array dimension (like x). • Patterns of array use in overall much restricted compared with forall! • This is deliberate! HPspmd is “just” SPMD programming (c.f. ZPL). www.hpjava.org

14. Multigrid Application • Paper describes two parallel applications of HPJava, with programming examples. • Multigrid: fast solver for 2D Poisson equation. • Adapted from existing HPF code. • Restrict and interpolate operations transfer data up and down stack of grids of different mesh-size. • Straightforwardly expressed in HPJava using writeHalo() , remap(), and reduction communication primitives, plus overall style of parallelism. www.hpjava.org

15. Restrict Operation www.hpjava.org

16. Restrict Operation in HPJava static void restr(int npc, int npf, double fc [[-,-]], double uf [[-,-]], double ff [[-,-]], double tf [[-,-]]) { Range xf = ff.rng(0), ff = ff.rng(1); int nc = npc - 1, nf = npf - 1; Adlib.writeHalo(uf); overall(i = xf for 2 : nf - 2 : 2) overall(j = yf for 2 : nf - 2 : 2) tf [i, j] += 2.0 * (ff [i, j] - 4.0 * uf [i, j] + uf [i - 1, j] + uf [i + 1, j] + uf [i, j - 1] + uf [i, j + 1]); Adlib.remap(fc [[1 : nc - 1, 1 : nc - 1]], tf [[2 : nf - 2 : 2, 2 : nf - 2 : 2]]); } www.hpjava.org

17. CFD Application • Solves Euler equations by a finite volume approach. • Computational code based on Java Wind Tunnel applet, MIT. • Approx 2000 lines of Java, parallelized in a few days using HPJava. • “Interacting applet” demo version, as well as computational kernel for distributed memory (c.f. JG benchmark). • Applet version identical computational code, but uses a version of the Adlib communication that communicates between Java threads rather than JVMs. • See www.hpjava.org/demo.html for demo and source code. www.hpjava.org

18. www.hpjava.org

19. Preliminary Benchmark Results • System: IBM SP3 supercomputing system with AIX 4.3.3 operating system and 42 nodes. • CPU: A node has four processors (Power3 375 MHZ) and 2 gigabytes of shared memory. • Network MPI Setting: Shared “css0” adapter with User Space (US) communication mode. • Java VM: IBM’s JIT • Java Compiler: IBM J2RE 1.3.1 with “-O” option. • HPF Compiler: IBM xlhpf95 with “-qhot” and “-O3” options. • Fortran 95 Compiler: IBM xlf95 with “-O5” option. www.hpjava.org

20. www.hpjava.org

21. www.hpjava.org

22. Speedup is relatively modest. This seems to be due to the complex pattern of communication in this algorithm. www.hpjava.org

23. www.hpjava.org

24. Conclusions • HPJava is an application-oriented framework for SPMD parallel computing, using libraries acting on distributed arrays. • Implements an HPspmd parallel language model. • Also provides multiarray syntax for Java. • Performance OK, but still needs some global optimization on loop constructs, array subscript expressions, etc to be fully competitive. • Incorporates a “good” preprocessor for Java • Free and unrestricted source if you want a framework for adding experimental extensions to Java language… • All software/documentation/examples available for free download from: www.hpjava.org www.hpjava.org