A High-Level Framework for Parallelizing Legacy Applications for Multiple Platforms

1. A High-Level Framework for Parallelizing Legacy Applications for Multiple Platforms Ritu Arora Texas Advanced Computing Center Email: rauta@tacc.utexas.edu

2. Outline Motivation and Goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

3. Plenty of Parallel Programming Languages and Paradigms MPI OpenMP CUDA OpenCL Implicitly Parallel Languages (X10, Fortress, SISAL) PGAS languages (UPC, Co-Array Fortran) Offload programming for MIC SHMEM Cilk/Intel Cilk Plus Charm++ HPF Hybrid programming (MPI + OpenMP)

4. There is a need to develop a tool (high-level framework) that offers a low-risk way for domain-experts to try HPC but first… CUDA help OpenMP Help MPI Help

5. … Understanding the Mindset of the User Community is Important Source: Understanding the High Performance Computing Community: A Software Engineer’s Perspective, Basiliet al. “…the history of HPC is littered with new technologies that promised increased scientific productivity but are no longer available.” “A new technology that can coexist with older ones has a greater chance of success than one requiring complete buy-in at the beginning.” “For many frameworks, a significant barrier to their use is that you can’t integrate them incrementally.” “Frameworks provide programmers a higher level of abstraction, but at the cost of adopting the framework’s perspective on how to structure the code.”

6. Standard and Non-Standard Steps for Parallelization that are Repeatable • Examples of standard steps in developing an MPI application (common in all MPI programs) • Every MPI program has #include "mpi.h" • Every MPI program has MPI_InitandMPI_Finalizefunction calls • Non-standard steps in developing an MPI application • for-loop parallelization, data distribution, mapping of tasks to processes, and orchestration of exchange of messages • Steps for splitting the work in a for-loop amongst all the processes in MPI_COMM_WORLD are standard for a given load-balancing scheme

7. Goals Develop a high-level framework for semi-automatic parallelization that can leverage the investment made in legacy applications The framework should be built on top of successful programming paradigms like MPI, OpenMP, and CUDA Provide support for incremental parallelism through the framework Abstract the standard and non-standard steps in parallelization

8. Outline Motivation and Goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

9. How Does the Framework Work?

10. Providing Specifications Through Hi-PaL (1) • General Structure of Hi-PaL Code to Generate MPI Code • General Structure of Hi-PaL Code to Generate OpenMP Code

11. Providing Specifications Through Hi-PaL (2) A set of Hi-PaL API has been developed for precisely capturing the end-users’ specifications at a high-level

12. Parallelizing Poisson solver (1) • Code snippet of serial Poisson Solver Code

13. Parallelizing Poisson solver (2) • Hi-PaLCode to Generate MPI Code for Poisson Solver

14. Generated MPI Code for Poisson Solver (1)

15. Generated MPI Code for Poisson Solver (2)

16. Snippet of Exchange Template for MPI template <typename T> T** exchange(T** data, intnrows, intncols, int P, int Q, int p, int q, MPI_Commcomm2d, MPI_Commrowcomm, MPI_Commcolcomm) { //other code above. Create datatype for the recvtype code below MPI_Type_vector(nrows-2, 1, ncols, datatype, &temptype); MPI_Type_extent(datatype, &sizeoftype) ; intblens[2] = {1, 1}; MPI_Aintdispl[2] = {0, sizeoftype}; MPI_Datatype types[2] = {temptype, MPI_UB}; MPI_Type_struct (2, blens, displ, types, &vectype); MPI_Type_commit(&vectype); MPI_Cart_shift(rowcomm, 0, -1, &prev, &next); MPI_Cart_shift(colcomm, 0, -1, &down, &up); // send and receive the boundary rows MPI_Irecv(&data[0][1], ncols-2, datatype, up, 0, …); MPI_Irecv(&data[nrows-1][1], ncols-2, datatype, down, 0, …); MPI_Isend(&data[1][1], ncols-2, datatype, up, 0, colcomm, …); MPI_Isend(&data[nrows-2][1], ncols-2, datatype, down, …); …}

17. Providing the Specifications Through Command-Line Interface

18. Providing the Specifications Through Graphical User Interface

19. Currently Available Support For… Support for Fortran will be added …

20. Time for demos Using the framework through GUI Using the framework through Hi-PaL

21. Outline Motivation and Goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

22. Results: Poisson Solver (Hi-PaLbased MPI)

23. Results: Genetic Algorithm for Content Based Image Retrieval (Hi-PaL based MPI & OpenMP)

24. Results: Seismic Tomography Code (GUI-based CUDA)

25. Results: Circuit Satisfiability Code (GUI-based OpenMP + Offload)

26. Outline Motivation and Goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

27. Summary of Features & Benefits of the Framework • Enhances the productivity of the end-users in terms of the reduction in the time and effort • reduction in manual effort by over 90% while ensuring that the performance of the generated parallel code is within 5% of the sample hand-written parallel code • Leverages the knowledge of expert parallel programmers • Separates the sequential and parallel programming concerns while preserving the existing version of sequential applications

28. Outline Motivation and goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

29. Project Status

30. Outline Motivation and goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

31. Future Work • Migrate from DMS to Rose source-to-source compiler and integrate all the interfaces • Usability studies: • Should be able to rank the user preferences for the interface • Prioritize the development effort • Integration with PerfExpert and Eclipse • Ability to handle irregular meshes and specify pipeline mode of communication through GUI • Address the demands for • a directives-based interface • the option of editing the log-file to repeat the code-generation process without going through the GUI

32. Outline Motivation and goals Overview of the Framework with demos Results Features and Benefits Project Status Future Work Conclusion Q & A

33. Conclusion • Through this research and development effort, we have demonstrated • an approach for lowering the adoption barriers to HPC by raising the level of abstraction of parallel programming • an interactive tool for teaching parallel programming: think “Alice” and “DrJava” • the usage of multiple interfaces to accommodate the preferences of the user-community: one size does not fit all • that it is possible to achieve abstraction and performance at the same time

34. Acknowledgement On behalf of my co-authors, student and XSEDE intern (Julio Olaya), I would like to thank NSF, XSEDE and TACC!

35. Thanks for listening! Any questions, comments or concerns? Contact: rauta@tacc.utexas.edu