1 / 26

MATLAB HPCS Extensions

MATLAB HPCS Extensions. Presented by: David Padua University of Illinois at Urbana-Champaign. Contributors. Gheorghe Almasi - IBM Research Calin Cascaval - IBM Research Siddhartha Chatterjee - IBM Research Basilio Fraguela - University of Illinois Jose Moreira - IBM Research

thao
Download Presentation

MATLAB HPCS Extensions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MATLAB HPCS Extensions Presented by: David Padua University of Illinois at Urbana-Champaign PERCS Program Review

  2. Contributors • Gheorghe Almasi - IBM Research • Calin Cascaval - IBM Research • Siddhartha Chatterjee - IBM Research • Basilio Fraguela - University of Illinois • Jose Moreira - IBM Research • David Padua - University of Illinois PERCS Program Review

  3. Objectives • To develop MATLAB extensions for accessing, prototyping, and implementing scalable parallel algorithms. • To give programmers of high-end machines access to all the powerful features of MATLAB, as a result. • Array operations / kernels. • Interactive interface. • Rendering. PERCS Program Review

  4. Uses of the MATLAB Extension • Interface for parallel libraries users. • Interface for parallel library developers. • Input to a “conventional” compiler. • Input to linear algebra compiler. • A library generator/tuner for parallel machines. • Leverage NSF-ITR project with K. Pingali (Cornell) and J. DeJong (Illinois). PERCS Program Review

  5. Design Requirements • Minimal extension. • A natural extension to MATLAB that is easy to use. • Extensions for direct control of parallelism and communication on top of the ability to access parallel library routines. It does not seem it is possible to encapsulate all the important parallelism in library routines. • Extensions that provide the necessary information and can be automatically and effectively analyzed for compilation and translation. PERCS Program Review

  6. The Design • No existing MATLAB extension had the characteristics we needed. • We designed a data type that we call hierarchically tiled arrays (HTAs). These are arrays whose components could be arrays or other HTAs. Operators on HTAs represent computations or communication. PERCS Program Review

  7. Approach • In our approach, the programmer interacts with a copy of MATLAB running on a workstation. • The workstation controls parallel computation on servers. PERCS Program Review

  8. Approach (Cont.) • All conventional MATLAB operations are executed on the workstation. • The parallel server operates on the HTAs. • The HTA type is implemented as a MATLAB toolbox. This enables implementation as a language extension and simplifies porting to future versions of MATLAB. PERCS Program Review

  9. Interpretation and Compilation • A first implementation based on the MATLAB interpreter has been developed. • This implementation will be used to improve our understanding of the extensions and will serve as a basis for further development and tuning. • Interpretation overhead may hinder performance, but parallelism can compensate for the overhead. • Future work will include the implementation of a compiler for MATLAB and our extensions based on the effective strategies of L. DeRose and G. Almasi. PERCS Program Review

  10. From: G. Almasi and D. Padua MaJIC: Compiling MATLAB for Speed and Responsiveness. PLDI 2002 PERCS Program Review

  11. Hierarchically Tiled Arrays • Array tiles are a powerful mechanism to enhance locality in sequential computations and to represent data distribution across parallel systems. • Several levels of tiling are useful to distribute data across parallel machines with a hierarchical organization and to simultaneously represent both data distribution and memory layout. • For example, a two-level hierarchy of tiles can be used to represent: • the data distribution on a parallel system and • the memory layout within each component. PERCS Program Review

  12. Hierarchically Tiled Arrays (Cont.) • Computation and communication are represented as array operations on HTAs. • Using array operations for communication and computation raises the level of abstraction and, at the same time, facilitates optimization. PERCS Program Review

  13. Using HTAs for Locality Enhancement Tiled matrix multiplication using conventional arrays for I=1:q:n for J=1:q:n for K=1:q:n for i=I:I+q-1 for j=J:J+q-1 for k=K:K+q-1 C(i,j)=C(i,j)+A(i,k)*B(k,j); end end end end end end PERCS Program Review

  14. Here, C{i,j}, A{i,k}, B{k,j} represent submatrices. The * operator represents matrix multiplication in MATLAB. Using HTAs for Locality Enhancement Tiled matrix multiplication using HTAs for i=1:m for j=1:m for k=1:m C{i,j}=C{i,j}+A{i,k}*B{k,j}; end end end PERCS Program Review

  15. A{1,1} B{1,1} A{1,2} B{2,2} A{1,3} B{3,3} A{1,4} B{4,4} A{2,2} B{2,1} A{2,3} B{3,2} A{2,4} B{4,3} A{2,1} B{1,4} A{3,3} B{3,1} A{3,4} B{4,2} A{3,1} B{1,3} A{3,2} B{2,4} A{4,4} B{4,1} A{4,1} B{1,2} A{4,2} B{2,3} A{4,3} B{3,4} Using HTAs to Represent Data Distribution and Parallelism Cannon’s Algorithm PERCS Program Review

  16. A{1,1} B{1,1} A{1,2} B{2,2} A{1,3} B{3,3} A{1,4} B{4,4} A{2,2} B{2,1} A{2,3} B{3,2} A{2,4} B{4,3} A{2,1} B{1,4} A{3,3} B{3,1} A{3,4} B{4,2} A{3,1} B{1,3} A{3,2} B{2,4} A{4,4} B{4,1} A{4,1} B{1,2} A{4,2} B{2,3} A{4,3} B{3,4} PERCS Program Review

  17. A{1,2} B{1,1} A{1,3} B{2,2} A{1,4} B{3,3} A{1,1} B{4,4} A{2,3} B{2,1} A{2,4} B{3,2} A{2,1} B{4,3} A{2,2} B{1,4} A{3,4} B{3,1} A{3,1} B{4,2} A{3,2} B{1,3} A{3,3} B{2,4} A{4,1} B{4,1} A{4,2} B{1,2} A{4,3} B{2,3} A{4,4} B{3,4} PERCS Program Review

  18. A{1,2} B{2,1} A{1,3} B{3,2} A{1,4} B{4,3} A{1,1} B{1,4} A{2,3} B{3,1} A{2,3} B{4,2} A{2,4} B{1,3} A{2,1} B{2,4} A{3,3} B{4,1} A{3,4} B{1,2} A{3,1} B{2,3} A{3,2} B{3,4} A{4,4} B{1,1} A{4,1} B{2,2} A{4,2} B{3,3} A{4,3} B{4,4} PERCS Program Review

  19. Cannnon’s Algorithm in MATLAB with HPCS Extensions C{1:n,1:n} = zeros(p,p); %communication … for k=1:n C{:,:} = C{:,:}+A{:,:}*B{:,:}; %computation A{i,1:n} = A{i,[2:n, 1]}; %communication B{1:n,i} = B{[2:n,1],i}; %communication end PERCS Program Review

  20. Cannnon’s Algorithm in C + MPI for (km = 0; km < m; km++) { char *chn = "T"; dgemm(chn, chn, lclMxSz, lclMxSz, lclMxSz, 1.0, a, lclMxSz, b, lclMxSz, 1.0, c, lclMxSz); MPI_Isend(a, lclMxSz * lclMxSz, MPI_DOUBLE, destrow, ROW_SHIFT_TAG, MPI_COMM_WORLD, &requestrow); MPI_Isend(b, lclMxSz * lclMxSz, MPI_DOUBLE, destcol, COL_SHIFT_TAG, MPI_COMM_WORLD, &requestcol); MPI_Recv(abuf, lclMxSz * lclMxSz, MPI_DOUBLE, MPI_ANY_SOURCE, ROW_SHIFT_TAG, MPI_COMM_WORLD, &status); MPI_Recv(bbuf, lclMxSz * lclMxSz, MPI_DOUBLE, MPI_ANY_SOURCE, COL_SHIFT_TAG, MPI_COMM_WORLD, &status); MPI_Wait(&requestrow, &status); aptr = a; a = abuf; abuf = aptr; MPI_Wait(&requestcol, &status); bptr = b; b = bbuf; bbuf = bptr; } PERCS Program Review

  21. Speedups on afour-processor IBM SP-2 PERCS Program Review

  22. Speedups on anine-processor IBM SP-2 PERCS Program Review

  23. Flattening • Elements of an HTA are referenced using a tile index for each level in the hierarchy followed by an array index. Each tile index tuple is enclosed within {}s and the array index is enclosed within parentheses. • In the matrix multiplication code, C{i,j}(3,4) would represent element 3,4 of submatrix i,j. • Alternatively, the tiled array could be accessed as a flat array as shown in the next slide. • This feature is useful when a global view of the array is needed in the algorithm. It is also useful while transforming a sequential code into parallel form. PERCS Program Review

  24. C{1,1}(1,1) C(1,1) C{1,2}(1,1) C(1,5) C{1,1}(1,2) C(1,2) C{1,1}(1,3) C(1,3) C{1,1}(1,4) C(1,4) C{1,2}(1,2) C(1,6) C{1,2}(1,3) C(1,7) C{1,2}(1,4) C(1,8) C{1,1}(2,1) C(2,1) C{1,1}(2,2) C(2,2) C{1,1}(2,3) C(2,3) C{1,1}(2,4) C(2,4) C{1,2}(2,1) C(2,5) C{1,2}(2,2) C(2,6) C{1,2}(2,3) C(2,7) C{1,2}(2,4) C(2,8) C{1,1}(3,1) C(3,1) C{1,2}(3,1) C(3,5) C{1,1}(3,2) C(3,2) C{1,1}(3,3) C(3,3) C{1,1}(3,4) C(3,4) C{1,2}(3,2) C(3,6) C{1,2}(3,3) C(3,7) C{1,2}(3,4) C(3,8) C{1,1}(4,1) C(4,1) C{1,1}(4,2) C(4,2) C{1,1}(4,3) C(4,3) C{1,1}(4,4) C(4,4) C{1,2}(4,1) C(4,5) C{1,2}(4,2) C(4,6) C{1,2}(4,3) C(4,7) C{1,2}(4,4) C(4,8) C{2,1}(1,1) C(5,1) C{2,2}(1,1) C(5,5) C{2,1}(1,2) C(5,2) C{2,1}(1,3) C(5,3) C{2,1}(1,4) C(5,4) C{2,2}(1,2) C(5,6) C{2,2}(1,3) C(5,7) C{2,2}(1,4) C(5,8) C{2,1}(2,1) C(6,1) C{2,1}(2,2) C(6,2) C{2,1}(2,3) C(6,3) C{2,1}(2,4) C(6,4) C{2,2}(2,1) C(6,5) C{2,2}(2,2) C(6,6) C{2,2}(2,3) C(6,7) C{2,2}(2,4) C(6,8) C{2,1}(3,1) C(7,1) C{2,2}(3,1) C(7,5) C{2,1}(3,2) C(7,2) C{2,1}(3,3) C(7,3) C{2,1}(3,4) C(7,4) C{2,2}(3,2) C(7,6) C{2,2}(3,3) C(7,7) C{2,2}(3,4) C(7,8) C{2,1}(4,1) C(8,1) C{2,1}(4,2) C(8,2) C{2,1}(4,3) C(8,3) C{2,1}(4,4) C(8,4) C{2,2}(4,1) C(8,5) C{2,2}(4,2) C(8,6) C{2,2}(4,3) C(8,7) C{2,2}(4,4) C(8,8) Two Ways of Referencing the Elements of an 8 x 8 Array. PERCS Program Review

  25. Status • We have completed the implementation of practically all of our initial language extensions (for IBM SP-2 and Linux Clusters). • Following the toolbox approach has been a challenge, but we have been able to overcome all obstacles. PERCS Program Review

  26. Conclusions • We have developed parallel extensions to MATLAB. • It is possible to write highly readable parallel code for both dense and sparse computations with these extensions. • The HTA objects and operations have been implemented as a MATLAB toolbox which enabled their implementation as language extensions. PERCS Program Review

More Related