1 / 14

Matrix Multiplication

Matrix Multiplication. To make this discussion easier we will assume square matrices The product of two n by n matrices A and B is given by Note that all valid products are of the form. Sequential Matrix Multiplication. MODULE matrix1; CONST n = 3; TYPE

star
Download Presentation

Matrix Multiplication

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Matrix Multiplication • To make this discussion easier we will assume square matrices • The product of two n by n matrices A and B is given by • Note that all valid products are of the form ICSS571 - Matrix Multiplication

  2. Sequential Matrix Multiplication MODULE matrix1; CONST n = 3; TYPE matrix = ARRAY [0..n-1],[0..n-1] OF INTEGER; VAR a,b,c : matrix; i,j,k : INTEGER; BEGIN FOR i := 1 TO n - 1 DO FOR j := 0 TO n - 1 DO c[i,j] := 0; FOR k := 0 TO n-1 DO c[i,j] := c[i,j] + a[i,k] * b[k,j]; END END END END matrix1. The complexity of this algorithm is clearly (n3). For the 3 by 3 matrix case this requires 27 multiplications Gee would’nt it be neat to do this in parallel  ICSS571 - Matrix Multiplication

  3. Dissection Time a00 a01 a02 a10 a11 a12 a20 a21 a22 b00 b01 b02 b10 b11 b12 b20 b21 b22 x = a00*b00+a01*b10+a02*b20 a00*b01+a01*b11+a02*b21 a00*b02+a01*b12+a02*b22 a10*b00+a11*b10+a12*b20 a10*b01+a11*b11+a12*b21 a10*b02+a11*b12+a12*b22 a20*b00+a21*b10+a22*b20 a20*b01+a21*b11+a22*b21 a20*b02+a21*b12+a22*b22 ICSS571 - Matrix Multiplication

  4. Parallelize • Organize the PE grid as a N x N x N cube • Place the data in the processors so that each computes a sum for one of the Cij’s so the multiplication can be done in one step • All that is left to sum the products ICSS571 - Matrix Multiplication

  5. Parallelize a02*b20 a02*b21 a02*b22 a12*b20 a12*b21 a12*b22 a22*b20 a22*b21 a22*b22 Sum Reduction a01*b10 a01*b11 a01*b12 a11*b10 a11*b11 a11*b12 a21*b10 a21*b11 a21*b12 a00*b00 a00*b01 a00*b02 a10*b00 a10*b01 a10*b02 a20*b00 a20*b01 a20*b02 ICSS571 - Matrix Multiplication

  6. The Algorithm The algorithm for parallel matrix multiplication • Load the arrays into the cube • Everyone multiplies • Do a REDUCE.SUM from back to front • Result is in the front 3x3 plane of the cube ICSS571 - Matrix Multiplication

  7. Sequential Matrix Multiplication MODULE matrix2; CONST n = 3; TYPE matrix = ARRAY [0..n-1],[0..n-1] OF INTEGER; CONFIGURATION grid [0..n-1],[0..n-1],[0..n-1]; CONNECTION front: grid[i,j,k] -> grid[0,j,k]; VAR a,b,c : grid OF INTEGER; i,j,k : INTEGER; BEGIN (* load the processor planes *) c:=a*b; SEND.front:#SUM(c,c); (* retrieve the result *) END matrix2. The complexity of this algorithm is clearly O(log2n). However, the number of processors required is O(n3) ICSS571 - Matrix Multiplication

  8. Using Fewer Processors b22 b12 b02 b21 b11 b01 b20 b10 b00 a02 a01 a00 a12 a11 a10 a22 a21 a20 ICSS571 - Matrix Multiplication

  9. Using Fewer Processors b22 b12 b02 b21 b11 b01 b20 b10 a02 a01 a12 a11 a10 a22 a21 a20 ICSS571 - Matrix Multiplication

  10. Using Fewer Processors b22 b12 b02 b21 b11 b20 a02 a12 a11 a22 a21 a20 ICSS571 - Matrix Multiplication

  11. Using Fewer Processors b22 b12 b21 a12 a22 a21 ICSS571 - Matrix Multiplication

  12. Using Fewer Processors b22 a22 ICSS571 - Matrix Multiplication

  13. Improving Efficiency a00*b00+a01*b10+a02*b20 a00*b01+a01*b11+a02*b21 a00*b02+a01*b12+a02*b22 a10*b00+a11*b10+a12*b20 a10*b01+a11*b11+a12*b21 a10*b02+a11*b12+a12*b22 a20*b00+a21*b10+a22*b20 a20*b01+a21*b11+a22*b21 a20*b02+a21*b12+a22*b22 b22 b12 b02 b21 b11 b01 b20 b10 b00 a02 a01 a00 a12 a11 a10 a22 a21 a20 ICSS571 - Matrix Multiplication

  14. Improving Efficiency ICSS571 - Matrix Multiplication

More Related