1 / 19

CSE 690: GPGPU Lecture 7: Matrix Multiplications

CSE 690: GPGPU Lecture 7: Matrix Multiplications . Klaus Mueller Computer Science, Stony Brook University. Basic Concept. Triple loop. GPU Algorithms. First algorithm: render a rectangle of size NxN represent the matrices as NxN textures each (i,j) is then a fragment

twyla
Download Presentation

CSE 690: GPGPU Lecture 7: Matrix Multiplications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CSE 690: GPGPULecture 7: Matrix Multiplications Klaus Mueller Computer Science, Stony Brook University

  2. Basic Concept • Triple loop

  3. GPU Algorithms • First algorithm: • render a rectangle of size NxN • represent the matrices as NxN textures • each (i,j) is then a fragment • each fragment program is a loop or an unrolled loop -> may get too long • must pull in the same data many times -> poor data reuse, needs bandwidth • makes no use of 4-way RGBA parallelism -> wastes speedup

  4. GPU Algorithms • Better algorithm: • use RGBA channels, pack a 2x2 submatrix • use swizzling to facilitate data reuse • swizzling improves fragment code length by factor 2 • may need multiple passes for larger matrices

  5. GPU Algorithms • Using multi-texturing • requires l passes

  6. GPU Algorithms • Can use RGBA parallelism as well • each texel represents a 2x2 submatrix • use swizzling as usual • needs l/2 passes

  7. GPU Algorithms • Instead of a 2x2 submatrix, pack 4x1 column vectors • makes 4-times reuse of texels read from B, but uses texels from A only once

  8. GPU Algorithms • Instead of a 2x2 submatrix, pack 4x1 column vectors • 6 fetches are needed for 4 mad’s (mult-add’s) -> 1.5 times more than before • but less rows and columns are accessed per pass -> improves cache hit frequency

  9. GPU Algorithms • Originally only compute one product per shader • practically can unroll the loop 3-6 times (compute 3-6 products) • maximal fragment program length is the limit • reduces the number of passes required

  10. Reality Check • Would like to compare CPU and GPU efficiencies for GPGPU tasks • The task of matrix multiplication is insightful here • features much data reuse • graphics programs are generally more stream-like and have less data reuse • this may lead to some limitations

  11. Platforms • Pentium 4 3Ghz CPU, 512KB L2 cache • 12 GFLOPS peak compute • 44.1GB/sec cache BW • Using sgemm routine from ATLAS package • NVIDIA • GeForce 5900 Ultra • GeForce 6800 Ultra • ATI • Radeon 9800 XT • Radeon X800 XT PE

  12. Performance

  13. Bandwidth vs. Peak FLOPS

  14. Analysis • Currently: • GPUs can fetch 16 floats and perform 16 4-component mad’s per clock • our app fetches 8 floats to perform one 4-component mad -> not enough computations • need more math ops per float fetched (> 8)

  15. Analysis • Pentium processors have large L1 caches to boost memory bandwidth (bw) • bw / compute ratio better • main reason for only small performance gain achieved with GPUs

  16. Analysis • Pentium processors have large L1 caches to boost memory bandwidth (bw) • bw / compute ratio better • main reason for only small performance gain achieved with GPUs • for matrix multiplications

  17. Analysis • Expectations • make sure that there is enough arithmetic per data item fetched • lots of data resuse in the algorithm / task will make the CPU look better • streaming data OK -> they don’t “suffer” from reuse • matrix multiplication is an excellent reality-check example

  18. Analysis • What do GPUs need: • bigger caches to enable larger blocks • currently there are enough registers to store a 6x6 submatrix • but currently shaders can only produce a small number of outputs -> limits the amount of blocking • Provide full-floating point accumulator registers • Widen path between texture and register files

  19. References • E. Larsen and D. McAllister, “Fast matrix multiplies using graphics hardware,” Supercomputing 2001. • J. Hall, N. Carr and J. Hart, “Cache and bandwidth aware matrix multiplication on the GPU,” Tech Report UIUCDCS-R-2003-2328-1 • K. Fatahalian, J. Sugerman, and P. Hanrahan, “Understanding the efficiency of GPU algorithms for matrix-matrix multiplication,” Graphics Hardware Workshop 2004.

More Related