A Neural Network Implementation on the GPU

1. A Neural Network Implementation on the GPU By Sean M. O’Connell CSC 7333 Spring 2008

2. Introduction • Neural Network processing • CPUs vs GPUs • Modern GPU parallelization • Applying GPU architecture to NN • Exploiting parallel NN node computations • Mappings to GPU

3. NN Implementation Details • Each layer fully connected to next one • Step activation function • Back-propagation

4. GPU Architecture • Very different from CPU • Memory layout • Textures • Vertex arrays • Matrices • Devise a new GPU framework / arch.

5. Node Weights

6. Node Output • Node input uses previous layer’s output

7. Neural Network Layers • Back-propagation error data stored in ‘error’ texture

8. Implementation Details • OpenGL 2.0 • Pixels plotted to screen • GLSL pixel shaders • Frame Buffer Objects • Vertex Buffer Objects

9. Pseudo Code

10. Test Hardware • Intel Core Duo 2.2Ghz • 2GB DDR600 RAM • Nvidia Geforce 7900GTX 512MB

11. Results GPU Neural Network Training CPU Neural Network Training

12. Results

13. Conclusion • GPU 157x FASTER for 4000 nodes • Lots of improvements can be made • GPU well suited for A.I.

14. Questions? References [1] Machine Learning. Tom M. Mitchell. The McGraw Hill Companies, 1997. [2] OpenGL – The Industry Standard for High Performance Graphics. http://www.opengl.org