1 / 24

A Fully saturated OpenCL Particle swarm optimizer

A Fully saturated OpenCL Particle swarm optimizer. Donald Pupecki. Why GPUs?. GPUs are scaling faster than CPUs[1] Intel recently wrote a paper claiming a two year old GPU was “only 10x faster” than their top-end CPU. [2]. Why OpenCL?. Portable. Can run on most parallel platforms CPU

allayna
Download Presentation

A Fully saturated OpenCL Particle swarm optimizer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Fully saturated OpenCL Particle swarm optimizer Donald Pupecki

  2. Why GPUs? • GPUs are scaling faster than CPUs[1] • Intel recently wrote a paper claiming a two year old GPU was “only 10x faster” than their top-end CPU. [2]

  3. Why OpenCL? • Portable. • Can run on most parallel platforms • CPU • GPU • FPGA • Cell B.E. • Industry Wide Support • Binding For many languages

  4. Why PSO? • Particle Swarm Optimization • Created in 1995 by Kennedy, Eberhart , Shi[5][6] • Inspired by BOIDS, a bird flocking simulation by Craig Reynolds[3][4] • Provides good exploration, exploitation sometimes requires parameter tuning. [7] • A subset of Swarm Intelligence, which is itself a subset of Evolutionary Computation • Video: http://vimeo.com/17407010

  5. Is it parallelizable? • The PSO, like most evolutionary computations is iterative. • General process: Within each iteration there is a lot to do.

  6. Parallel Gotchas • Bank conflicts[18] • Divergent Warps • Struct Sizes • Coalescing • Debugging • SIMD • No Branch Prediction

  7. Goals: Implement More Test Functions • The previous version of this work only implemented 4 test functions. • More non-parabolic test functions could help confirm the soundness of the algorithm.

  8. Test Functions Implemented [14][15]

  9. Goals: Shift Test Functions • PSOs have a tendency to flock to the middle. [10] • Remedied by “Region Scaling”[10][11][12] or “Center offset” approach. • Used center offset with an offset of 2.5

  10. Goals: More Parallelism • Version 1.0 only does fitness calculations over Num Particles. • Dimensional Calculations almost all contain products or sums over all dimensions and should be done in parallel and combined with a reduction.

  11. Goals: Move Everything to Card • Less than 1/10th of the time in the 1.0 version was spent doing calculation.

  12. OCLPSO 1.1 • Get everything to the card! • Needed to implement a LCRNG on card. • Eliminated large data transfers. • Added hill climber to fall back on. • Got inconsistent results. • Note on synchronizations. Barrier, Mem Fence, FP Atomics, Compute Capability 2.x and the lack of global sync.

  13. OCL PSO 2.0 • Rewrite. • Introduced the Fully Saturated concept. • Compute Units are now self contained. • All dimensions of all particles of all swarms are evaluated in parallel.

  14. OCLPSO 2.0 vs. CUDA PSO 2.0 • The CUDA PSO[16] [17] was submitted for the GECCO competition on GPGPGPU.com • It features a ring topology for its bests calculation.

  15. OCLPSO 2.0 vs. SPSO • The 2006 Standard PSO was written by Clerc and Kennedy. • Pretty frills-free compared to newer (2010) Standard PSO codes or TRIBES. • Non-parallel but extremely efficient.

  16. Conclusions • OCL PSO 2.0 performs competitively as compared to CUDA PSO and trumps SPSO. • This is despite its lack of neighborhood bests(it uses only a global best). • This is also without the hill climb component as I needed to strip as much complexity out as I could in the final rewrite.

  17. Future work • Re-implement hill climber, it was a great help for peaks. • Re-add neighborhoods. • Implement on newest generation of CUDA 2.x cards supporting global floating point atomics • Implement TRIBES on the GPU.

  18. References [1] Nvidia, “TESLA DPU computing: Supercomputing at 1/10th the Cost.” http://www.nvidia.com/docs/IO/96958/Tesla_Master_Deck.pdf , 2011 [2] Nvidia, “Why Choose Tesla,” http://www.nvidia.com/object/why-choose-tesla.html, 2011 [3] BOIDS paper or wiki:boids http://www.red3d.com/cwr/boids/ [4] Reynolds, Craig (1987), "Flocks, herds and schools: A distributed behavioral model.", SIGGRAPH '87: Proceedings of the 14th annual conference on Computer graphics and interactive techniques (Association for Computing Machinery): 25–34 [5] Kennedy, J.; Eberhart, R. (1995). "Particle Swarm Optimization". Proceedings of IEEE International Conference on Neural Networks. IV. pp. 1942–1948. doi:10.1109/ICNN.1995.488968. http://www.engr.iupui.edu/~shi/Coference/psopap4.html. [6] Shi, Y.; Eberhart, R.C. (1998). "A modified particle swarm optimizer". Proceedings of IEEE International Conference on Evolutionary Computation. pp. 69–73. [7] Clerc, M.; Kennedy, J. (2002). "The particle swarm - explosion, stability, and convergence in a multidimensional complex space". IEEE Transactions on Evolutionary Computation [8] M. Nobile. “Particle Swarm Optimization”, http://vimeo.com/17407010, 2011

  19. References – Cont. [9] M. Clerc, Confinements and Biases in Particle Swarm Optimization. http://clerc.maurice.free.fr/pso/ , 2006. [10] C. K. Monson and K. D. Seppi, "Exposing Origin-Seeking Bias in PSO," presented at GECCO'05, Washington, DC, USA, 2005, pp. 241-248. [11] P. J. Angeline. Using selection to improve particle swarm optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 1998), Anchorage, Alaska, USA, 1998. [12] D. K. Gehlhaar and D. B. Fogel. Tuning evolutionary programming for conformationally flexible molecular docking. In Evolutionary Programming, pages 419–429, 1996. [13] D. Pupecki, “OpenCL PSO, Development, Benchmarking, Lessons, Future Work,” 2011. http://web.cs.sunyit.edu/~pupeckd/oclpsopres.pdf [14] Molga, M., Smutnicki, C., 2005. “Test functions for optimization needs”, http://www.zsd.ict.pwr.wroc.pl/files/docs/functions.pdf [15] Pohlheim, Hartmut. Genetic and Evolutionary Algorithm Toolbox for usewith MATLAB . Technical Report, Technical University Ilmenau, 1998. [16] L. Mussi, F. Daolio, and S. Cagnoni. Evaluation of parallel particle swarm optimization algorithms within the CUDA architecture. Information Sciences, 2011, in press.

  20. References – Cont. [17] L. Mussi, Y.S.G. Nashed, and S. Cagnoni. GPU-based Asynchronous Particle Swarm Optimization. Proc.GECCO 2011, 2011. [18] NVIDIA, OpenCL Programming Guide for the CUDA Architecture, http://developer.download.nvidia.com/compute/cuda/3_1/toolkit/docs/NVIDIA_OpenCL_ProgrammingGuide.pdf, 2011 [19] M. Clerc. TRIBES - un exempled’optimisation par essaimparticulaire sans param`etres de contrˆole. In Optimisation par EssaimParticulaire (OEP 2003), Paris, France, 2003.

More Related