Liao Huilian SZU TI-DSPs LAB Aug 27, 2007

1. Optimizer based on particle swarm optimization and LBG (PSO-LBG) — application in vector quantization Liao Huilian SZU TI-DSPs LAB Aug 27, 2007 School of Software Engineering, Shenzhen University

2. Outline • Vector quantization (VQ) • LBG • Particle swarm optimization (PSO) • Optimizer based on PSO and LBG (PSO-LBG) • PSO-LBG • 2-dimensional simulation • Performance comparison • Conclusion • Acknowledgement School of Software Engineering, Shenzhen University

3. Outline • Vector quantization (VQ) • LBG • Particle swarm optimization (PSO) • Optimizer based on PSO and LBG (PSO-LBG) • PSO-LBG • 2-dimensional simulation • Performance comparison • Conclusion • Acknowledgement School of Software Engineering, Shenzhen University

4. Vector quantization (VQ) School of Software Engineering, Shenzhen University

5. LBG • LBG, a well-known method of VQ, was proposed by Linde, Buzo and Gray in 1980 • Apply two optimality criteria iteratively: • Nearest neighbour criterion during assigning training vectors • Centroid criterion during updating codewords (code vectors) • Drawbacks: • Local optimization • Sensitive to the selection of initial codebook School of Software Engineering, Shenzhen University

6. LBG School of Software Engineering, Shenzhen University

7. LBG School of Software Engineering, Shenzhen University

8. Outline • Vector quantization (VQ) • LBG • Particle swarm optimization (PSO) • Optimizer based on PSO and LBG (PSO-LBG) • PSO-LBG • 2-dimensional simulation • Performance comparison • Conclusion • Acknowledgement School of Software Engineering, Shenzhen University

9. Particle swarm optimization • PSO was proposed by Eberhart and Kennedy in 1995 • Advantages: • Simplicity of implementation • Few parameters • High convergence rate • Population based optimization • Remember the best location of itself (Pbest) • Remember the best experience in the swarm (Gbest) School of Software Engineering, Shenzhen University

10. Particle swarm optimization School of Software Engineering, Shenzhen University

11. Outline • Vector quantization (VQ) • LBG • Particle swarm optimization (PSO) • Optimizer based on PSO and LBG (PSO-LBG) • PSO-LBG • 2-dimensional simulation • Performance comparison • Conclusion • Acknowledgement School of Software Engineering, Shenzhen University

12. PSO-LBG • Based on conventional PSO and LBG algorithms • PSO-LBG • Structure of particle • Particle-pair model • Updating process • Apply in Vector Quantization (VQ) School of Software Engineering, Shenzhen University

13. Structure of particle School of Software Engineering, Shenzhen University

14. Updating model School of Software Engineering, Shenzhen University

15. Updating process • PSO-LBG performs three steps at each iteration: • Step1: Basic PSO operations • Step2: Classical vector quantizer, i.e. LBG algorithm • Step3: Deal with codewords “flying” over the boundary of training vector space School of Software Engineering, Shenzhen University

16. Step1－Basic PSO operations • Difference between PSO-LBG and PSO • Velocity updating: (additive inertia weight ) • The parameters, and are much smaller than general PSO-based algorithms • Apply a particle-pair instead of a large number of particles School of Software Engineering, Shenzhen University

17. Why small parameters? • One point larger parameters • The solution of PSO-LBG represents N points in the training vector space School of Software Engineering, Shenzhen University

18. Why just two particles? • Three particles consisting of two codewords: P1={y1, y2}; P2={y2, y1} and P3={y3, y4}. P3 has a poorer performance • During the following iterations, particle P1andP2 are comparative • The fly direction of particle P3 is uncertain School of Software Engineering, Shenzhen University

19. Why just two particles? Stable convergence Unstable convergence School of Software Engineering, Shenzhen University

20. Updating steps 2 & 3 • Apply LBG with only 3 iterations to avoid converging early • Deal with the codewords “flying” over the boundary of search space: Replace this kind of codeword with the training vector that has higher distortion School of Software Engineering, Shenzhen University

21. Demonstration in 2-dimensional space • Three objectives PSO-LBG intends to achieve: • Disperse codewords • Move towards global optimum codebook • Codewords are settled reasonably both in high density regions and low density areas of training vectors space School of Software Engineering, Shenzhen University

22. Demonstration in 2-dimensional space LBG =61.26 Initial codebook =561.15 PSO-LBG =46.85 School of Software Engineering, Shenzhen University

23. Performance comparison • Performance is evaluated by and PSNR • : Mean square error between the training vectors and corresponding nearest codewords • PSNR: Peak signal to noise ratio School of Software Engineering, Shenzhen University

24. Performance comparison • Comparison is conducted among: • LBG • Fuzzy k-means (FKM) • Fuzzy reinforced learning vector quantization (FRLVQ) • FRLVQ-FVQ: Apply FRLVQ as the pre-process of FVQ • PSO-LBG School of Software Engineering, Shenzhen University

25. Experimental images Pepper Kgirl Lena Cameraman School of Software Engineering, Shenzhen University

26. PSNR comparison on Lena School of Software Engineering, Shenzhen University

27. Convergence comparison on Lena School of Software Engineering, Shenzhen University

28. Computation time on Lena School of Software Engineering, Shenzhen University

29. Codebook characteristic on Lena School of Software Engineering, Shenzhen University

30. PSNR comparison on pepper School of Software Engineering, Shenzhen University

31. PSNR comparison on cameraman School of Software Engineering, Shenzhen University

32. PSNR comparison on Kgirl School of Software Engineering, Shenzhen University

33. Conclusion • Experimental results demonstrate that PSO-LBG Outperforms existing algorithms in the field of vector quantization • Future work • Application in gene clustering School of Software Engineering, Shenzhen University

34. Acknowledgement • My supervisor: Prof. Ji • All of you School of Software Engineering, Shenzhen University

35. Thank you! School of Software Engineering, Shenzhen University