Optimisation bio-inspirée Julien Marot julien.marot@univ-amu.fr Institut Fresnel

1. OptimisationMaster TSI Optimisation bio-inspirée Julien Marot julien.marot@univ-amu.fr Institut Fresnel julien.marot@fresnel.fr

2. Table of contents 1 J. Marot CM 12 h TD 8 h Z. Li CM 8 h TD 4 h • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • Mixed GWO • The phase emphasizer and the adaptive GWO • Performances • Conclusion

3. 1. Definitions 2 Optimization problem: Maximizing or minimizing a function or a problem by systematically choosing input values from within an allowed set and computing the value of the function. Problem to optimize Param 1 Param 2 Fitness … … Param n Optimisation tool

4. Bio-inspiredalgorithm Swarmoptimization 3 Problem to optimize Param 1 Param 2 Fitness … … Param n Wolf’s position Wolf’s value Bird’s value Bird’s position

5. 1. Definitions 4 Bio-inspired optimization: Optimization methods inspired from natural process. Good capacity in solving computationally expensive problems, with limited function evaluation. ex: Genetic Algorithm (GA) Swarmoptimization: Simulate the behaviour of a group of particles ex: ParticleSwarmOptimization (PSO), AntColonyAlgorithm (ACO) Grey Wolf Optimizer (GWO)

6. Table of contents 5 • Definitions • Particle Swarm Optimization • The Classic GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

7. 2. Particle Swarm Optimization Swarm of birds 6

8. 7 • Gets inspiredfrom social interaction of animals • Uses a number of agents (particles) that constitute a swarm moving around in the search space looking for the best solution • Each particle is treated as a point in a N-dimensional space which adjusts its “flying” according to its own flying experience as well as the flying experience of other particles • Each particles affords: • Velocity • Personal flying experience • Global flying experience of the swarm A particle has to decidewheresheshould go

9. In what direction is the food ? 8

10. Update rules For eachparticle Update the velocity  Then move , MyPreviousvelocity Mypersonal best position The global best position of the wholeswarm

11. Table of contents 10 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

12. 3. The classical Grey Wolf Optimizer (GWO) 11 Mimics the hunting behavior of a wolf pack one wolf = one particle prey = global minimum

13. Wolf pack hierarchy 12

14. Exploration vs Exploitation 13

15. The algorithm’s workflow 14 Computefitnesses Select α, β and δ Createwolfpack iter=1 Start Computea (Phase distinguisher) Update wolves’ position iter++ Computefitnesses Update leaders no yes End

16. Bio-inspired optimization: the lock awaken 15 About the classical GWO: Designed to solve fully continuous problems Robust to local minima About bio-inspired optimization: No method for non-binary or non-specific discrete problems No method for mixed problems (continuous + discrete) Create a bio-inspiredoptimizationmethodbased on GWO to tackle mixed problems

17. Table of contents 16 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

18. Table of contents 17 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

19. 4. The mixedGWO a. Improved Discrete GWO 18 DiscreteProblem to optimize Param 1 Param 2 Fitness … … Param n Wolf’s position Wolf’s value Wolf’s indexes Exemple:

20. 19 Computefitnesses Select α, β and δ Createwolfpack iter=1 Start Computea (Phase distinguisher)

21. 20 Computefitnesses Select α, β and δ Createwolfpack iter=1 Start Computea (Phase distinguisher) Select randomwolvesρ1 and ρ2 yes a>1? no Update wolves’ position

22. Searchspace: continous vs discrete 21

23. 22 ρ2 α δ For each wolf q: Select a leader l to follow: ρ1 β q (exploitation) (exploration)

24. 23 4 ? δ For each wolf q: Select a leader l to follow: Select the displacement factor: 12 ? 42 ? q

25. 24 δ 42 For each wolf q: Select a leader l to follow: Select the displacement factor: Compute the new indexes: q

26. 25 Computefitnesses Select α, β and δ Createwolfpack iter=1 Start Computea (Phase distinguisher) Select randomwolvesρ1 and ρ2 yes a>1? iter++ no Update wolves’ position yes Computefitnesses Update leaders no End

27. Table of contents 26 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

28. 27 b. Global Continuous GWO Computefitnesses Select α, β and δ Createwolfpack iter=1 Start Computea (Phase distinguisher) Update wolves’ position iter++ Computefitnesses Update leaders no yes End

29. 28 Computefitnesses Select α, β and δ Createwolfpack iter=1 Start Computea (Phase distinguisher) Select randomwolvesρ1 and ρ2 yes a>1? iter++ no Update wolves’ position yes Computefitnesses Update leaders no End

30. Table of contents 29 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

31. 30 c. mixedGWO ParameterKiisdiscrete? no Wolf q=1 Start iter=1 i=1 yes Update with ID-GWO Update with GC-GWO iter++ i++ yes i ≤ N ? no yes q ≤ Q ? no End q++

32. Table of contents 31 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

33. d. The phase emphasizer and the adaptive GWO 32 Modified GWO (mGWO): N. Mittal, U. Singh et B. Singh Sohi, “Modified grey wolf optimizer for global engineering optimization", 2016.

34. 33 Adaptive mixed GWO (amixedGWO):

35. Table of contents 34 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Conclusion

36. e. Performances 35 Tested on 9 fixed-dimension multimodal benchmark functions in continuous space continuous problems

37. 36 Average minima obtained on continuousfunctions

38. 37 Average minima obtained on continuousfunctionsfrom CEC2014

39. 38 discrete problems 3 discrete variables Average and Medium values obtained on discretefunctions

40. Table of contents 39 • Definitions • Particle Swarm Optimization • The Classical GWO • The mixedGWO • Improved discrete GWO • Global continuous GWO • mixedGWO • The phase emphasizer and the adaptive GWO • Performances • Applications • Conclusion

41. First Application: SVM+LBP training for gender recognition 40 Gender recognition: automatically label an face as a woman’s or a man’s face Needed: A machine learning algorithm (SVM) Training images (FERET) Image representation (LBP)

42. 41 Problem to minimize: False recognition rate (FRR) of a SVM trained for gender recognition For non-separable data: Gaussian (RBF) kernel:

43. 42 Adaptive version of the classical GWO: Second run η2 > 1 and η2 < η1 First run η1 > 1 Thirdrun η3 < 1 Start End

44. Second Application: Joint denoising and unmixingof multispectral images Problem’s definition 43 Multispectral image: Image composed of more than 3 channels original Classic image (RGB) Multi-spectral image Spatial dimension Spectral dimension Spatial dimension

45. 44 Spectra unmixing: Identification of endmembers (e.g. spectra of wood or ground) Image denoising: Process of removing noise (“cleansing”) from an image

46. 45 Goal: Find the best balance between image denoising and spectra unmixing using the mixedGWO Criteria to minimize for a multispectral image : Parameters to estimate: um For the denoising: For the unmixing:

47. 46 School case Input image of size and Reference computed with Multiway Wiener Fourier (MWF) with and Expectedspectrum obtainedwith:

48. 47

49. 48 Estimatedparameters Obtained Spectrum ReconstructedError (RE)

50. Realistic case 49 Input image of size and Reference computed by applying a Wiener filteringprocess in Fourier domain to the impaired image Noise-free image Impaired with Reference image