Parallel Simulated Annealing with Adaptive Neighborhood determined by GA

1. Parallel Simulated Annealing with Adaptive Neighborhood determined by GA Doshisha University, Kyoto, Japan Mitsunori MIKI Tomoyuki HIROYASU ○ Toshihiko FUSHIMI

2. Introduction • Optimization problems become more complicated and larger. • Heuristic search • Simulated Annealing (SA) based on the simulation of the physical process “annealing”. • GA, CA, NN etc. Important matters • Parallelization • Adaptive parameter tuning

3. Algorithm of Simulated Annealing high Algorithm Energy 1. Generation 2. Judge Transition low Design space Metropolis probability 3. Cooling good acceptance 1 -⊿E Temperature bad acceptance Exp( ) (⊿E = Enext - Enow)

4. Neighborhood range The neighborhood range in the continuous Euclid space is the extent for generating next solution. • Too large neighborhood range • Can’t search optimum effectively. • The range has to be small. • Too small neighborhood range • Often trapped in a local minimum. Global optimum • The range has to be large.

5. Background For the control of the neighborhood range, some method are proposed. • The adaptive neighborhood mechanism. [Corana 1987] • The advanced adaptive neighborhood mechanism. [Miki 2002] These methods control the neighborhood range using an appropriate acceptance ratio. This type of adaptive neighborhood method is very effective and useful, but the target acceptance ratio should be determined experimentally. Propose a new adaptive neighborhood mechanism

6. Purpose Controlling the neighborhood range adaptively during the search. Parallel Simulated Annealing with Adaptive Neighborhood determined by Genetic Algorithm (PSA/ANGA) Characteristics • This method is Parallel model. • This method parallels neighborhood ranges on each processes. • This neighborhood range is controlled by GA.

7. Effect of Neighborhood Ranges • The neighborhood range has a significant effect on the accuracy of the solution. • In order to verify this effect, some numerical experiments were carried out with various fixed neighborhood ranges. Fixed neighborhood range Search space large Compare the qualities of the solutions. Neighborhood range Obtain the effect of the neighborhood ranges. small

8. Test problems Rastrigin function Mathematical test functions Griewangk function Rosenbrock function Rastrigin Griewangk Rosenbrock

9. Appropriate neighborhood range The neighborhood range has a significant effect on the performance of SA. Rastrigin Appropriate neighborhood range Good solution

10. Appropriate neighborhood range The neighborhood range has a significant effect on the performance of SA. Appropriate neighborhood range Griewangk Good solution

11. Appropriate neighborhood range The neighborhood range has a significant effect on the performance of SA. Appropriate neighborhood range Rosenbrock Good solution

12. Concept of PSA/ANGA There are the appropriate neighborhood ranges in SA when solving the continuous optimization problems. • The appropriate neighborhood ranges depend on problems. • It is difficult to find the appropriate neighborhood ranges in advance. PSA searches the solution with various neighborhood range. The neighborhood range determined adaptively by GA. Parallel Simulated Annealing with Adaptive Neighborhood determined by Genetic Algorithm (PSA/ANGA)

13. Algorithms of PSA/ANGA 1 Fitness = Energy Multiple SA processes searches the solution with various neighborhood range. GA operators are applied on neighborhood ranges. large Neighborhood range small

14. Abstract of numerical experiments PSA/ANGA is compared with a parallel SA with optimum fixed neighborhood range, PSA/FN. Comparative method Optimum fixed neighborhood range Parallel SA with Fixed Neighborhood (PSA/FN) Use the optimum fixed neighborhood range determined by preliminary numerical experiments.

15. Parameters used

16. Performance of the proposed method Proposed method The proposed method, PSA/ANGA, provides better performance than PSA/FN in all problems.

17. History of Neighborhood range • History of the neighborhood ranges in 32 SA processes. • The appropriate neighborhood range varies dynamically during the search. Rastrigin The appropriate neighborhood range is automatically determined using GA.

18. History of energy (Rastrigin) • The proposed method, PSA/ANGA, shows fast convergence of the energy and obtains lower energy than PSA/FN. • Accuracy of the solution improves because the neighborhood ranges were changed adaptively.

19. Conclusions A new Parallel Simulated Annealing method with adaptive neighborhood range mechanism is proposed. Parallel SA with Adaptive Neighborhood determined by Genetic Algorithm (PSA/ANGA) The appropriate neighborhood range varies according to the condition of the search. The proposed method adapts to these appropriate neighborhood ranges. PSA/ANGA shows good performance on the some test functions. The method is effective in SA for continuous optimization problems.

20. questions and answers Thank you for your kind attention.