Omni-Optimizer A Procedure for Single and Multi-objective Optimization

1. Omni-OptimizerA Procedure for Single and Multi-objective Optimization Prof. Kalyanmoy Deb and Santosh Tiwari

2. Motivation • Generic Programming Practices • Unified algorithm for all types of optimization problems • An efficient implementation of NSGA-II framework (procedure) • Developing an efficient and self-adaptive optimization paradigm

3. Literature Survey • CHC (Cross generation elitist selection, Heterogeneous recombination, Cataclysmic mutation) – Explicit Diversity • GENITOR (Steady state GA), more like (µ+1)-ES so far as selection mechanism is concerned. – High selection pressure • NPGA (Niched Pareto Genetic Algorithm), uses sharing parameter σshare – # of niches obtained depend on the sharing parameter

4. Literature Survey contd… • PESA (Pareto Envelope-based Selection Algorithm), Hyper-grid division of phenotypic space, selection based on crowding measure • NSGA-II (Non-dominated Sorting Genetic Algorithm) • SPEA2 (Strength Pareto Evolutionary Algorithm), Fine grained fitness assignment mechanism utilizing density information, Only archive members participate in mating – Excellent Diversity in phenotypic space • NCGA (Neighborhood Cultivation Genetic Algorithm), used neighborhood crossover, based on NSGA-II and SPEA2 • RPSGAe (Reduced Pareto Set Genetic Algorithm with elitism) • ENORA (Evolutionary Algorithm of Non-dominated Sorting with Radial Slots)

5. Salient Features of the Algorithm • Based on NSGA-II framework • Based on the concept of Pareto dominance • Incorporates elitism • Explicit diversity preserving mechanism • Can be used for single-objective as well as multi-objective problems • Can be used for uni-global as well as multi-global problems • Independent of the number of niches that an optimization problems exhibits

6. Moving beyond NSGA-II • Restricted Selection Scheme • Tournament selection based on usual domination • Non-dominated sorting based on epsilon dominance • Crowding Distance Assignment • Genotypic as well as Phenotypic space niching • Choose best members from above average population • Remove worst members from below average population • More robust recombination and variation operators • Two point crossover for binary variables • Highly disruptive real variable mutation

7. Restricted Selection • Helps in preserving multi-modality • Experiments show that it gives faster overall convergence

8. Epsilon Domination Principle • A finite percentage (based on function value) of solutions assigned a particular rank • Allows somewhat inferior solutions to remain in the population • Provides guaranteed diversity • Helps to obtain multi-modal solutions in case of single objective problems • Epsilon is generally taken in the range 10-3 ~ 10-6

9. Modified Crowding Distance • Genotypic as well as Phenotypic space niching

10. Highly Disruptive Mutation Operator

11. Computational Complexity • Restricted selection O (nN2) • Ranking procedure O (MN2) • Crowding procedure max{ O (MN log N), O (nN log N) } • Overall iteration-wise complexity max {O (nN2), O (MN2), O (nN log N)}

12. Implementation Details • Code written in simple C and strictly conforms to ANSI/ISO standard • Data structure used is a custom doubly linked list (gives O(1) insertion and deletion) • Randomized quick sort used for sorting • Almost all the functions perform in-place operation (addresses are passed, significantly decreases stack overheads)

13. Simulation Results • GA parameters for all problems chosen as follows • ηc = 20 • ηm = 20 • P (crossover) = 0.8 • P (mutation) = 1/n, where n = # of real variables • δ = 0.001 • Population size and number of generations different for different problems

14. Simulation Results contd… • 20 variable Rastrigin function • # of function evaluation • Least = 19260 • Median = 24660 • Worst = 29120 • 20 variable Schwefel function • # of function evaluation • Least = 54950 • Median = 69650 • Worst = 103350 • Other single objective problems can be found in the paper • In all cases, better results are found in comparison to previous reported studies

15. Single objective multi-modal function f(x) = sin2 (πx) x є [0,20]

16. Single objective multi-modal function Unconstrained Himmelblau’s function

17. Multi-objective Uni-Global Test Problems 30 variable ZDT2 (100×100)

18. Multi-objective Uni-Global Test Problems 10 variable ZDT4 (100×250)

19. Multi-objective Uni-Global Test Problems CTP4 (100×7000)

20. Multi-objective Uni-Global Test Problems CTP8 (100×100)

21. Multi-objective Uni-Global Test Problems DTLZ4 (300×100)

22. Multi-objective Multi-Global Test Problem F1 (x) = summation (sin (πxi) ) xiє [0,6] F2 (x) = summation (cos (πxi) ) xiє [0,6] Efficient points in phenotypic space

23. Multi-objective Multi-Global Test Problem Genotypic space plots

24. Few Sample Simulations • F(x) = sin2 (10,000*pi*x) • Himmelblau’s Functions • ZDT Test Problems • CTP Test Problems • Test Problem TNK • Multi-global Multi-objective Test Problem

25. Further Ideas and Future Work • Incorporating PCX instead of SBX for crossover • Automatically fine-tuning mutation index so as to achieve arbitrary precision • Self-adaptation of parameter δ • Segregating population into niches without the introduction of DM • Dynamic population sizing • Using hierarchical NDS for the crowding distance assignment