300 likes | 401 Views
Iterative Prototype Optimisation with Evolved Improvement Steps. Ji ří Kubalík and Jan Faigl Czech Technical University Department of Cybernetics Technicka 2, 166 27, Prague, Czech Republic email: {kubalik,xfaigl}@labe.felk.cvut.cz. Motivation. Typically ,
E N D
Iterative Prototype Optimisation with Evolved Improvement Steps Jiří Kubalík and Jan Faigl Czech Technical University Department of Cybernetics Technicka 2, 166 27, Prague, Czech Republic email: {kubalik,xfaigl}@labe.felk.cvut.cz
Motivation • Typically, • evolutionary optimisation framework considers an evolution of a population of candidate solutions to the problem at hand • candidate solution encodes a complete solution (a complete set of problem control parameters, a complete schedule in JSP, a complete tour for TSP, etc.) • the optimal or well-fit solution is hard to find for large problem instances • Here, • EA does not handle the solved problem as a whole • EA is employed within the iterative optimisation framework • its role is to evolve the best modification of the current solution prototype in each iteration
Outline of POEMS algorithm generate(Prototype); repeat BestSequence run_EA(Prototype); if(apply_to(BestSequence, Prototype) is_better_thanPrototype); thenPrototype apply_to(BestSequence, Prototype); until(POEMS termination condition); returnPrototype; • Prototype initialised randomly or using some problem-specific knowledge
Implementation Issues • Action Sequence Representation • Actions – set of primitive actions • determines the explorative power of the algorithm • linear chromosomes of maximal length MaxGenes << ProblemSize • gene = (action_type, parameters) • nop(no operation) • void action with no effect on the prototype, regardless of the values of its parameters • one or more nop actions allowed in a chromosome • variable effective length of chromosomes • At least one action must be active Ex.: MaxGenes =5, Actions ={action1, action2, action3} Ch = (action3, 0.8),(nop, 0.7),(action2, 0.2),(nop, 0.5),(action2, 0.2)
Implementation Issues • Genetic Operators • Tournament selection • Crossover – generalised uniform • Given two parental chromosomes, any combination of their actions forms a valid offspring Ex.: Par1 = (action3, 0.8),(nop, 0.7),(action1, 0.2),(nop, 0.5),(action2, 0.2) Par2 = (nop, 0.8),(action1, 0.8),(action3, 0.6),(nop, 0.5),(nop, 0.5) Off1 = (action1, 0.2),(action3, 0.6),(nop, 0.7),(action1, 0.2),(nop, 0.5) Off2 = (nop, 0.8),(action2, 0.2),(action3, 0.6),(nop, 0.5),(action_1, 0.8) • Mutation – action_type or parameters changed (1 gene per sequence) • Optionally, multiple duplicates of the same genes are eliminated Off1 = (action1, 0.2),(action3, 0.6),(nop, 0.7),(action1, 0.2),(nop, 0.5) Off1’ = (action1, 0.2),(action3, 0.6),(nop, 0.7),(nop, 0.2),(nop, 0.5)
Implementation Issues • Evaluation criterion • maximizes improvement of the prototype • penalizes sequences that do not change the prototype • Evolutionary Model • EA should converge fast, since it runs many times through the whole run • Generational / Steady-state
Standard Generational EA initialize(OldPop); BestSequence best_of(OldPop); repeat NewPop BestSequence; // elitism repeat Parents select(OldPop); Children cross_over(Parents); mutate(Children); evaluate(Children); NewPop Children; until(NewPop is completed); BestSequence= best_of(NewPop); switch(OldPop, NewPop); until(EA termination condition); return BestSequence;
Steady-state EA initialize(Population); repeat Parents select(Population); Children cross_over(Parents); mutate(Children); evaluate(Children); Replacement = find replacement(Population); Population[Replacement] Child1; Replacement = find replacement(Population); Population[Replacement] Child2; until(EA termination condition); BestSequence best of(Population); return BestSequence;
TSP: Experimental Setup • Primitive Actions • move(city1, city2) moves city1 right after city2 in the tour • invert(city1, city2) inverts a subtour between city1 and city2 • swap(city1, city2) swaps city1 and city2 • POEMS configuration: • chromosome length: 10 • population size: Cities ×2 • EA evaluations: 10.000 • POEMS evaluations: Cities ×1000 • Other algorithms • parameters set as recommended • use the same number of fitness evaluations, if possible
Heuristic used for tour initialisation • Only links between cities from specified neighborhood are considered when generating • the prototype in POEMS, • tours form the initial population (E-R), and • the initial tour for 2-opt. • No heuristic used for initialising the ring of neurons for SOM
100 cities 200 cities 1000 cities 2000 cities 500 cities TSP: Test Datasets • TSP Data – uniformly sampled points
POEMS: Random vs. Heuristic Init. • 100 cities 4738.5 831.9 1029.4 818.6
Random vs. Heuristic initialization • 200 cities 9574.2 1190.1 1357.4 1132.9
Random vs. Heuristic initialization • 500 cities 25520.1 6234.9 2134.1 1746.3
Random vs. Heuristic initialization • 1000 cities 51117.9 17997.0 2946.8 2523.0
Random vs. Heuristic initialization • 2000 cities 102331.6 34878.0 4344.5 3692.2
Example of POEMS execution: TSP Initial tour 965.134 after iteration 1 after iteration 2 after iteration 3 Final tour 824.8
Example of Evolved Solution Initial prototype of length 4403 Initial prototype of length 3655
Deceptive function– 100 bits • Hierarchical If and only If • Ch_length = 128 bits • Royal Road Problem • Ch_length = 96 bits Binary String Opt.: Test Problems • OneMax – counts 1s in a chromosome • Ch_length = 100 • Rosenbrock • Ch_length = 4 × 2 × 12 • F(x’1, x’2)< 0.001 is optimum • F103 • Ch_length = 5 × 2 × 10 • F(x’1, x’2)< 0.001 is optimum
Binary String Opt.: Experimental Setup • Primitive Actions • invert(gene) … inverts the specified gene of the prototype • Linkage • tight – groups of dependent genes form compact clusters within the chromosome • loose – dependent genes are loosely coupled within the chromosome • POEMS configuration: • chromosome length: 10 • population size: 200 • EA evaluations: 3.000 • POEMS evaluations: 106 • Evolutionary model: generational, steady-state • Other algorithms • parameters set as recommended • use the same number of fitness evaluations, if possible
Real-valued Opt.: Implementation Issues • Primitive Actions • add(value) … adds value(maxixi) to the respective prototype variable xi, where maxi is an upper bound of variable xi. • value is initialised from interval (0.0, 1.0). • sub(value) … subtracts value(ximini) from xi. • sample_right(value) … picks a new value of the respective prototype variable xi, from interval (xi , maxi). • samples the interval (xi , maxi) non-linearly preferring smaller changes to the larger ones. • sample_left(value) … picks a new value of the respective prototype variable xi, from interval (mini , xi). • Chromosome structure – an ordered list of D actions, each of them operating on the respective prototype variable, where D is a dimension of the problem.
Real-valued Opt.: Implementation Issues • Reuse of the evolved code • Initial population of EA in later iterations a portion of action from the previous EA • to keep the search direction that appeared beneficial in the past, • to reuse the unexploited yet possibly useful material evolved in the last iteration. • POEMS configuration: • chromosome length: D • population size: 200 • EA evaluations: 1500 (10D), 2000 (30D) • POEMS evaluations: D×104
Real-valued Opt.: Experiments • 10D and 30D functions collected for the Special Session on Real-parameter Optimization of the IEEE Congress on Evolutionary Computation 2005. • 25 functions in 4 groups • P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y.-P. Chen, A. Auger, and S. Tiwari,“Problem definitions and evaluation criteria for the CEC 2005 Special Session onReal-Parameter Optimization”, tech. rep., Nanyang Technological University, Singapore,May 2005. http://www.ntu.edu.sg/home/epnsugan • POEMS compared to the algorithms presented at the Special Sesion. • Observations: • POEMS exhibits about-average performance among the compared alg. • It is competitive to most of the algorithms on multimodal prob. • Improvement step size tends to decrease during the course of the run
13 Shifted Expanded Griewank’s plus Rosenbrock’s Function (F8F2)
Conclusions • Pros and Cons • General optimisation approach for combinatorial, discrete, and real-valued optimisation problems • Linkage independent • Inability to treat problems with building blocks of bigger size • Local (point-to-point) search approach – prone to get stuck in a local optimum • The latter stage of the iterational algorithm is the harder it is to find any improvement modification ( local search) • Scope of possible applications • Rescheduling – insert a new order into the current schedule • SAT problem – dependencies among variables are necessary for effective searching for the optimum solution (but not known a priori)