Optimization in Dynamic Environments

1. Optimization in Dynamic Environments Ernesto Costa DEI/CISUC ernesto@dei.uc.pt http://www.dei.uc.pt/~ernesto/

2. Summary • Agents, Problems and Environments • Agents: Natural Selection and Genetics • Problems:Optimization • Environments: Dynamic • Optimization and Dynamic Environments • State of the Art • The Challenge / Problem

3. Agents, Problems and Environments Agent Problem Behavior Performance Environment

4. Agents and Evolutionary Computation • Darwin • Evolution by Natural selection • Mendel • Genetics and Inheritance procedure EC t = 0; inicialization P(t); evaluation P(t); while not stop_condition do t = t+1; P1(t) = selection (P(t-1)); P2(t) = op_modification (P1(t)); evaluation (P2(t)); P(t) = combine (P2(t) ,P(t-1)); End_do; return_best (P(t)); end_proc.

5. Problems: Combinatorial Optimization Problems that: have a finite number, F, of feasible solutions each solution has na associate cost, c. goal: a solution f in F that minimizesc Examples: knapsack minimum spanning tree bin packing set covering vehicle routing ...

6. Choose the items that maximize your profit ans such that the total weight is less that some given limit (knapsack capacity)! Combinatorial Optimization 0/1 Knapsack Items W[i], P[i] w1, p1 w2, p2 w3, p3 . . . wn, pn Binary representation: vector x=(x1,...,xn)

7. 4 3 5 3 2 4 3 3 5 4 5 6 Combinatorial Optimization (2a) Minimum Spanning Tree Given G=(V,E): a connected weighted undirected graph V={v1, ...,vn} E={e1,...,em} W={w1,...,wm}: weight or cost of each edge Find a subgraph S of G : S contains all the vertices of G S is connected and contains no cycles S has minimum cost A minimum spaning tree (MST)

8. Problems: Function Optimization Rastringin Function n=2, A=10

9. Environment: Dynamic Changes in the environment: Restrictions:Knapsack capacity C Problem Instance:MST #V, #E, W Goal: Rastringin Parameter A A different, time dependent, fitness landscape!

10. Environment: Dynamic Types of dynamics • Discrete vs Continuous • Periodic vs Non-Periodic • cycle length • Dimension of change • small vs big • Predictability of change

11. Environment: Dynamic Further Aspects • Change detection • Explicitly known • Average or best fitness drop • Reevaluating a set of individuals every generation • Keep a model of the environment (model and real ≠) • Does the EA change (e.g. representation)?

12. State of the Art • The problem • Standard EA • loose diversity (converge to an optimum) • No memory of the past • Solution • Start from scratch??? • New optimization algorithm (new Agent) • Kind of open-ended evolution • Using past information • Diversity • Memory The challenge!!!

13. State of the Art • Promoting Diversity • Hypermutation • Maintaining Diversity • Avoid convergence • Random immigrants • Use of Memory • Redundant Representations • Multiploidy • Explicit Memory • Interplay between memory and the evolving population • Multiple Populations • Self-Adaptive Memory

14. The Challenge • Choose a problem • Modify the standard Genetic Algorithm • Diversity mechanisms • Memory mechanism • Make Experiments with (some) previous approaches • Analyse Results • Propose New Solutions • New Results?!

15. The problem: Benchmarks Moving Parabola

16. The problem: Benchmarks Moving Peaks • Having several peaks • Position • Height • Width • Changes • One or several parameters • Possible to test different dynamics • A C-version available (Jurgen Branke) • A Matlab version (R. Morrison)

17. The problem: Measuring Performance • On-line performance • The average of the averages so far • Off-line performance • The average of the best so far • Best-of-generation averages for many runs on the same problem • Question:we want to measure the performance of the EA across the entire range of the fitness landscape dynamics

18. The IS is a complex system that includes cells, molecules and organs that constitutes an identification mechanism capable of perceiving and combating: dysfunction of our own cells (infectious self) action of exogenous infectious microorganisms (infectious non-self) The IS insures the integrity of the self! Other Ideas Immune System

19. Memorizing Invasion Detection Maturation Reaction Other Ideas Immune System How it works? Challenge: can we use it for dynamic environments???

20. Let’s Work! • References • Evolutionary Optimization in Dynamic Environments, Jürgen Branke, Kluwer Academic Publishers,2002. • Evolutionary Algorithms for Dynamic optimization Problems (EvoDOP 2003) in GECCO 2003, Jürgen Branke (Organizer) • http://www.aifb.uni-karlsruhe.de/~jbr/MovPeaks/

21. Environment: Dynamic 0/1 Knapsack Restrictions: changing the knapsack capacity, C

22. 4 3 5 3 2 4 8 9 4 7 3 3 11 14 7 2 5 4 5 6 10 6 8 4 1 2 Environment: Dynamic Minimum Spanning Tree Problem Instance: different vertices, edges and weights

23. Environment: Dynamic Rastringin Function Goal: different Max A=1..9

24. The problem: Measuring Performance Dynamic Environments • Question:we want to measure the performance of the EA across the entire range of the fitness landscape dynamics • Adapting the offline performance • Moment of changes are known • Using the error • Optimum is known • Current error • Offline error

25. The problem: Measuring Performance Dynamic Environments • Accuracy: recovery capacity • Adaptability: speed of recovery • K= # changes during the run • r= # generations between two consecutive changes • Erri,j= difference between current best at generation j after change #i and the optimum

26. Combinatorial Optimization (2b) Minimum Spanning Tree Formally Any subgraph S can be represented by a binary vector x={x1,...xm}, with xi= 1 if ei is in S If T is the set of all spanning trees in G then the MST is defined by: