Co-Evolution and Speciation

1. Co-Evolution and Speciation 학습목표 공진화와 종분화의 개념을 이해하고, 몇가지 문제에서의 응용가능성 점검

2. Outline • What is co-evolution • Co-evolving sorting networks (Hillis’s seminal work) • Simulated evolution on a Connection Machine • Co-evolving parasites improves simulated evolution • Different types of co-evolution • Inter-population co-evolution: An example in design, Knowledge discovery (data mining), Interactive evolution • Intra-population co-evolution: Co-evolving a backgammon player, Taking two interwound spirals apart by co-evolution, Iterated prisoner’s dilemma • Niching, speciation and crowding

3. What is Co-Evolution • Without co-evolution • The fitness landscape is fixed • The same individual always gives the same fitness • With co-evolution • The fitness landscape is not fixed • The fitness of an individual depends on other individuals • The same individual may not have the same fitness in different populations • In other words, co-evolution can be regarded as a kind of landscape coupling where adaptive moves by one individual will deform landscapes of others • Basic concept: individuals interact with each other through changing each other’s fitness landscapes

4. Sorting Networks • A sorting algorithm in essence, but can be represented graphically for the ease of understanding • Used widely in switching circuits, routing algorithms, and other areas in interconnection networks • Two issues • Number of comparators • Number of layers • Best known networks with 16 inputs still the best known today

5. Sorting Networks • Comparators • Graphical representation of a sorting network small sorted output unsorted input large input element unsorted input sorted output a layer

6. Problem Formulation • Search the space of sorting networks for the optimal one with the minimal number of comparators • Search the space of small networks for the optimal one that sorts all inputs correctly

7. Evolution without Co-Evolution • Phenotypic representation • An ordered sequence of ordered integer pairs • {(0, 1), (2, 3), ……} • Experiments • 64536 individuals for up to 5000 generations • The best found: 65 comparators • Problems • Local optima, i.e., the population stagnates at a sub-optimal solution and is unable to make further progress • Inefficiency in fitness evaluation. Often spent much time on some “easy” cases

8. Co-Evolution  Best network evolved: 61 comparators • Two separate populations • Host population: sorting networks • Parasite populations: test cases (10 to 20) • They co-locate at each node of the grid • They interact/co-evolve through fitness evaluation • For the host population, an individual is evaluated by the test cases at its site • For the parasite populations, an individual is evaluated by the number of cases that the network fails to sort • Co-evolution brings two benefits • It creates competition (“arms race”) between two populations and avoids early stagnation of a population • Saves time in fitness evaluation

9. Beyond Hillis’s Work • Basic concept and ideas about co-evolution: individuals interact There are a number of different ways of interacting fitness evaluations • Select a random individual in another population in fitness evaluation • Select the best • Select a group (even the whole population) • Inter-population versus intra-population • Competitive co-evolution versus cooperative co-evolution Population 2 Population 1

10. Different Types of Co-Evolution • Based on the number of population involved: • Inter-population co-evolution • Two or more populations • Intra-population co-evolution • Within a single population • Based on the relationship among individuals • Competitive co-evolution • Individuals compete for higher fitness to solve a problem (often a dynamic problem) • Cooperative co-evolution • Individuals cooperate with other in order to solve a problem

11. Co-Evolution in Design • Many design problems do not have a fixed goal or a fixed set of specifications • People do change minds Current Problem k Current Problem k+1 evolve … … provide fitness provide fitness provide fitness Current Solution k+1 Current Solution k evolve … …

12. Knowledge Discovery (Data Mining) • One application: fraud detection • We want to discover all kinds of frauds, but we do not really know what they are (what kind of patterns they have) • In general, we want to find something interesting without knowing what “interesting” actually means corporate database top performing rules rules evolving new interestingness analysis human ranking Interestingness is co-evolving with interesting rules

13. Interactive Evolution • Evolution with a human being in the loop • Often used in creative design or creative problem solving • May be time-consuming • Case1: no co-evolution • Case 2: co-evolution population fitness evaluation by human replacement genetic operation computer programs fitness evaluation human beings

14. Intra-Population Co-Evolution • There is only one population • However, fitness of one individual depends on other individuals in the population • Example 1: Playing backgammon • TD-Gammon • a grand master level computer program based on NN • It learned by self-playing • Self-playing = Co-evolution • Is it because machine learning algorithms or co-evolution?

15. Separating Interwound Two Spirals • Task: Given 194 training points, learn to separate two spirals • Very tough problem for machine learning algorithms, e.g., decision trees, neural networks, … • There was an attempt to evolve a solution to it. But the solution generalized poorly • Competitive co-evolution based on covering really helps • Fitness evaluation without co-evolution: number of test cases correctly classified • Fitness evaluation with co-evolution: based on pair-wise competition, it depends only on the number of test cases correctly classified but NOT covered by its opponent

16. Iterated Prisoner’s Dilemma • Non-zero sum, non-cooperative games • The 2 player version • The purpose here is not to find the optimal solution for some simplified conditions, but to study how to find it • Fitness evaluation • Entirely determined by the total payoff obtained through playing against each other • The initial population was generated at random Player A C D 3 5 C 3 0 Player B 0 1 D 1 5

17. Why Co-Evolution • We do not know the fitness function • There are too many cases to test in order to obtain a fitness value. Co-evolution can be used to FOCUS search in the most important area • The problem is inherently changing in time • Increase and maintain diversity • Self-learning

18. Why Niching • In evolutionary computation, niching refers to the formation of groups of individuals in a population • Individuals within a group are similar to each other • Individuals from different groups are dissimilar to each other • Niching helps to explore and maintain diversity • Niching helps in machine learning, e.g., classification • Niching helps multi-objective function optimization • Niching helps simulation of complex and adaptive systems • Niching helps cooperative co-evolution • Niching is fun!

19. Different Niching Techniques • Can be divided roughly into two major categories • Sharing, also known as fitness sharing • Crowding • Other niching methods include sequential niching and parallel hillclimbing

20. sshare an individual a scaling constant 1 – (d / sshare) if d < sshare a distance fshare(i) = fraw(i) / Sj=1sh(dij) m Fitness Sharing: Implementation • Define a sharing radius sshare: Anything within this radius will be regarded to be similar to the individual and thus needs to share fitness • Define a similarity measure, i.e., distance: The shorter the distance between two individuals, the more similar they are • Define a sharing function • Define shared fitness The individual in the center needs to share fitness with all other in the circle sh(d) = 0 otherwise where m is the population size

21. scaling factor fshare(i) = fraw(i) / Sj=1sh(dij) m Fitness Sharing: Extensions • Sharing can be done at genotypic or phenotypic level • Genotypic: Hamming distance • Phenotypic: Euclidean distance (Overlap in test case covering in classification)  The key issue is how to define the “distance” measure • Sharing radius sshare can be difficult to set, the same, fixed: It should be sufficiently small in order to discriminate between two neighboring peaks • Population size should be sufficiently large to locate all peaks • Population may not be able to converge to exact optima • Population may not be stable, i.e., may lose peaks located • Calculate shared fitness needs time • Fitness sharing often needs raw fitness scaling. b Why?

22. Implicit Fitness Sharing (1) • The idea comes from an immune system: antibodies which best match an invading antigen receive the payoff for that antigen • Similar situation occurs in games: a strategy receives payoff when it achieves the best score against a test case • Implicit fitness sharing is most often used in learning. While (explicit) fitness sharing is done through individuals, implicit fitness sharing is test data based! • The algorithm for calculating fitness: For each data point i to be matched, do the following C times 1. Select a sample of s individuals from the population 2. Find the individual in the sample that achieves the highest score against the data point i 3. This best individual receives the payoff. In the case of a tie, payoff is shared equally

23. Implicit Fitness Sharing (2) • It has been shown that implicit and explicit fitness sharing have the same theoretical basis. s here plays the role of sshare in (explicit) fitness sharing • Larger C  better result but more time-consuming • Comparison between implicit and explicit sharing: they are better under different circumstances • Implicit fitness sharing covers optima more comprehensively, even when those optima have small basin of attraction, when the population is large enough for a species to form at each optimum • (Explicit) fitness sharing can find the optima with larger basins of attraction and ignore the peaks with narrow bases, when the population is not large enough to cover all optima

24. Niching vs. Speciation • Although some people distinguish between the two, we will treat them as the same thing • If there is any difference: • niching is concerned more with locating peaks (basins of attraction), while speciation is more focused on actually converging to optima

25. What Is Crowding • Crowding techniques insert new elements into the population by replacing similar elements. It’s about replacement strategies • Crowding techniques strive to maintain the preexisting diversity of a population • Crowding methods do not modify fitness

26. Crowding: Early Days • Cavicchio’s preselection scheme: • Purpose: prevent premature convergence, preserve diversity • What it is: an offspring (usually the fitter one) replaces the worse parent • How well it works for niching: poorly, unable to form niches • De Jong’s Crowding (crowding factor model): • Purpose: preserve population diversity • What it is: restricted replacement, offspring replaces the most similar parent in a same • How well it works for niching: poorly, only finding two peaks • Common feature: both involve modification of replacement strategies. A restriction is often imposed

27. Deterministic Crowding: Algorithm(1) • Problems with De Jong’s crowding: high replacement error (i.e., replacing an individual of one class by another one from a different class) • Mahfoud’s deterministic crowding algorithm: Input: g – number of generations to run the algorithm m – population size Output: P(g) – final population Algorithm: • P(c)  initialize () • FOR t  1 to g DO • P(t)  shuffle (P(t-1)) • FOR i  0 TO m/2 – 1 DO • DCA () • OD • OD

28. Deterministic Crowding: Algorithm(2) DCA (): P1  a2i+1(t) P2  a2i+2(t) {C1, C2}  recombine (P1, P2) C1’  mutate (C1) C2’  mutate (C2) IF [d(P1, C1’)+d(P2, C2’)] <= [d(P1, C2’)+d(P2, C1’)] THEN IF f(C1’) > f(P1) THEN a2i+1(t)  C1’ FI IF f(C2’) > f(P2) THEN a2i+2(t)  C2’ FI ELSE IF f(C2’) > f(P1) THEN a2i+1(t)  C2’ FI IF f(C1’) > f(P2) THEN a2i+2(t)  C1’ FI FI

29. Deterministic Crowding: Discussion • Capable of niching, i.e., locating and maintaining multiple peaks • Minimal replacement error • Few parameters to tune • Fast, no distance calculations needed • Population size is important. Should be large if unsure • Should use full crossover, i.e. crossover rate = 1.0 • The following appears to all crowding methods: • Unlike sharing methods, crowding methods do not allocate individuals proportional to peak fitness. Instead, the number of individuals congregating about a peak is largely determined by the size of that peak’s basin of attraction under crossover • Similarity can be measured at either genotypic or phenotypic level

30. Speciation (in a narrow sense) • In a broad sense, niching and speciation can be regarded as the same thing • In a narrow sense, the main emphasis of niching is on distributing individuals among different peaks, not finding exact optima efficiently. Speciation focuses on search within each peak • A speciation method restricts mating to similar individuals and discourages mating of individuals from different peaks (species) • In order to apply speciation techniques properly, individuals representing each peak must be found first. Speciation techniques cannot be used independently • Niching and speciation are complementary • Similarity can be measured at either genotypic or phenotypic level

31. Mating Restriction • Use tags: each individual consists of a tag and a functional string • Use distance: either in the genotypic or phenotypic space template tag functional string #1#0 1010 1010 …… 101 • Tag participate in (crossover and) mutation, but not fitness evaluation • Templates can also be used • Has been shown to be effective for multimodal function optimization • Only individuals with the same tag are allowed to mate Define a threshold parameter smate -Two individuals are allowed to mate only when their distance is smaller than smate -GAs with niching and mating restriction were found to better distribute the population across the peaks than GAs with sharing alone Mating restriction always applies during recombination

32. Summary • Co-evolution is closely related to fitness sharing although fitness sharing was first motivated by multimodal function optimization • Niching and speciation are useful • There are different niching methods • All niching methods involve fitness evaluation. However, they do interact with selection and crossover • References • T. Back, O.B. Fogel and Z. Michalewicz, Handbook of Evolutionary Computation, IOP Pub. Ltd & Oxford Univ. Press, 1997. Section C6.1 and C6.2 (only 9 pages) • P. Darwen and X. Yao, “A dilemma for fitness sharing with a scaling function,” Proc. Of IEEE ICEC, 1995, pp. 166~171 • P. Darwen and X. Yao, “Every niching method has its niche: fitness sharing and implicit sharing compared,” LNCS, vol. 1141, pp. 398~407, 1996.