Introduction to Genetic Algorithms: Basic Structure and Implementation

1. COMP305. Part II. Genetic Algorithms. Genetic Algorithms

2. Topic 5. Similarity Templates (Schemata). Genetic Algorithms

3. GAs by John Holland. • Holland introduced • a “population” of binary strings which he called “chromosomes”. • The “population” evolves using kind of “natural selection” together with the genetics-inspired operators of crossover, mutation, and inversion. • Bits in a “chromosome” represent genes, and each “gene” is an instance of a particular “allele”, 0 or 1. • The selection operator chooses those chromosomes in the population that will be allowed to reproduce, and on average the fitter chromosomes produce more offspring than the less fit ones. • Crossover exchange subparts of two chromosomes • Mutation randomly changes the allele values of some locations in the chromosome. • Inversion reverses the order of a contiguous section of the chromosome rearranging the genes order. Genetic Algorithms

4. Basic Structure of a Genetic Algorithm. • Randomly generate initial population of n strings (“chromosomes”) • Evaluate the fitness of each string in the population • 3. Repeat the following steps until next generation of n individual strings produced • a.Select pair of parent chromosomes from current population • according to their fitness, i.e. chromosomes with higher fitness • are selected more often • b. Apply crossover (with probability) • c. Apply mutation (with probability of occurrence) • 4. Apply generational replacement • 5. Go to 2 or terminate if termination condition met Genetic Algorithms

5. Basic Structure of a Genetic Algorithm. • Randomly generate initial population of n strings (“chromosomes”) • Evaluate the fitness of each string in the population • 3. Repeat the following steps until next generation of n individual strings produced • a.Select pair of parent chromosomes from current population • according to their fitness, i.e. • chromosomes with higher fitness • are selected more often • b. Apply crossover (with probability) • c. Apply mutation (with probability of occurrence) • 4. Apply generational replacement • 5. Go to 2 or terminate if termination condition met • Each iteration in the cycle • produces a new “generation” of chromosomes. • The entire set of generations is called a run. • Typical GA run is from 50 to 500 or more generations. • At the end of a run often there is at least one highly fit chromosome in the population. Genetic Algorithms

6. Example Implementation of a GA. • Let the length of the string l = 8, and the number of chromosomes in the population (population size) n = 4. • Fitness functionf(x) is equal to the number of ones in the bit string x. • Selection operator. Fitness-proportionate selection, i.e. the number of times an individual is expected to reproduce is equal to its fitness fidivided by the average fitness of the population fNi= fi/ f • Crossover probabilitypc = 0.7 • Mutation probabilitypm = 0.001 • Make a run ofthree generations. Genetic Algorithms

7. Example of a Genetic Algorithm. 2.Evaluate the fitness of each string in the population Chromosome Chromosome Fitness = number of ones index string in the string 8 11100110 f8 = 5 12 00101110 f12 = 4 13 11011110 f13 = 6 14 01111110 f14 = 6 f Average fitness f = (5+4+6+6)/4 = 5.25 The average fitness in the population of possible solutions increases with every generation. 01 2 3 Generation № Generation 3 5.0 3.0 1.0 Genetic Algorithms

8. Why Genetic Algorithms work? • It might be seen that • similar “looking” chromosomes do have similar fitness values, and therefore • are taken for reproduction equally often. • Chromosome Chromosome Fitness Times to be selected • index string for reproduction • 1 00000110 f1 = 2 N1 = 1 • 2 11101110 f2 = 6 N2 = 2 • 3 00100000f3 =1N3 = 0 • n = 4 00110010 f4 = 3 N4 = 1 Generation 0 Genetic Algorithms

9. Why Genetic Algorithms work? • It might be seen that • similar “looking” chromosomes do have similar fitness values, and therefore • are taken for reproduction equally often. • Chromosome Chromosome Fitness Times to be selected • index string for reproduction • 8 11100110 f8 = 5 N8 = 1 • 911001110f9 = 5N9 = 1 • 1000100110f10 = 3N10 = 0 • 11 00111110 f11 = 5N11= 2 Generation 2 Genetic Algorithms

10. Similarities in GAs chromosomes. • Similarities among highly fit strings guide the search. • Chromosome Chromosome Fitness • index string • 8 11100110 f8 = 5 • 1211011110f12 = 6 • 13 00101110f13 = 4 • 1401111110f14 = 6 Generation 3 Genetic Algorithms

11. Similarity Template = Schema. • Holland introduced a • similarity template = schema = building block • to describe subset of strings with similarities at certain positions. • Chromosome Chromosome Fitness • index string • 8 11100110 f8 = 5 • 1211011110f12 = 6 • 13 00101110f13 = 4 • 1401111110f14 = 6 Generation 3 Genetic Algorithms

12. Similarity Template = Schema. Definition: A schema (plural,schemata) is a similarity template describing a subset of strings with similarities at certain string positions. Other name for a schema is abuilding block of a chromosome. Genetic Algorithms

13. Similarity Template = Schema. • To describe building blocks of binary chromosomes, the following three letter alphabet is used: • {0, 1, *} • Here, * is a do not caresymbol. Genetic Algorithms

14. Similarity Template = Schema. • To describe building blocks of binary chromosomes, the following three letter alphabet is used: • {0, 1, *} • Here, * is a do not caresymbol. • Example: 111**0is a schema of the chromosome • 111100 Genetic Algorithms

15. Similarity Template = Schema. • Example: 111**0is a schema of the chromosome • 111100 • The meaning of the schema is that it works a pattern matching device: a schema matches a particular string iff at every location in the schema a 1 matches 1 in the string, a 0 matches a 0, or a * matches either. Genetic Algorithms

16. Similarity Template = Schema. As an example consider strings and schemata of length 5. Example 1: The schema *0000 matches two chromosomes: 1) 00000 and 2) 10000 Example 2: The schema *000* matches four chromosomes: 1) 00000 , 2) 10000 , 3) 00001 , and 4) 10001 Genetic Algorithms

17. Similarity Template = Schema. • A question opposite to the previous examples is • How many building blocks there are in a particular chromosome of a fixed length? • Answer: To match a particular chromosome of a fixed length the schema must • a) be of the same length, • b) have the same symbol as the chromosome does or a “*“ - do not care symbol at any particular locus Genetic Algorithms

18. Similarity Template = Schema. • A question opposite to the previous examples is • How many building blocks there are in a particular chromosome of a fixed length? • Answer: To match a particular chromosome of a fixed length the schema must • a) be of the same length, • b) have the same symbol as the chromosome does or a “*” - do not care symbol at any particular locus • Thus, in a binary chromosome of a length lthere are • 2l building blocks, • as a symbol at a particular locus in the chromosome must correspond to the same or the do not care symbol in the corresponding locus in the schema. Genetic Algorithms

19. Similarity Template = Schema. • How many building blocks there are in a particular chromosome of a fixed length? • Answer: in a binary chromosome of a length l there are • 2l building blocks, • as a symbol at a particular locus in the chromosome must correspond to the same or the do not care symbol in the corresponding locus in the schema. • Example 3: • The 2bit (l=2) chromosome 11 has 22=4 building blocks • 1) *1 , • 2) 1* , • 3) **, and • 4) 11 Genetic Algorithms

20. Similarity Template = Schema. • How many building blocks there are in a particular chromosome of a fixed length? • Answer: in a binary chromosome of a length l there are • 2l building blocks. • Example 4: • The 3bit (l=3) chromosome 101 has 23=8 building blocks • 1) 10*, • 2) 1** , • 3) ***, • 4) 1*1 , • 5) *01 , • 6) **1 , • 7) *0* , and • 8) 101 Genetic Algorithms

21. Similarity Template = Schema. • Definition: • The ORDER of a schema is the number of non-* symbols it contains. • Example 5: • Order of the schema 10*00*** is4. • Example 6: • Order of the schema *** is0. • One might say that the order of the schema *** is lower than the one of the schema 10*00***. Genetic Algorithms

22. Similarity Template = Schema. Definition: The DEFINING LENGTH of a schema is the distance, i.e. the number of positions, between the outermost non-* symbols. Example 7: The defining lengthof the schema 10*00* is3. 10*00* outermost non-* symbols 3 positions between the outermost non-* symbols Genetic Algorithms

23. Similarity Template = Schema. • Definition: • The DEFINING LENGTH of a schema is the distance, i.e. the number of positions, between the outermost non-* symbols. • Example 7: • The defining lengthof the schema 10*00* is3. • Example 8: • The defining length of the schema *11*** is0. • One might say that the defining length of the schema *11*** is shorter than the one of the schema 10*00* Genetic Algorithms

24. Schema Theorem. • Highly fit, short defining length, low order schemas propagate from generation to generation and give exponential increase of samples to the observed best. Genetic Algorithms