1 / 26

The Genetic Algorithm

The Genetic Algorithm. The Research of Robert Axelrod The Algorithm of John Holland Reviewed by Eyal Allweil and Ami Blonder. The Prisoner’s Dilemma. The Iterated Prisoner’s Dilemma (IPD). Two tournaments held by Robert Axelrod

lacota-bailey
Download Presentation

The Genetic Algorithm

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Genetic Algorithm The Research of Robert Axelrod The Algorithm of John Holland Reviewed by Eyal Allweil and Ami Blonder Genetic Algorithm

  2. The Prisoner’s Dilemma Genetic Algorithm

  3. The Iterated Prisoner’s Dilemma (IPD) • Two tournaments held by Robert Axelrod • The highest average score was Anatol Rapoport’s Tit-For-Tat (TFT) • Eight representatives explain 98% of the variance in the tournament results. • We will also hear criticism of these claims. ( Ken Binmore) Genetic Algorithm

  4. Introduction: Axelrod’s Motivation • Axelrod wanted to prove a point: The success of Tit-For-Tat in his Iterated Prisoner’s Dilemma was not based on the preconceptions of those who submitted entries. • The hammer and the nail. Genetic Algorithm

  5. More practical motivations • We have a group aim, and we want agents that can fulfill it. More accurately, we want our agents to be as efficient as possible in achieving this goal. • Example: The Mars element-gathering, subsumption architecture experiment Genetic Algorithm

  6. The Genetic Algorithm • Construct an initial population • Test and score population • Calculate number of offspring • Reproduction: Choose parental pairs • Reproduction: Crossover / Mutation Genetic Algorithm

  7. Encoding a Strategy (1) Axelrod encoded strategies as sequences of 70 bits. How does this work? Note: 0 = C, 1 = D Genetic Algorithm

  8. Encoding a Strategy (2) Genetic Algorithm

  9. 1. Constructing an Initial Population • Two possibilities for creating an initial population: • Initial strategies can be assembled (as in the first two tournaments) or - • They can be randomly created Genetic Algorithm

  10. Constructing an Initial Population (cont’d) • Axelrod used normalized population of 20 • A rule of thumb is that the product of the number in the population and the number of generations should exceed 100,000 • In addition, the number of individuals in the population must considerably exceed the number of genes in each individual's chromosome. Genetic Algorithm

  11. 2. Testing the Population • In the Iterated Prisoner’s Dilemma, the outcome of each interaction is added to produce a player’s score for that generation. • In general, an ordering function is needed. • This function must be efficient! Genetic Algorithm

  12. 3. The Number of Offspring(in Axelrod’s simulation) • Strategies that were one standard deviation above the average score produced two matings. • Strategies that were one standard deviation below the average were barren- no offspring. • Other strategies produced a single mating • Each generation is disjoint from its predecessor Genetic Algorithm

  13. The Number of Offspring (cont’d) • There are other possibilities available when calculating offspring: • Normalization / growth of population size • Preservation of arbitrary amount of previous generation’s strategies (not done in Axelrod model) Genetic Algorithm

  14. 4. Choosing Parents • In Axelrod’s simulations, pairs were chosen randomly to mate and produce, each, two offspring. • Other possibilities exist: • Mating by excellence (short term exploitation) • Mating by geographic proximity Genetic Algorithm

  15. 5. Reproduction: Crossover • “Crossover selects one or more places to break the parents’ chromosomes in order to construct two offspring each of whom has some genetic material from both parents.” • Other forms of crossover are possible. • Syntactic integrity must be preserved! • What are the advantages of high/low crossover? (more on this later) Genetic Algorithm

  16. 5. Reproduction: Mutations • In every offspring born, there is a small chance of bit reversal- a change in strategy. • Don’t forget syntactic integrity! • What are the advantages of high/low mutation rates? Genetic Algorithm

  17. Technical Details • Population size: 20 • Round length: 151 meetings • Each population member met one of 8 “representatives” (not each other) • Therefore 24000 meetings per generation • Number of generations: 50 • 40 2-parent (sexual) experiments, 40 asexual ones Genetic Algorithm

  18. Conclusions (1) : Effectiveness • “the problem for evolution can be conceptualized as a search for relatively high points in a multidimensional field of gene combinations, where height corresponds to fitness.” • Axelrod: TFT-like strategies are the big winner! • The genetic algorithm produces results better than the second Axelrod tournament: 450 weighted score vs. 428 for TFT • But these results were in 11 (out of 40) experiments which resulted in a strategy which tried to exploit “sucker” strategies! (more later) Genetic Algorithm

  19. Successful Alleles: TFT-like behavior • Don’t rock the boat ( C after RRR ) • Be provocable ( D after RRS ) • Accept apologies: ( C after TSR ) • Forget: ( C after SRR ) • Accept a rut: ( D after PPP ) Genetic Algorithm

  20. Conclusions (2) : Sexual Reproduction • In biology, sexual reproduction carries a stiff price – useless males • Computationally, sexual reproduction is cheap. • Asexual runs of IPD resulted in lower average scores (5 out of 11 had higher median scores than TFT) • Parasite theory. (only if we have time) Genetic Algorithm

  21. Conclusions (3) : Arbitrariness • Hitch-hiking genes- typically present, not typically employed. (TFT example – PPR) • Premises and their results ( TFT example – the original six bit premise ) • Two basin theory ( Ken Binmore ) : By choosing the correct premises, we can produce different (stable) evolutionary end-products Genetic Algorithm

  22. Conclusions (4) : Tradeoffs • The trade off that exists is between flexibility and specialization. • This can be translated into short and long term gains (exploitation vs. exploration) • Varying mutation based on how dynamic the environment is. • Invasion Genetic Algorithm

  23. Conclusions (5) : Irreversibility • The possibility exists of getting stuck in a local maximum. • This results from adaptation to a set of premises, which strengthens them in the future. • There are those who claim that TFT is such a local maximum. Genetic Algorithm

  24. Conclusions and Criticism • Kristain Lindgren (1991) – found cyclical history of stability / instability • Lombard (1996) : more extensive simulations, copy-and-innovate instead of standard genetic algorithm • Lombard: Noise is a crucial factor Genetic Algorithm

  25. More Criticism and Conclusions • Binmore -TAT-FOR-TIT is a better strategy? • Were more experiments called for? Probst (1996) found exploitive machines thriving after running longer experiments! • His criticism is against the simulation results, not against the use of the algorithm in general. • But this is a lesson that applies only in pairwise interactions. In multi-person interactions, it need not be the injured party who punishes a cheater. Genetic Algorithm

  26. References • Axelrod, Robert. The Complexity of Cooperation • Online: Review of The Complexity of Cooperation (Ken Binmore) • Online: Simulation for the Social Scientist (Nigel Gilbert and Klaus G. Troitzsch) Genetic Algorithm

More Related