1 / 12

CAP6938 Neuroevolution and Artificial Embryogeny Intro to Neuroevolution

CAP6938 Neuroevolution and Artificial Embryogeny Intro to Neuroevolution. Dr. Kenneth Stanley January 30, 2006. Main Idea: Combine EC and Neural Networks. “Evolving brains”: Neural networks compete and evolve Idea dates back to the late 80’s

gwyn
Download Presentation

CAP6938 Neuroevolution and Artificial Embryogeny Intro to Neuroevolution

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CAP6938Neuroevolution and Artificial EmbryogenyIntro to Neuroevolution Dr. Kenneth Stanley January 30, 2006

  2. Main Idea:Combine EC and Neural Networks • “Evolving brains”: Neural networks compete and evolve • Idea dates back to the late 80’s • Natural: Only way that intelligence ever really was created • Leads to many research challenges

  3. Advantage: Applies to Both Supervised and RL Problems • If targets are provided, they can be used to calculate fitness • Else, sparse reinforcement can also be used to calculate fitness • RL is harder and frequently more interesting Forward Left Right Front Left Right Back

  4. What’s It Used For? • Supervised classification • Autonomous control • Robots • Vehicles • Video game characters • Factory optimization • Game playing: Go, Tic-tac-toe, Othello • Warning systems • Visual recognition, roving eyes

  5. Earliest NE Methods Only evolved Weights • Genome is a direct encoding • Genes represent a vector of weights • Could be a bit string or real valued • NE optimizes the weights for the task • Maybe a replacement for backprop ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

  6. The Competing Conventions Problem (Whitley, also Radcliffe) • Also called permutation problem (Radcliffe) • Many permutations of same vector represent exactly the same functionality • Then how can crossover work? A C A B C A B C A B C B A B C B A C 3!=6 permutations of the same network!

  7. Competing Conventions Destroys Crossover • n! permutations of an n-hidden-node 1-layer net • [A,B,C] X [C,B,A] can be [C,B,C] • 144 total possible crossovers of size 3 • 72 are trivial (offspring is a duplicate) • 48 of the remaining 72 are defective • 66.6% of nontrivial mating is defective! • Consider also differing conventions: • [A,B,C]X[D,B,E] • Loss of coherence in GA is severe

  8. TWEANNS • “Topology and Weight Evolving Artificial Neural Networks” • Population contains diverse topologies • Why leave anything to humans? • Topology can be represented many ways • Topology evolution can combine w/ backprop • Remember: Topology defines the search space • The more connections, the more dimensions

  9. “Competing Conventions” with Arbitrary Topologies • Topology matching problem • Life is even worse with mating arbitrary topologies • How do they match up? • Radcliffe (1993) : “Holy Grail in this area.”

  10. More TWEANN Problems • Diverse topologies present many problems • How should evolution begin? Randomly? • Defects in the initial population • Searching in unnecessarily large space

  11. More TWEANN Problems 2 • Innovative structures have more connections • Innovative structure cannot compete with simpler ones • Yet the money is on innovation in the long run • Need some kind of protection for innovation

  12. Next Class: Sample Neuroevolution Methods • Past approaches to the problems • CE: Topology evolution gains prominence • ESP: Fixed-topologies strikes back Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor by Byoung-Tak Zhang and Heinz Muhlenbein(1993)A Comparison between Cellular Encoding and Direct Encoding for Genetic Neural Networks by Frederic Gruau, Darrell Whitley, Larry Pyeatt (1996)Solving Non-Markovian Control Tasks with Neuroevolution by Faustino J. Gomez and Risto Miikkulainen (1999) Homework due 2/6/05: 1 page project proposal including project description and goals, a falsifiable hypothesis on what you expect to happen, why it involves structure, and what platform you will use (language and OS). If partners, describe briefly division of labor.

More Related