1 / 25

CAP6938 Neuroevolution and Developmental Encoding Real-time NEAT

This talk by Dr. Kenneth Stanley explores the challenges and solutions for implementing real-time NeuroEvolution of Augmenting Topologies (NEAT), focusing on steady state evolution and simultaneous evaluation. The talk covers topics such as replacing individuals one at a time, dynamic compatibility thresholding, and the implementation of NEAT in the NERO game. The talk concludes with future applications and improving the neural model.

russelv
Download Presentation

CAP6938 Neuroevolution and Developmental Encoding Real-time NEAT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CAP6938Neuroevolution and Developmental EncodingReal-time NEAT Dr. Kenneth Stanley October 18, 2006

  2. Generations May Not Always Be Appropriate • When a population is evaluated simultaneously • Many are observable at the same time • Therefore, entire population would change at once • A sudden change is incongruous, highly noticeable • When a human interacts with one individual at a time • Want things to improve constantly

  3. Steady State GA: One Individual Is Replaced at a Time • Start by evaluating entire first generation • Then continually pick one to remove, replace it with child of the best Start: Evaluate All 2) Create offpsring from good parents Repeat… 3) Replace removed individual 1) Remove poor individual

  4. Steady State During Simultaneous Evaluation: Similar but not Identical • Several new issues when evolution is real-time • Evaluation is asynchronous • When to replace? • How to assign fitness? • How to display changes

  5. Regular NEAT Introduces Additional Challenges for Real Time • Speciation equations based on generations • No “remove worst” operation defined in algorithm • Dynamic compatibility thresholding assumes generations

  6. Speciation Equations Based on Generations

  7. How to Remove the Worst? • No such operation in generational NEAT • Worst often may often be a new species • Removing it would destroy protection of innovation • Loss of regular NEAT dynamics

  8. Dynamic Compatibility Thresholding Assumes A Next Generation

  9. Real-time NEAT Addresses Both the Steady State and Simultaneity Issues • Real-time speciation • Simultaneous and asynchronous evaluation • Steady state replacement • Fast enough to change while a game is played • Equivalent dynamics to regular NEAT

  10. Main Loop (Non-Generational)

  11. Choosing the Parent Species

  12. Finally: How Many Ticks Between Replacements? • Intuitions: • The more often replacement occurs, the fewer are eligible • The larger the population, the more are eligible • The high the age of maturity, the fewer are eligible

  13. rtNEAT Is Implemented In NERO • Download at http://nerogame.org • rtNEAT source available • Simulated demos have public appeal • Over 70,000 downloads • Appeared on Slashdot • Best Paper Award in Computational Intelligence and Games • Independent Games Festival Best Student Game Award • rtNEAT licensed • Worldwide media coverage

  14. Media Coverage

  15. Media Coverage

  16. NERO: NeuroEvolving Robotic Operatives • NPCs improve in real time as game is played • Player can train AI for goal and style of play • Each AI Unit Has Unique NN

  17. NERO Battle Mode • After training, evolved behaviors are saved • Player assembles team of trained agents • Team is tested in battle against opponent’s team

  18. NERO Training: The Factory • Reduces noise during evaluation • All evaluations start out similarly • Robot bodies produced by “factory” • Each body sent back to factory to respawn • Bodies retain their NN unless chosen for replacement • NN’s have different ages • Fitness is diminishing average of spawn trials:

  19. NERO Inputs and Outputs

  20. Enemy/Friend Radars

  21. Enemy On-Target Sensor

  22. Object Rangefinder Sensors

  23. Enemy Line-of-Fire Sensors

  24. Further Applications? • New kinds of games • New kinds of AI in games • New kinds of real-time simulations • Training applications • Interactive steady-state evolution

  25. Next Topic: Improving the neural model • Adaptive neural networks • Change over a lifetime • Leaky integrator neurons and CTRNN Evolutionary Robots with On-line Self-Organization and Behavioral Fitness by Dario Floreano and Joseba Urzelai (2000)Evolving Adaptive Neural Networks with and Without Adaptive Synapses by Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen (2003)

More Related