Learning and Evolution: Lessons from the Baldwin-Effect

1. Learning and Evolution:Lessons from the Baldwin-Effect Georg Theiner P747 Complex Adaptive Systems March 11th, 2003

2. Outline • A brief history of modern evolutionary biology • What is the Baldwin Effect? • Hinton & Nowlan's (1987) simulation • JAVA-applet of BE • The trade-offs between phenotypic plasticity and rigidity • Subsequent studies • Discussion

3. Lamarckian Evolution • Published Philosophie Zoologique (1809) • Assumption: Change in the environment causes changes in the needs of organisms living in that environment, which in turn causes changes in their behavior. • Mechanisms of evolution • First Law: Use or disuse causes structures (organs) to enlarge or shrink • Second Law: All such acquired changes are heritable • Example: long legs and webbed feet of wading birds, long neck of giraffe Jean-Baptiste Lamarck (1744-1829)

4. Darwinian Evolution • Published The Origin of Species (1859) • direct manipulation of one's genetic make-up impossible • acquired characteristics are not directly passed on to offspring • Mechanism of evolution: • Genetic variation in species through random mutations • Natural selection operates on phenotypes Charles Darwin (1809-82)

5. Baldwinian Evolution • Published "A New Factor in Evolution" (1896) • Independently identified by Baldwin, Morgan, and Osborn in 1896 • New factor = phenotypic plasticity: the ability of an organism to adapt to its environment during its lifetime • Examples: ability to learn, to increase muscle strength with exercise, to tan with exposure to sun James Mark Baldwin (1861-1934)

6. The Baldwin Effect • A cluster of effects emerging from an interaction between 2 adaptive processes: • genotypic evolution of population (global search) • individual organism's phenotypic flexibility (local search) • Concerned with benefits and costs of lifetime learning • lifetime learning can alter the genetic composition of an evolving population

7. Hypothesized examples: bird song (Simpson 1953) human language capacity (Pinker and Bloom 1990, Deacon 1997) consciousness, intelligence (Dennett 1991, 1995) learning capacity eventually becomes genetically encoded  resembles Lamarckian sequence consistent with Darwinian mechanism for inheritance of traits

8. The Baldwin Effect, Step 1 • Evolutionary value of learning: accelerates evolution of an adaptive trait • As a result of mutation, an organism becomes capable of learning how to do X • Learning how to do X increases an organism's fitness • Creates new selective pressures: because selection is now also working on the ability to perform X. • Since the successful X-er has greater reproductive success, eventually the population may consist entirely of individuals able to learn how to do X.

9. The Baldwin Effect, Step 2 • Since learning can be costly, evolution favors rigid solutions in which acquiring X is part of an organism's genetic make-up (phenotypic rigidity) • Chance of reproductive success be proportional to how quickly (reliably) X can be learnt • New selective pressures cause competition between slow and fast learners • Some individuals are innately better equipped for performing X, have reproductive advantage • Eventually, capacity to X comes under direct genetic control = genetic assimilation, canalization of a trait (Waddington 1942)

10. Hinton & Nowlan Simulation (1987) • Organism with neural net, 20 connections (phenes) • 20 genes, one-to-one mapping on phenes • Each gene can have 3 alleles • 0 = no connection • 1 = connection • ? = undetermined, learning • one Good Phenotype: net works just in case all nodes are connected • one Good Genotype: all 1's

11. "Needle in a haystack"-fitness landscape • Evolutionary search modeled by GA • Population of 1000 organisms • Each allele is randomly initialized • p = 0.5 for ? • p = 0.25 for 0 and 1 • performs no better than random fitness combination of alleles

12. Problem of passing on the good genome • Even if good solution discovered, not easily passed on • unless fit organism finds very-close-to-fit mate, good genome will be destroyed • expected number of good (immediate) offspring < 1 • can be bypassed in artificial simulations using elitism operator, asexual reproduction

13. The importance of lifetime learning • Augment evolutionary search with phenotypic plasticity • Each organism performs 1000 learning trials during lifetime • learning mechanism: random guess • if correct net is found, stop; else keep searching • all phenes equally hard to learn • requires that organism recognizes the correct solution

14. Use a version of Holland's GA (1975) Perform 1000 matings Selection algorithm: Roulette Wheel Select parents with probability proportional to fitness Fitness function F of an individual A in a population i is F(A[i]) = 1 + [(G – g) / G] * (N – 1) G = number of allowed guesses g = number of guesses until solution found N = length of genotype in our case: 1 + (19n/1000) Determine next generation

15. Wheel is spun twice (2 parents) for each mating, single offspring is generated • cross-over point for combining parental alleles is chosen randomly • offspring inherit only genome, never learnt connection settings • Model parameters are fine-tuned • typical genotype has about 10 connections genetically determined (0's or 1's) • about 2^10 learning trials

16. Results 1 • Phenotypic plasticity smoothes "needle in a haystack" fitness landscape • by allowing an organism to explore neighboring regions of phenotypic space • no unlikely saltations necessary to climb fitness peak

17. Results 2 • if no phenotypic plasticity, about 2^20 (~ 1 million) organisms have to be produced to succeed in search • with learning, finding the correct net requires only 16 x 1000 organisms • little selection pressure to fix all phenes genetically

18. JAVA-Simulation (Watson and Wiles 2001) • Run with "Show all data" check-box to see frequency of 0's and ?'s • Alter random number seed • Additional evolutionary operators • mutation • chance (as specified in Advanced Options) that a given allele will be flipped to either 0, 1, or ? (with equal p) • maintain diversity, avoid local fitness maxima • elitism • forces best individual of each population to be included unchanged in next generation

19. Alternative Selection Algorithms • Ranked Roulette Wheel • slice of wheel is proportional to ranked fitness • minimizes real differences in fitness • less selection bias for top-fit individuals • Tournament • randomly picks 2 individuals from population, chooses fitter one with p = k (as set in Advanced Options) • runs much faster • preserves genetic diversity much longer • Standard combinations for optimization algorithms • Standard roulette without elitism • tournament with elitism

20. Fundamental insight of BE • Trade-offs between learning (plasticity) and instinct (rigidity)

21. French and Messinger (1994) • amount of plasticity and amount of benefit of learnt behavior is crucial to size of BE • having blue eyes vs. humming Middle C vs. winking • x-axis: agent's normalized distance from Good Gene (number of bits differing by total number of bits) • y-axis: probability of learning the Good Phene

22. BE is significant only for a narrow window of plasticity • if too low or too high, virtually no convergence towards Good Gene

23. Mayley (1996a, 1996b, 1997) • Possible selective disadvantage of learning: Hiding Effect • phenotypic fitness differentials are compensated by learning capacity • genetic differences are hidden from selection by learning • trade-offs between Baldwin and Hiding effect

24. Discussion • Unrealistic assumptions about fitness landscape • extremely rugged fitness landscape makes pure evolutionary search very hard • How smooth are real search spaces? • Unrealistic assumption about learning mechanism • instead e.g. use hillclimbing procedure for local optimization • enhances BE only if learning procedure is not too sophisticated, otherwise insufficient selective pressure for hard-wiring

25. Learning trials are "cheap" genetic experiments • but biological reality of those two search strategies differs in many respects • Unrealistic assumption about genome-phenome mapping • mapping could be one-to-many • genetic specification and successful guessing of a trait are treated interchangeably • transformation of phenotype to genotype (development) is trivialized

26. Do we need an explicit fitness function? • French & Messinger (1994): introduce spatial dimension • consider 3 areas of plasticity: Good Phene = more efficient metabolism, movement, reproduction • world determines fitness of a given genotype • Using simple models to understand complex phenomena • Controlled experiments are practically unfeasible • How simple is too simple?

27. Selective Bibliography on BE