Advanced Search Techniques CS570 Lecture Notes by Jin Hyung Kim Computer Science Department KAIST

1. Advanced Search Techniques CS570 Lecture Notes by Jin Hyung Kim Computer Science Department KAIST

2. Optimization and Search • Search for maximal value of Objective function find i = MAX { Obj(pi) } where pi is parameter • Traveling Sales Person problem • find minimal distance path • Best-first search keeping track of maximal object function value

3. Search for Optimal Parameter • Finding Optimal Weight matrix = Neural Network training • Deterministic Method • Step-by-step procedure • Hill-Climbing search • gradient search = error back propagation algrthm • Statistical Method • Pseudo-random change • retain if it improve

4. Hill Climbing Search 1. Set n to be the initial node 2. If obj(n) > max { obj(childi (n)) } then exit 3. Set n to be the highest-value child of n 4. Return to step 2 • Terminate at Local maxima • no guarantee on Global optimal • Gradient Search • Hill climbing for continuous, differentiable functions • step width ? • Slow in near optimal

5. Avoiding Local Optimal Problem • Many starting points • Momentum factor • Add Stochastic flavor • Simulated Annealing

6. Simulated Annealing • Solves combinatorial optimization • variant of Metropolis algorithm • by S. Kirkpatric (83) • finding minimum-energy solution of a neural network = finding low temperature state of physical system • To overcome local minimum problem • Instead always going downhill, try to go downhill ‘most of the time’

7. Iterative + Statistical • Simple Iterative Algorithm (TSP) 1. find a path p 2. make p’, a variation of p 3. if p’ is better than p, keep p’ as p 4. goto 2 • Metropolis Algorithm • 3’ : if (p’ is better than p) or (random < Prob), then keep p’ as p • a kind of Monte Carlo method • Simulated Annealing • T is reduced as time passes

8. About T • Metropolis Algorithm • Prob = p(DE) = exp ( DE / T) • Simulated Annealing • Prob = pi(DE) = exp ( DE / Ti) • if Ti is reduced too fast, poor quality • if Tt >= T(0) / log(1+t) - Geman • System will converge to minimun configuration • Tt = k/1+t - Szu • Tt = a T(t-1) where a is in between 0.8 and 0.99

9. Function Simulated Annealing current select a node (initialize) fort 1 todo Tschedule[t] ifT=0 thenreturncurrent next a random selected successor of current E  value[next] - value[current] ifE > 0 thencurrentnext elsecurrentnext only with probability eE /T

10. Genetic Algorithm / Evolutionary Strategy • Motivated as natural selection and genetics • mechanisms of evolution • J. Holland, Adaptation in Natural and Artificial Systems, U of Michigan Press, 1975 • one compete for opportunity of reproduction • survival of fittest individuals • a.k.o. Probabilistic Search Algorithm • cost minimization • high probability of locating global optimal solution • Int’l conference on Genetic Algorithms

11. Natural Selection • Genetic contents = survival capacity = features • each feature = gene • set of gene = chromosome • Generate-and-Test type algorithm • Evolution = change of species’ feature • “Survival of the fittest” • Gene of the fittest survive, gene of weaker die out • Reproduction • new combinations of genes are generated from parents - crossover

12. Genetic Algorithm • Encoded representation of solution as bit strings • reproduction • selected for next generation • crossover • generic material is exchanged • mutation • random alteration of bits of strings • each solution is associated with fitness value

13. GA Structure Initialize population Termination Cond ? Select solutions for next population Perform Crossover and Mutation Evaluate population

14. GA Component • Population of binary strings • Control parameters • Fitness function • Genetic operators • crossover • mutation • Selection Mechanism • Mechanism to encode solution to binary string

15. Encoding Mechanism • Encode each solution to unique binary string • TSP problem • path representation • [5 4 3 1 2] - city 5 first, then city 4 , …. • Order representation • order in the remaining city list • [5 4 3 1 2] = (5 4 3 1 1) • [1 2 3 4 5] = (1 1 1 1 1) • The representation want to be valid after crossover • Partially matched crossover (p 159)

16. Fitness Function • Objective function that optimized • fitness of a string • normalize to range of 0 to 1, if possible • Selection is based on the evaluation

17. Selection • Models nature’s survival-of-the-fittest • fitter string has more offspring • high chance of surviving • Proportionate selection scheme • fi / aver(f) • Monte Calro method to handle fractions • Ranking Scheme - predefined surv. Prob. • Elite Saving Scheme - local minima • Tournament Selection • etc.

18. Crossover • Pairs of strings are picked up at random • Single-point crossover • randomly cut into two parts • exchange to form a new string • Multiple-point crossover • a variant is Uniform Crossover - using bit mask • (Alternative) if random > Pc (Xover-rate) Crossover otherwise remain unaltered

19. Mutation • (independently) Flipping bits with random • In order to get diversity • controlled by Mutation rate, Pm • if no mutation, only space governed by initial population is searched • If Pm is too large, a random search • Dynamic change of Pm ge • Adaptive mutation • depending on distance of crossover result • near ->more mutation

20. Generation Model • Discrete generation model • Continuous generation model • asynchronous operation • one-at-the-time model • Typical parameter values • population size : 30 -200 • crossover rate : 0.5 -1 • mutation rate : 0.001 - 0.05

21. 1 1 0 1 1 0 1 1 0 1 0 0 1 0 0 1 1 1 0 1 1 0 1 1 1 0 1 1 1 0 0 1 1 1 1 1 1 0 0 1 1 0 0 1 1 0 1 1 1 1 1 1 0 0 0 1 1 0 0 0 1 0 1 1 Generation n 2 9 11 13 selection crossover mutation 8 1 15 13 Generation n+1

22. Genetic Algorithm (defun distribution (population) (let* ((genotypes (noduplicates population)) (sum (apply #’+ (mapcar #’fitness genotypes)))) (mapcar #’(lambda (x) (cons (/ (fitness x) sum) x)) genotypes))) (defun reproduce (population) (let ((offstring nil)(d (distribution population))) (dotimes (I (/ (length population) 2)) (let ((x (select d))(y (select d))) (crossover x y) (setq offstring (nconc (list x y) offstring)))) offstring))

23. (defun select (distribution) (let ((random (random 1.0))(prob 0) genotype) (some #’(lambda (pair) (setq prob (+ prob (first pair))) (if (> random prob) nil (setq genotype (rest pair)))) distribution) (mutate genotype))) (defun mutate (genotype) (mapcar #’(lambda (x) (if (> (random 1.0) 0.03) x (if (= x 1) 0 1))) genotype))) (defun crossover (x y) (if (> (random 1.0) 0.6) (list x y) (let* ((site (random (length x))) (swap (rest (nthcdr site x)))) (setf (rest (nthcdr site x))(rest (nthcdr site y)) (rest (nthcdr site y)) swap))))

24. Artificial Life • "The study of man-made systems that exhibit behaviors characteristic of natural living systems." Artificial Life is a field of study devoted to understanding life by attempting to abstract the fundamental dynamical principles underlying biological phenomena, and recreating these dynamics in other physical media -- such as computers -- making them accessible to new kinds of experimental manipulation and testing. - C. G. Langton • "Artificial Life is the enterprise of understanding biology by constructing biological phenomena out of artificial components, rather than breaking natural life forms down into their component parts. It is the synthetic rather than the reductionist approach." - T. S. Ray

25. Artificial Life related topics • Autonomous Agents • Cellular Automata • Genetic Algorithms / Programming • Neural Networks • Simulators

26. Complex Adaptive Systems • Within science, complexity is a watchword for a new way of thinking about the collective behavior of many basic but interacting units, be they atoms, molecules, neurons, or bits within a computer. To be more precise, our definition is that complexity is the study of the behavior of macroscopic collections of such units that are endowed with the potential to evolve in time. Their interactions lead to coherent collective phenomena, so-called emergent properties that can be described only at higher levels than those of the individual units. In this sense, the whole is more than the sum of its components... Peter Coveney and Roger Highfield, Frontiers of Complexity: The Search for Order in a Chaotic World, 1995.

27. John Conway's Game of Life • played on a field of cells, each of which has eight neighbors (adjacent cells). A cell is either occupied (by an organism) or not. The rules for deriving a generation from the previous one are these: • Death : If an occupied cell has 0, 1, 4, 5, 6, 7, or 8 occupied neighbors, the organism dies (0, 1 neighbors: of loneliness; 4 thru 8: of overcrowding). • Survival : If an occupied cell has two or three neighbors, the organism survives to the next generation. • Birth : If an unoccupied cell has three occupied neighbors, it becomes occupied. • http://www.tech.org/~stuart/life/life.html