Introduction to Genetic Algorithms

1. Introduction to Genetic Algorithms

2. Contact Information Name: 任庆生 Office: 教三楼403 Tel: 62932089 E-mail: ren-qs@cs.sjtu.edu.cn

3. Reference 刘勇，康立山，陈毓屏，非数值并行算法(二)—遗传算法，科学出版社，1995 陈国良，王煦法，庄镇泉，王东生，遗传算法及其应用，人民邮电出版社，1996 David E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley Publishing Company, 1989 Holland J H, Adaptation in Natural and Artificial Systems, The MIT Press,1995 ftp.cs.sjtu.edu.cn/ren-qs/教学/遗传算法

4. Grading Homework: 20% Final project: 80% . The deadline is Aug 31, 2004

5. Course Overview An introduction to emulating the problem solving according to nature's method: via evolution, selective breeding, "survival of the fittest". We will present the fundamental algorithms and present several examples, especially some problems that are hard to solve otherwise.

6. Dealing with Hard (NP-Complete) Problems Some Problems We Just Don't Know How to Solve Efficiently! . . . but we do know how to critique a “solution”. Coloring graphs is hard, but counting colors and violations is easy (a violation is two adjacent vertices with the same color). Finding the shortest salesman's path is hard, but measuring a path is easy. Scheduling examinations or assigning teachers to classes is hard, but counting the conflicts (ideally there are none) is easy. Computer programs are hard to write, but counting bugs is easy.

7. GAs Emulate Selective Breeding Designing tender chickens is hard; taste-testing them is easy. Designing thick-skinned tomatoes is hard; dropping is easy. So, the breeders iterate: • Selection: find which one is better and which one is worse. • Crossover: Let the better members breed. • Mutation: X-ray them.

8. Darwin’s Theory of Evolution A process called natural selection, ‘selects’ individuals best adapted to the environment. Those fittest survive longest Characteristics, encoded in genes are transmitted to offspring and tend to propagate into new generations An offspring’s characteristics are partially inherited from parents and partly the result of new genes created during the reproduction process In sexual reproduction (crossover), the chromosomes of offspring are a mix of their parents. Traits found in parents are passed on to their offspring Variations (mutations) are present naturally in all species producing new traits. Over long periods of time, variations can accumulate and produce new species.

9. Some Examples Choosing among 1,500 features for OCR. Scheduling the Chili, NY, annual soccer invitational. N Queens, Graphs, Salesmen, etc., etc.

10. Subsetting 1,500 OCR Features The polynet OCR engine trains and executes rapidly. Performance was competitive. We wanted to embed it in hardware, but it used 1,500 features. We could deal with 300 features. So, we bred high-performance feature subsets.

11. Soccer Scheduling Bill Gustafson's MS Project, May, 1998 The Chili Soccer Association hosts an annual soccer tournament. 131 teams, 209 games, 14 fields, 17 game times.

12. Soccer Scheduling Hard Constraints A field can have one game at a time. A team can only play one game at a time. Teams must play on appropriate size fields. Late games must be played on lighted fields. A team must rest one game period (two is better) between games. Teams can only play when they can be present (some cannot come Friday evening).

13. Soccer Scheduling Soft Constraints A team's games should be distributed evenly over the playing days. Teams should play in at most two playing areas. Each team should play at least once in the main playing area. Teams should play in areas where they have a preference. Games should finish as early as possible on Sunday. Etc...

14. Placing N Non-Attacking Queens Found by my genetic algorithm!

15. Placing N Non-Attacking Queens Queens attack on chess-board rows, columns, and diagonals. Any permutation in N rows avoids row & column attacks. Exhaustive search works for N  10, but N! grows rapidly. A GA can place 1,000 Queens in 1,344 fitness evaluations.

16. 100 Non-Attacking Queens in 130 Fitness Evaluations

17. Graph Coloring: Edge Ends Get Different Colors

18. Graph Coloring = Map Coloring

19. Graph Coloring = Map Coloring

20. Traveling Salesman Route Min. Another classic, hard, NP-complete problem. We tried cities on a HW grid, so best distance is known. Perfection is hard to achieve. A clever algorithm costs O(cities2) to evaluate a fittness. But, we get pretty good answers.

21. Some GA Application Types

22. History Holland Bagley Cavicchio Hollstien De Jong Goldberg Medical image processing Prisoner’s dilemma Holland Others

23. Classes of Search Techniques

24. Traditional Optimization Method • The Problem: Minimize f(x), with x  X  Rn f(x): Continuously differentiable Q. How does an optimization algorithm work? • Analogy: • A blind person with a stick goes down a hill • Knows the current elevation and the slope • Can measure a few more points, one at a time and at a cost • Wants to decide whichdirection to go, and by how far to reach the bottom

25. x2 x3 x1 • Mathematically • Start with an initial point x0, evaluate f(x0), f(x0) (2f(x0)) • Based on the information, select a search direction “d0” emanating from x0 • Find the next point x1 along the direction d0, and evaluate f(x1), f(x1) (2f(x1)) • Repeat the above, and successfully generate x2, x3, .., such that f(x) is decreased at each iteration  Iterative Descent • Key Questions? • Which direction to go? How far? Is the algorithm guaranteed to reach x*? How fast to reach x*? ...

26. Local and Global Minimum • Which of the above is a local minimum point? Global minimum? How to mathematically define these terms? • Local minimum: A, B, and C. Global: C • f(x*)  f(x)  x with ||x – x*|| <  x* is a local minimum • f(x*)  f(x) x  X  x* is a global minimum • Minimum is strict if “” is replaced by “<” • For traditional optimization method, we can always get local ones • For GA we can get the global one in theory.

27. Comments about GA Stochastic algorithm randomness has an essential role in genetic algorithms both selection and reproduction needs random procedures Consider population of solutions evaluates more than a single solution at each iteration assortment, amenable for parallelisation Robustness Ability to perform consistently well on a broad range of problem types no particular requirements on the problems before using GAs

28. Benefits of Genetic Algorithms Concept is easy to understand Modular, separate from application Supports multi-objective optimization Good for “noisy” environments Always an answer; answer gets better with time Inherently parallel; easily distributed

29. Benefits of Genetic Algorithms (cont.) Many ways to speed up and improve a GA-based application as knowledge about problem domain is gained Easy to exploit previous or alternate solutions Flexible building blocks for hybrid applications Substantial history and range of use

30. Uses of GAs GAs (and SAs): the algorithms of despair. Use a GA when you have no idea how to reasonably solve a problem calculus doesn't apply generation of all solutions is impractical but, you can evaluate posed solutions

31. Outline of This GA Course The basic frame of genetic algorithm. More about representation and operators Theory analysis of GA Combinatorial Problem Constraint optimization

32. Outline of This GA Course Genetic algorithm and Artificial neural network Parallel GA Genetic programming

33. Famous Problems & Concepts N Queens Traveling salesman Knight's tour Bin packing Scheduling Function optimization Graph coloring, Ramsey problems Satisfiability