Genetic Algorithms

1. Genetic Algorithms

2. What Do the Following 3 Things Have in Common?

3. Genetic Algorithms (GAs) • GAs design jet engines. • GAs draw criminals. • GAs program computers.

4. A Potpourri of Applications • General Electric’s Engineous(generalized engineering optimization). • Face space (criminology). • Genetic programming (machine learning).

5. Gas Turbine Design • Jet engine design at General Electric (Powell, Tong, & Skolkick, 1989) • Coarse optimization - 100 design variables. • Hybrid GA + numerical optimization + expert system. • Found 2% increase in efficiency. • Spending $250K to test in laboratory. • Boeing 777 design based on these results.

6. Engineous • A new software system called Engineous combines artificial intelligence and numerical methods for the design and optimization of complex aerospace systems. Engineous combines the advanced computational techniques of genetic algorithms, expert systems, and object-oriented programming with the conventional methods of numerical optimization and simulated annealing to create a design optimization environment that can be applied to computational models in various disciplines. • Engineous has produced designs with higher predicted performance gains that current manual design processes - on average a 10-to-1 reduction of turnaround time - and has yielded new insights into product design. It has been applied to the aerodynamic preliminary design of an aircraft engine turbine, concurrent aerodynamic and mechanical preliminary design of an aircraft engine turbine blade and disk, a space superconductor generator, a satellite power converter, and a nuclear-powered satellite reactor and shield. Source: Tong, S.S. ; Powell, D. ; Goel, S. Integration of artificial intelligence and numerical optimization techniques for the design of complex aerospace systems

7. Engineous Engineous Software, Inc. provides process integration and design optimization software solutions and services. The company's iSIGHT software integrates key steps in the product design process, then automates and executes those steps through design exploration tools like optimization, DOE, and DFSS techniques. The FIPER infrastructure links these processes together in a unified environment. Through FIPER, models, applications and "best" processes are easily shared, accessed, and executed with other engineers, groups, and partners. Engineous operates numerous sales offices in the U.S., as well as wholly owned subsidiaries in Asia and Europe. Customers include leading Global 500 companies such as Canon, General Electric, General Motors, Pratt & Whitney, Honeywell, Lockheed Martin, Toshiba, MHI, Ford Motor Company, Chrysler, Toyota, Nissan, Renault, Hitachi, Peugeot, MTU Aero Engines, TRW, BMW, Rolls Royce, and Johnson Controls, Inc. Additional information may be found at www.Engineous.com and http://components.Engineous.com.

8. Recent news on Engineous Dassault Systèmes to Acquire Engineous Software 06/17/2008 Paris, France, and Providence, R.I., USA, June 17, 2008 – Dassault Systèmes (DS) (Nasdaq: DASTY; Euronext Paris: #13065, DSY.PA), a world leader in 3D and Product Lifecycle Management (PLM) solutions and Engineous Software, a market leader in process automation, integration and optimization, today announced an agreement in which DS would acquire Engineous Software. This acquisition will extend SIMULIA’s leadership in providing Simulation Lifecycle Management solutions on the V6 IP collaboration platform. The proposed acquisition, for an estimated price of 40 million USD, should be completed before the end of July subject to specific closing conditions. http://www.3ds.com/company/news-media/press-releases-detail/release//single/1789/?no_cache=1

9. Criminal-likeness Reconstruction • No closed form fitness function (Caldwell & Johnston, 1991). • Human witness chooses faces that match best. • GA creates newfaces from whichto choose.

10. What are GAs? • GAs are biologically inspired class of algorithms that can be applied to, among other things, the optimization of nonlinear multimodal functions. • Solves problems in the same way that nature solves the problem of adapting living organisms to the harsh realities of life in a hostile world: evolution. Let’s watch a video...

11. What is a Genetic Algorithm (GA)? • A GA is an adaptation procedure based on the mechanics of natural selection and genetics. • GAs have 2 essential components: • Survival of the fittest (selection) • Variation

12. Nature as Problem Solver • Beauty-of-nature argument • How Life Learned to Live (Tributsch, 1982, MIT Press) • Example: Nature as structural engineer

13. Owl Butterfly

14. Evolutionary is Revolutionary! • Street distinction evolutionary vs. revolutionary is false dichotomy. • 3.5 Billion years of evolution can’t be wrong. • Complexity achieved in short time in nature. • Can we solve complex problems as quickly and reliably on a computer?

15. Why Bother? • Lots of ways to solve problems: • Calculus • Hill-climbing • Enumeration • Operations research: linear, quadratic, nonlinear programming • Why bother with biology?

16. Combinatorial Problem • deals with the study of finite or countable discrete structures • deciding when certain criteria can be met, and constructing and analyzing objects meeting the criteria e.g. Satisfiability Is there a truth assignment to the boolean variables such that every clause is satisfied? http://www.cs.sunysb.edu/~algorith/files/satisfiability.shtml

17. Non-linear Programming nonlinear programming (NLP) is the process of solving a system of equalities and inequalities, collectively termed constraints, over a set of unknown real variables, along with an objective function to be maximized or minimized, where some of the constraints or the objective function are nonlinear Consider the following nonlinear program: minimise x(sin(3.14159x)) subject to 0 <= x <= 6 In the diagram above there are many local minima; that is, points at the bottom of some portion of the graph http://people.brunel.ac.uk/~mastjjb/jeb/or/nlp.html

18. Why bother with biology? Gradient search technique • Robustness= Breadth+Efficiency. • A hypothetical problem spectrum:

19. GAs Not New Professor of Psychology and Electrical Engineering & Computer Science Ph.D. University of Michigan Area: Cognition and Cognitive Neuroscience • John Holland at University of Michiganpioneered in the 50s. • Other evolutionaries: Fogel, Rechenberg, Schwefel. • Back to the cybernetics movement and early computers. • Reborn in the 70s. http://www.lsa.umich.edu/psych/people/directory/profiles/faculty/?uniquename=jholland

20. Genetic algorithms • Variant of local beam search with sexual recombination.

21. How GAs are different from traditional methods? • GAs work with a coding of the parameter set, not the parameter themselves. • GAs search from a population of points, not a single point. • GAs use payoff (objective function) information, not derivatives or other auxilliary knowledge. • GAs use probabilistic transition rules, not deterministic rules.

22. Traditional Approach Problem:Maximise the function f(S1, S2, S3, S4, S5) = sin(S1)2 * sin(S2)2+s3 - loge(S3)*S4-S5 Traditional approach: twiddle with the switch parameters. y = f(S1, S2, S3, S4, S5) output S1, S2, S3, S4, S5 Setting of five switches

23. Genetic Algorithm Problem:Maximise the function f(S1, S2, S3, S4, S5) = sin(S1)2 * sin(S2)2+s3 - loge(S3)*S4-S5 • Natural parameter set of the optimisation problem is represented as a finite-length string y = f(S1, S2, S3, S4, S5) GA doesn’t need to know the workings of the black box. Setting of five switches f(x) output 0 31 x S1, S2, S3, S4, S5

24. Main Attractions of Genetic Algorithm GA Traditional Optimization Approaches • Simplicity of operation and power of effect • unconstrained • Limitations: continuity, derivative existence, unimodality • work from a rich database of points simultaneously, climbing many peaks in parallel • move gingerly from a single point in the decision space to the next using some transition rule • population of strings = points • Population of well-adapted diversity

25. GA maximise 2x1 + x2 - 5loge(x1)sin(x2)subject tox1x2 <= 10 | x1 - x2 | <= 20.1 <= x1 <= 5 0.1 <= x2 <= 3 • works from a rich database of points simultaneously, climbing many peaks in parallel Source: Nonlinear Programming, by J E Beasley http://people.brunel.ac.uk/~mastjjb/jeb/or/nlp.html

26. Initial Population

27. Genetic Algorithm GA • Initial Step: random start using successive coin flips 01101 11000 01000 10011 GA uses coding population GAs are blind, only payoff values associated with individual strings are required • blind to auxiliary information • Searches from a population Uses probabilistic transition rules to guide their search towards regions of the search space with likely improvement

28. Reproduction

29. Genetic Algorithm REPRODUCTION • Selectionaccording to fitness 01101 11000 01000 10011 GA uses coding population Weighted Roulette wheel Mating pool (tentative population) • Replication CROSSOVER • Crossover– randomized information exchange Crossover point k = [1, l-1] GA builds solutions from the past partial solutions of the previous trials

30. Mutation

31. Genetic Algorithm MUTATION • Reproductionand crossover may become overzealous and lose some potentially useful genetic material • Mutation protects against irrecoverable loss; it serves as an insurance policy against premature loss of important notions • Mutation rates: in the order of 1 mutation per a thousand bit position transfers

32. Sample Problem

33. Genetic Algorithm: Example Problem:Maximise the function f(x) = x2 on the integer interval [0, 31] • Coding of decision variables as some finite length string X as binary unsigned integer of length 5 [0, 31] = [00000, 11111] • Constant settings Pmutation=0.0333 DeJong(1975) suggests high crossover Probability, low mutation probability (inversely proportional to the pop.size), and A moderate population size Pcross=0.6 Population Size=30

34. Genetic Algorithm: Example SAMPLE PROBLEM • Maximize f(x) = x2; where x is permitted to vary between 0 and 31 • Select initial population at random (use even numbered population size) Sum 1170 Ave. 293 Max. 576

35. Genetic Algorithm: Example SAMPLE PROBLEM • Maximize f(x) = x2; where x is permitted to vary between 0 and 31 • Reproduction: select mating pool by spinning roulette wheel 4 times. pselect 14.4% 01101 11000 01000 10011 30.9% 5.5% 49.2 The best get more copies. The average stay even. The worst die off. Weighted Roulette wheel

36. Choosing offspring for the next generation • intSelect(intPopsize, double Sumfitness, Population Pop){ • partSum = 0 • rand=Random * Sumfitness • j=0 • Repeat • j++; • partSum = partSum + Pop[j].fitness • Until (partSum >= rand) or (j = Popsize) • Return j • } [0,1]

37. Genetic Algorithm SAMPLE PROBLEM • Crossover – strings are mated randomly using coin tosses to pair the couples - mated string couples crossover using coin tosses to select the crossing site Sum 1754 Ave. 439 Max. 729

38. Algorithmic Steps

39. The Genetic Algorithm • Initialize the algorithm. Randomly initialize each individualchromosomein the population of size N (N must be even), and compute each individual’s fitness.

40. The Genetic Algorithm • Initialize the algorithm. Randomly initialize each individual chromosome in the population of size N (N must be even), and compute each individual’s fitness. • SelectN/2pairsof individuals for crossover. The probability that an individual will be selected for crossover is proportional to its fitness.

41. The Genetic Algorithm • Initialize the algorithm. Randomly initialize each individual chromosome in the population of size N (N must be even), and compute each individual’s fitness. • Select N/2 pairs of individuals for crossover. The probability that an individual will be selected for crossover is proportional to its fitness. • Performcrossoveroperation on N/2pairs selected in Step1. Randomlymutatebits with a small probability during this operation.

42. The Genetic Algorithm • Initialize the algorithm. Randomly initialize each individual chromosome in the population of size N (N must be even), and compute each individual’s fitness. • Select N/2 pairs of individuals for crossover. The probability that an individual will be selected for crossover is proportional to its fitness. • Perform crossover operation on N/2 pairs selected in Step1. Randomly mutate bits with a small probability during this operation. • Computefitnessof all individuals in new population.

43. The Genetic Algorithm • (Optional Optimization) SelectNfittestindividuals from combined population of size 2N consisting of old and newpopulations pooled together.

44. The Genetic Algorithm • (Optional Optimization) Select N fittest individuals from combined population of size 2N consisting of old and new populations pooled together. • (Optional Optimization) Rescalefitnessof population.

45. The Genetic Algorithm • (Optional Optimization) Select N fittest individuals from combined population of size 2N consisting of old and new populations pooled together. • (Optional Optimization) Rescale fitness of population. • Determinemaximumfitnessof individuals in the population. If |max fitness – optimum fitness| < toleranceThenStop Else Go to Step1.

46. A Simple GA Example

47. Let’s see a demonstration for a GA that maximizes the function n =10 c = 230 -1 = 1,073,741,823

48. Simple GA Example Fitness Function or Objective Function • Function to evaluate: • coeff – chosen to normalize the x parameter when a bit string of length lchrom =30 is chosen. • When the x value is normalized, the max. value of the function will be: This happens when for the case when lchrom=30

49. Test Problem Characteristics • With a string length=30, the search space is much larger, and random walk or enumeration should not be so profitable. • There are 230=1.07(1010) points. With over 1.07 billion points in the space, one-at-a-time methods are unlikely to do very much very quickly. Also, only 1.05 percent of the points have a value greater than 0.9.

50. Comparison of the functions on the unit interval x2 f(x) 0 31 x x10 f(x) 0 1,073,741,823 x