1 / 71

Biologically Inspired Intelligent Systems

Biologically Inspired Intelligent Systems. Lecture 13 Roger S. Gaborski. Today’s Topics. Function Optimization Genetic Algorithms. Optimization. E xamine several different functions

noe
Download Presentation

Biologically Inspired Intelligent Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Biologically Inspired Intelligent Systems Lecture 13 Roger S. Gaborski

  2. Today’s Topics • Function Optimization • Genetic Algorithms Roger S. Gaborski

  3. Optimization • Examine several different functions • Develop solutions using different biologically inspired algorithms to find maximum (or minimum) of each function Roger S. Gaborski

  4. Testing Function Description and Implementation

  5. Agenda • 1) Function List • 2) Function Equation Presentation and Plots • 3) Function Implmentation with Matlab Roger S. Gaborski

  6. Function ListThese are common test functionGoogle for more information • 1)Michalewicz’s Function • 2)Rosenbrock’s Function • 3)DeJong's function • 4)Schwefel’s Function • 5)Ackley’s Function • 6)Rastrigin’s Function • 7)Easom’s Function • 8)Griewank’s Function • 9)Shubert’s Function • 10)Yang’s Function Roger S. Gaborski

  7. Michalewicz’s Function n dimensional function has n! ‘s global minima f(x,y) = -(sin(x)*(sin(2*x^2/pi)^20) + sin(y)*(sin(2*y^2/pi)^20)); Use range [0,3] for x and y

  8. Michalewicz’s Function

  9. Rosenbrock’s Function n-dimensional function has the global minimum f(x)=0; x(i)=1, i=1:n. f(x,y) = ((1-x)^2 + 100*(y-(x)^2)^2); Use range for x, y [-2,+2]

  10. Rosenbrock’s Function

  11. De Jong’s Function n-dimensional function has the global minimum f(x)=0, x(i)=0, i=1:n. Use range for x, y [-2,+2]

  12. De Jong’s Function

  13. Schwefel’s Function f(x,y) = x*sin(sqrt(abs(x))) + y*sin(sqrt(abs(y))); Use range for x, y [-500,+500]

  14. Schwefel’s Function

  15. Ackley’s Function f(x,y) = (-20*exp(-0.2*sqrt(0.5*(X(x)^2+Y(y)^2)))-exp(0.5*(cos(2*pi*X(x))+cos(2*pi*Y(y))))+20+exp(1)); Use range for x, y [-2,+2]

  16. Ackley’s Function

  17. Rastrigin’s Function f(x,y) =(-1)*(X(x).^2 + Y(y).^2 - 10*cos(2*pi*X(x)) - 10*cos(2*pi*Y(y)) + 20); Use range for x, y [-5,+5]

  18. Rastrigin’s Function

  19. Easom’sFunction f(x,y) =-1*(cos(x)*cos(y)*exp(-(x-pi)^2-(y-pi)^2)); Use range for x, y [-10,+10]

  20. Easom’s Function

  21. Griewank’s Function f(x,y) = (-1)*((x.^2 + y.^2)/4000 – cos(x).*cos(y)/sqrt(2)) + 1); Use range for x, y [-5,+5]

  22. Griewank’s Function

  23. Shubert’s Function f(x,y) =(1)*(((1*cos((1+1)*x+1))+(2*cos((2+1)*x+2))+(3*cos((3+1)*x+3))+(4*cos((4+1)*x+4))+(5*cos((5+1)*x+5))).*((1*cos((1+1)*y+1))+(2*cos((2+1)*y+2))+(3*cos((3+1)*y+3))+(4*cos((4+1)*y+4))+(5*cos((5+1)*y+5)))); Use range for x, y [-5,+5]

  24. Shubert’s Function

  25. Yang’s Function f(x,y) = ((exp(-(x/15)^10-(y/15)^10) - 2*exp(-x^2-y^2))*(cos(x)^2)*(cos(y)^2)); Use range for x, y [-5,+5]

  26. Yang’s Function

  27. Reference 1) http://www.geatbx.com/docu/fcnindex-01.html (include description on Michalewicz's function, Rosenbrock’s function, De Jong's function, Schewefel’s function ,Easom’s function, Ackley’s function, GriewankFunction,Rastrigin's function) 2) http://www.it.lut.fi/ip/evo/functions/node28.html (include description on shubert’s function, the minimum value is -186.7309 ) 3) http://wapedia.mobi/en/Xin-She_Yang's_functions (include description on Yang’s function, the global minimum is wrong, it should be -1 rather than 0) 4)Yang, X. S. (2009). "Firefly algorithms for multimodal optimization". Stochastic Algorithms: Foundations and Applications, SAGA 2009. Lecture Notes in Computer Sciences. 5792. pp. 169–178. (http://arxiv.org/abs/1003.1466v1)

  28. Some General Ideas Roger S. Gaborski

  29. Search Space • Potential solutions to a problem • Any point in the search space defines a potential solution • ‘Search’ – navigating through the search space • ‘Evolutionary Search’ is inspired by nature Roger S. Gaborski

  30. Populations • Evolutionary algorithms consider a large number, or population, of potential solutions at once • Use the whole population, or a subset of the population, to help navigate through the search space in search of the ‘optimal’ solution • Making use of previously evaluated solutions Roger S. Gaborski

  31. Populations • Perform search by evolving solutions • Maintain a population of potential solutions • Breed better solutions in the population • Keep ‘children’ that are created • Remove poorer performing solutions • Evolve solutions for a given number of generations or until an acceptable solution is found Roger S. Gaborski

  32. Genetic Algorithms • Holland – explained adaptive processes of natural systems and design artificial systems based on natural systems • Most widely used evolutionary algorithm • GAs use two spaces: • Search space: space of coded solutions • Coded solutions genotypes • Solution space; space of actual solutions • Actual solutions  phenotypes • Fitness is evaluated on phenotype solutions Roger S. Gaborski

  33. Genetic Algorithms • Maintain of population of individuals • Each individual consists of genotype and its phenotype • Genotypes are coded versions of parameters • A coded parameter is a gene • Collection of genes in one genotype is a chromosome • Phenotypes are the actual set of parameters Roger S. Gaborski

  34. Genetic Algorithms • GAs do not use the representation of the parameter space directly • The population consists of individuals commonly referred to as chromosomes • The genotypes are represented as binary strings • Genetic operators are applied to the binary strings • The most common operator is crossover Roger S. Gaborski

  35. Search Space and Solution Space Search Space Solution Space 11100110 GENOTYPES 11100000 11011000 14x+6y PHENOTYPES 14x+4y 13x-8y NOTE: Only the numerical values are determined by the genotype (ai,bi) ai x + bi y Evaluate Fitness of Each Solution Roger S. Gaborski

  36. Terminology • Interpretation and Evaluation • Selection and Reproduction • Variation • Reproduction Roger S. Gaborski

  37. Terminology • Interpretation and Evaluation • Decode binary strings into decimal values, such as, the x and y coordinates • Coordinates are evaluated using the objective function Roger S. Gaborski

  38. Terminology • Interpretation and Evaluation • Selection and Reproduction • Select two individuals from the current population • Many methods are available to implement the selection step of the algorithm Roger S. Gaborski

  39. Terminology • Variation • Two selected bit strings are modified by a crossover and mutation operator • Crossover – randomly select a position in the binary string. Create first child by recombining the first section from parent 1 with the second section from string 2 • The second child is forms by combining the second section of parent 1 with the first section of parent 2 • Mutation is implemented by simply selecting a bit and flipping its value, 01, 10 • Application of the operators is determined by a probability • Both classes of operators are biologically inspired Roger S. Gaborski

  40. Basic GA Operators • SELECTION • Out of an initial population of individuals, how do you select parents that will be used to breed the next population? • Randomly – just select two parents • Based on fitness – the higher an individuals fitness, the more likely it will be chosen as a parent • Tournament Selection- randomly select k individuals from the population. Return the best r, r can equal 1 • NOTE: Fitness is a potential issue, especially early on – what does it really mean? Roger S. Gaborski

  41. Basic GA Operators • Crossover: After selecting two chromosomes from the population, Parent1: ABCDEFG and Parent2: abcdefg Select a random position (for example, 4), split the two chromosomes at this point, interchange substrings and recombine ABCDEFG and abcdefg child1: ABCDefg child2: abcdEFG Roger S. Gaborski

  42. Basic GA Operators • Mutation: Randomly change the value of one element of the chromosome ABCDefg ABKDefg Roger S. Gaborski

  43. Simple Crossover and Mutation Example Crossover: Parent1: 011011101011 Parent2: 100110110101 Choose crossover point as position 3 Parent1: 011011101011 Parent2: 100110110101 Child1: 011110110101 Child2: 100011101011 Mutation (randomly choose position 8): Child1: 011110110101  011110100101 Roger S. Gaborski

  44. Issues with Single Point Crossover • v1 v2 v3 v4 v5 v6 • With single point crossover the probability is high that v1 and v6 will end up in different children. If the pair v1 and v6 is important to get a high fitness, single point cross with be a poor choice for crossover • Also, it is highly likely that v1 and v2 will remain together in the child. There is only a 1 out of 6 possibility that they will be separated Roger S. Gaborski

  45. Two Point Crossover • Parent1: ABCDEFGHIJK • Parent2: abcdefghijk • Two point crossover, pt1 = 3, pt2 = 6 • Child1: ABCdefGHIJK • Child2: abcDEFghijk • Two point crossover, pt1 = 2, pt2 = 6 • Child1: AbcdefGHIJK • Child2: aBCDEFghijk Roger S. Gaborski

  46. Uniform Crossover • For every position, flip a coin. If heads, flip, if tails, no change: • Parent1: ABCDEFGHIJK • HTTTHTHHTHH • Parent2: abcdefghijk • Child1: aBCDeFghIjk • Child2: AbcdEfGHiJK • Can generate several children by using another probability string Roger S. Gaborski

  47. GAs with Real Valued Vector • Instead of using binary values, use real values. • Crossover • Parents: 12.1 1.4 16.5 18.1 20.7 6.1 -7 -8.2 9.1 -10.1 -Children: 12.1 1.4 16.5 9.1 -10.1 6.1 -7 -8.2 18.1 20.7 Roger S. Gaborski

  48. Other Options with Floating Point Numbers • Instead of swapping values between parents to form children, average values from the two parents • Uniform crossover. For each head location, average the corresponding values Roger S. Gaborski

  49. Points in Space • Number Consider each vector a point in n dimensional space (n, of elements) • Draw a line between the two points. Select points off this line. Allow the line to extend beyond the points X Original Points X Children X Roger S. Gaborski

  50. Mutation with Real Value Vector • Use a Gaussian Random generator to generate a small random number. Add random number to chosen value. • The random number can be scaled in both range and magnitude. If numbers are in the -2.0 to +2.0 range, a mutation value of .1 might be reasonable, but if the numbers are in the 1000 – 2000 range, a random number of 10 might be more reasonable Roger S. Gaborski

More Related