1 / 47

Machine Learning Evolutionary Algorithm (2)

Machine Learning Evolutionary Algorithm (2). Evolution ary Programming. E P quick overview. Developed: USA in the 1960’s Early names: D. Fogel Typically applied to: traditional EP: machine learning tasks by finite state machines contemporary EP: (numerical) optimization

lcranford
Download Presentation

Machine Learning Evolutionary Algorithm (2)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Machine LearningEvolutionary Algorithm (2)

  2. Evolutionary Programming

  3. EP quick overview • Developed: USA in the 1960’s • Early names: D. Fogel • Typically applied to: • traditional EP: machine learning tasks by finite state machines • contemporary EP: (numerical) optimization • Attributed features: • very open framework: any representation and mutation op’s OK • crossbred with ES (contemporary EP) • consequently: hard to say what “standard” EP is • Special: • no recombination • self-adaptation of parameters standard (contemporary EP)

  4. Evolutionary Programming (EP) • There is no fixed structure for representation. • There is only mutation operation, and cross-over is not used in this method. • Each child is determined by its parent in a way of mutation. • So, we can conclude that there are three steps: • Initialize population and calculate fitness values for initial population • Mutate the parents and generate new population • Calculate fitness values of new generation and continue from the second step.

  5. EP technical summary tableau

  6. Prediction by finite state machines • Finite state machine (FSM): • States S • Inputs I • Outputs O • Transition function  : S x I  S x O • Transforms input stream into output stream • Can be used for predictions, e.g. to predict next input symbol in a sequence

  7. FSM example • Consider the FSM with: • S = {A, B, C} • I = {0, 1} • O = {a, b, c} •  given by a diagram

  8. FSM as predictor • Consider the following FSM • Task: predict next input • Quality: % of in(i+1) = outi • Given initial state C • Input sequence 011101 • Leads to output 110111 • Quality: 3 out of 5

  9. Introductory example:evolving FSMs to predict primes • P(n) = 1 if n is prime, 0 otherwise • I = N = {1,2,3,…, n, …} • O = {0,1} • Correct prediction: outi= P(in(i+1)) • Fitness function: • 1 point for correct prediction of next input • 0 point for incorrect prediction • Penalty for “too much” states

  10. Introductory example:evolving FSMs to predict primes • Parent selection: each FSM is mutated once • Mutation operators (one selected randomly): • Change an output symbol • Change a state transition (i.e. redirect edge) • Add a state • Delete a state • Change the initial state

  11. Genetic Programming

  12. THE CHALLENGE "How can computers learn to solve problems without being explicitly programmed? In other words, how can computers be made to do what is needed to be done, without being told exactly how to do it?"  Attributed to Arthur Samuel (1959)

  13. GP quick overview • Developed: USA in the 1990’s • Early names: J. Koza • Typically applied to: • machine learning tasks (prediction, classification…) • Attributed features: • competes with neural nets and alike • needs huge populations (thousands) • slow • Special: • non-linear chromosomes: trees, graphs • mutation possible but not necessary (disputed!)

  14. GP technical summary tableau

  15. Decision trees If-then production rules Horn clauses Neural nets Bayesian networks Frames Propositional logic Binary decision diagrams Formal grammars Coefficients for polynomials Reinforcement learning tables Conceptual clusters Classifier systems REPRESENTATIONS

  16. A COMPUTER PROGRAM

  17. Introductory example: credit scoring • Bank wants to distinguish good from bad loan applicants • Model needed that matches historical data

  18. Introductory example: credit scoring • A possible model: IF (NOC = 2) AND (S > 80000) THEN good ELSE bad • In general: IF formula THEN good ELSE bad • Only unknown is the right formula, hence • Natural fitness of a formula: percentage of well classified cases of the model it stands for • Natural representation of formulas (genotypes) is: parse trees

  19. AND = > NOC 2 S 80000 Introductory example: credit scoring IF (NOC = 2) AND (S > 80000) THEN good ELSE bad can be represented by the following tree

  20. Tree based representation

  21. Tree based representation (x  true)  (( x  y )  (z  (x  y)))

  22. Tree based representation i =1; while (i < 20) { i = i +1 }

  23. Tree based representation • In GA, ES, EP chromosomes are linear structures (bit strings, integer string, real-valued vectors, permutations) • Tree shaped chromosomes are non-linear structures • In GA, ES, EP the size of the chromosomes is fixed • Trees in GP may vary in depth and width

  24. CREATING RANDOM PROGRAMS

  25. Tree based representation • Symbolic expressions can be defined by • Terminal set T • Function set F (with the arities of function symbols) • Adopting the following general recursive definition: • Every t  T is a correct expression • f(e1, …, en) is a correct expression if f  F, arity(f)=n and e1, …, en are correct expressions • There are no other forms of correct expressions • In general, expressions in GP are not typed (closure property: any f  F can take any g  F as argument)

  26. CREATING RANDOM PROGRAMS • Available functions F = {+, -, *, %, IFLTE} • Available terminals T = {X, Y, Random-Constants} • The random programs are: • Of different sizes and shapes • Syntactically valid • Executable

  27. GA flowchart GP flowchart

  28. Mutation (+ 2 3 (* X 7) (/ Y 5)) + 2 3 * / X 7 Y 5

  29. * X 7 Mutation First pick a random node (+ 2 3 (* X 7) (/ Y 5)) + 2 3 / Y 5

  30. Mutation Delete the node and its children, and replace with a randomly generated program (+ 2 3 (+ (* 4 2) 3) (/ Y 5)) + 2 3 + / * 3 Y 5 4 2

  31. Crossover (+ X (* 3 Y)) (- (/ 25 X) 7) + - X * / 7 3 Y 25 X

  32. Crossover (+ X (* 3 Y)) (- (/ 25 X) 7) Pick a random node in each program + - X * / 7 3 Y 25 X

  33. Crossover (+ X (* (/ 25 X) Y)) (- 3 7) Swap the two nodes + - X * 3 7 / Y 25 X

  34. Mutation cont’d • Mutation has two parameters: • Probability pm to choose mutation vs. recombination • Probability to chose an internal point as the root of the subtree to be replaced • Remarkably pm is advised to be 0 (Koza’92) or very small, like 0.05 (Banzhaf et al. ’98) • The size of the child can exceed the size of the parent

  35. Recombination • Most common recombination: exchange two randomly chosen subtrees among the parents • Recombination has two parameters: • Probability pc to choose recombination vs. mutation • Probability to chose an internal point within each parent as crossover point • The size of offspring can exceed that of the parents

  36. Selection • Parent selection typically fitness proportionate • Over-selection in very large populations • rank population by fitness and divide it into two groups: • group 1: best x% of population, group 2 other (100-x)% • 80% of selection operations chooses from group 1, 20% from group 2 • for pop. size = 1000, 2000, 4000, 8000 x = 32%, 16%, 8%, 4% • motivation: to increase efficiency, %’s come from rule of thumb • Survivor selection: • Typical: generational scheme (thus none) • Recently steady-state is becoming popular for its elitism

  37. Initialisation • Maximum initial depth of trees Dmax is set • Full method (each branch has depth = Dmax): • nodes at depth d < Dmax randomly chosen from function set F • nodes at depth d = Dmax randomly chosen from terminal set T • Grow method (each branch has depth  Dmax): • nodes at depth d < Dmax randomly chosen from F  T • nodes at depth d = Dmax randomly chosen from T • Common GP initialisation: ramped half-and-half, where grow & full method each deliver half of initial population

  38. FIVE MAJOR PREPARATORY STEPS FOR GP • Determining the set of terminals • Determining the set of functions • Determining the fitness measure • Determining the parameters for the run • Determining the method for designating a result and the criterion for terminating a run

  39. Building a Better Mouse • Apply Genetic Programming to the problem of navigating a maze • What are our terminal and function sets? • Function Set ={If-Movement-Blocked, While-Not-At-Cheese*} • Terminal Set ={Move-Forward, Turn-Left, Turn-Right} * While-Not-At-Cheese will be used exclusively as the root node of the parse tree

  40. One possible solution: While not at the cheese If the way ahead is blocked Turn left 90 degrees Otherwise Move forward Turn right 90 degrees Cheese? Blocked? Building a Better Mouse How to get the starving mouse to the cheese? Is there a better solution for this maze? How good is this solution?

  41. A fitness function: Building a Better Mouse • Each function and terminal other than the root node shall cost one unit to execute • If the mouse spends more than 100 units, it dies of hunger • The fitness measure for a program is determined be executing the program, then squaring the sum of the total units spent and the final distance from the exit • A lower fitness measure is preferable to a higher fitness measure Cheese? Blocked?

  42. Building a Better Mouse While not at the cheese (12996) If the way ahead is blocked Turn left 90 degrees Otherwise Move forward one space While not at the cheese (12996) Move forward one space Turn right 90 degrees Turn left 90 degrees

  43. Building a Better Mouse While not at the cheese (12996) If the way ahead is blocked Turn left 90 degrees Otherwise Move forward one space While not at the cheese (12996) Move forward one space Turn right 90 degrees Turn left 90 degrees Mutation: While not at the cheese (12996) If the way ahead is blocked Turn left 90 degrees Otherwise Turn left 90 degrees

  44. Building a Better Mouse While not at the cheese (12996) If the way ahead is blocked Turn left 90 degrees Otherwise Move forward one space While not at the cheese (12996) Move forward one space Turn right 90 degrees Turn left 90 degrees Crossover: While not at the cheese (11664) If the way ahead is blocked Move forward one space Turn right 90 degrees Otherwise Move forward one space While not at the cheese (12996) Turn left 90 degrees Turn left 90 degrees

  45. Fitness measure: 2809 Building a Better Mouse (after 4202 generations, with 1000 programs per generation) While not at the cheese If the way ahead is blocked Turn right 90 degrees Move forward one space Move forward one space Move forward one space Otherwise Move forward one space Turn right 90 degrees Move forward one space Move forward one space Turn left 90 degrees If the way ahead is blocked Turn left 90 degrees Otherwise Move forward one space Is this better?

  46. We seek to find the Boolean function that returns particular Boolean output values (0 or 1). The Boolean even-k-parity function of k Boolean arguments returns T (true=1) if an even number of its Boolean arguments are T, and otherwise returns F (false =0) (fitness function). Let the input are D0, D1, and D2. The output is S. Terminal set: D0, D1 and D2 Function set: AND, OR, NAND, and NOR. Fitness cases: sum of 8-combinations of the three Boolean arguments D0, D1, and D2 (Truth Table) cases do not equals the correct value of the even-3-parity function (error) construct 4 genes for this problem using Genetic Programming compute the fitness for these genes perform one crossover (best two genes) and one mutation operator for worst gene construct the new population Example

  47. Summary

More Related