1 / 16

Iterative Improvement Algorithms

Iterative Improvement Algorithms. For some problems, path to solution is irrelevant: just want solution Start with initial state, and change it iteratively to improve it Examples: “Is this config of 8-puzzle solvable?” Placing queens on a chessboard How many airline flights to have to where.

evelyn
Download Presentation

Iterative Improvement Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Iterative Improvement Algorithms • For some problems, path to solution is irrelevant: just want solution • Start with initial state, and change it iteratively to improve it • Examples: • “Is this config of 8-puzzle solvable?” • Placing queens on a chessboard • How many airline flights to have to where

  2. Calculus approach • If you know the function, can take derivative: solve derivative = 0 • Example: Given fencing of length 100 feet, find dimensions to get maximum area

  3. Hill-climbing search(or gradient descent) • Example: Given 22 professors, allocate them to depts • M&CS: 10, Physics: 5, English: 7 • Goal: Maximize majors in 3 departments • For a given arrangement of profs, I can estimate how many majors; but simulation takes an hour • Start at a guess, then start hill climbing from there • Show on blackboard

  4. Hill-climbing in general • Move in direction of increasing value • Useful when path to solution is irrelevant • Drawbacks: • Local maxima • Plateaux • Ridges • Can get around this some with random-restart hill climbing

  5. Simulated Annealing • Technique inspired by engineering practice of cooling liquid • At each iteration make a random move • If position is better than current, do it • Over time, slowly drop “temperature” T • If position is worse, do it with probability P • P becomes smaller as T drops • P = exp(change in value / T) • Eventually, algorithm reverts to hill climbing • Popular in VLSI layout

  6. Genetic Algorithms(Evolutionary Computing) • Genetic Algorithms used to try to “evolve” the solution to a problem • Generate prototype solutions called chromosomes (individuals) • Backpack problem as example: • http://home.ksp.or.jp/csd/english/ga/gatrial/Ch9_A2_4.html • All individuals form the population • Generate new individuals by reproduction • Use fitness function to evaluate individuals • Survival of the fittest: population has a fixed size • Individuals with higher fitness are more likely to reproduce

  7. Reproduction Methods • Mutation • Alter a single gene in the chromosome randomly to create a new chromosome • Example • Cross-over • Pick a random location within chromosome • New chromosome receives first set of genes from parent 1, second set from parent 2 • Example • Inversion • Reverse the chromsome

  8. Interpretation • Genetic algorithms try to solve a hill climbing problem • Method is parallelizable • The trick is in how you represent the chromosome • Tries to avoid local maxima by keeping many chromosomes at a time

  9. Another Example:Traveling Sales Person Problem • How to represent a chromosome? • What effects does this have on crossover and mutation?

  10. TSP • Chromosome: Ordering of city numbers • (1 9 2 4 6 5 7 8 3) • What can go wrong with crossover? • To fix, use order crossover technique • Take two chromosomes, and take two random locations to cut • p1 = (1 9 2 | 4 6 5 7 | 8 3) • p2 = (4 5 9 | 1 8 7 6 | 2 3) • Goal: preserve as much as possible of the orderings in the chromosomes

  11. Order Crossover • p1 = (1 9 2 | 4 6 5 7 | 8 3) • p2 = (4 5 9 | 1 8 7 6 | 2 3) • New p1 will look like: • c1 = (x x x | 4 6 5 7 | x x) • To fill in c1, first produce ordered list of cities from p2, starting after cut, eliminating cities in c1 • 2 3 9 1 8 • Drop them into c1 in order • c1 = (2 3 9 4 6 5 7 1 8) • Do similarly in reverse to obtain • c2 = (3 9 2 1 8 7 6 4 5)

  12. Mutation & Inversion • What can go wrong with mutation? • What is wrong with inversion?

  13. Mutation & Inversion • Redefine mutation as picking two random spots in path, and swapping • p1 = (1 9 2 4 6 5 7 8 3) • c1 = (1 9 8 4 6 5 7 2 3) • Redefine inversion as picking a random middle section and reversing: • p1 = (1 9 2 | 4 6 5 7 8 | 3) • c1 = (1 9 2 | 8 7 5 6 4 | 3)

  14. Online Search • Online = “in real world”, “realtime” • trying to solve the 8-puzzle with an 8-puzzle in front of you • robot trying to navigate a maze • Most of the search techniques we talked about don’t work • Example: What goes wrong with BFS?

  15. Options • Depth-first search works just fine, assuming actions are reversible • Hill climbing works fine, but how to avoid local maxima? • Simulated annealing might help avoid them, but when you are stuck... • Randomized restarts undoable • How to handle?

  16. LRTA* • Learning Real Time A* • At each state you visit, record heuristic on distance to goal • If stuck in local minimum, find adjoining state with lowest heuristic estimate • Update this state with that estimate plus cost of getting there • Move on • Example

More Related