1 / 17

Advanced Search

CMSC 671. Advanced Search. Prof. Marie desJardins September 20, 2010. Overview. Real-time heuristic search Learning Real-Time A* (LRTA*) Minimax Learning Real-Time A* (Min-Max LRTA*) Genetic algorithms. REAL-TIME SEARCH. Real-Time Search. Interleave search and execution Advantages:

janine
Download Presentation

Advanced Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMSC 671 Advanced Search Prof. Marie desJardinsSeptember 20, 2010

  2. Overview • Real-time heuristic search • Learning Real-Time A* (LRTA*) • Minimax Learning Real-Time A* (Min-Max LRTA*) • Genetic algorithms

  3. REAL-TIME SEARCH

  4. Real-Time Search • Interleave search and execution • Advantages: • Explore domain with unknown properties • Provide variable control over amount of search (deliberation) vs. execution (reactivity) • “Anytime” problem-solving performance • Interruptible • Learn to improve quality of heuristic function over multiple planning episodes • Can solve very large problems (if they have the right problem structure) Sven Koenig, “Real-time heuristic search: Research issues,” In Proceedings of the AIPS-98 Workshop on Planning as Combinatorial Search: Propositional, Graph-Based, and Disjunctive Planning Methods, pages 75-79, 1998.

  5. LRTA* • Simplest version: one step lookahead with heuristic-value updating: • Initialize s to the start state • If s is a goal state, stop • Choose an action a that minimizes f(succ(s,a)) • Update f(s) to the max of current f(s), 1+f(succ(s,a)) • Execute action a • Set s to the current state • Go to step 2 Richard E. Korf, “Real-time heuristic search,” Artificial Intelligence 42(2-3): 189-211, March 1990.

  6. Search Example • What will each algorithm do? • Greedy search (with and without repeated states) • A* (with and without repeated states) • Hill-climbing • (One-step-lookahead) LRTA* f(n): 2 1 1 0 1 S A B C G D 0

  7. Min-Max LRTA* • Variation of LRTA* that can be used in nondeterministic domains • Initialize s to the start state • If s is a goal state, stop • Choose an action a whose worst possible outcome minimizes f(succ(s,a)) (minimax step) • Update f(s) to the max of current f(s), 1+f(succ(s,a)) (across all possible successors of s when performing a) • Execute action a • Set s to the current state • Go to step 2 Sven Koenig, “Minimax real-time heuristic search,” Artificial Intelligence 129 (1-2): 165-197, June 2001.

  8. More Variations • Multi-step lookahead (using a “local search space”)

  9. Incremental Heuristic Search • Reuse information gathered during A* to improve future searches • Variations: • Failure  restart search at the point where the search failed • Failure  update h-values and restart search • Failure  update g-values and restart search • Fringe Saving A*, Adaptive A*, Lifelong Planning A*, DLite*...

  10. GENETIC ALGORITHMS

  11. Genetic Algorithms • Probabilistic search/optimization algorithm • Start with k random states (the initial population) • Generate new states by “mutating” a single state or “reproducing” (combining via crossover) two parent states • Selection mechanism based on children’s fitness values • Encoding used for the “genome” of an individual strongly affects the behavior of the search

  12. GA: Genome Encoding • Each variable or attribute is typically encoded as an integer value • Number of values determines the granularity of encoding of continuous attributes • For problems with more complex relational structure: • Encode each aspect of the problem • Constrain mutation/crossover operators to only generate legal offspring

  13. Selection Mechanisms • Proportionate selection: Each offspring should be represented in the new population proportionally to its fitness • Roulette wheel selection (stochastic sampling): Random sampling, with fitness-proportional probabilities • Deterministic sampling: Exact numbers of offspring (rounding up for most-fit individuals; rounding down for “losers”) • Tournament selection: Offspring compete against each other in a series of competitions • Particularly useful when fitness can’t be readily measured (e.g., genetically evolving game-playing algorithms or RoboCup players)

  14. GA: Crossover • Selecting parents: Pick pairs at random, or fitness-biased selection (e.g., using a Boltzmann distribution) • One-point crossover (swap at same point in each parent) • Two-point crossover • Cut and splice (cut point could be different in the two parents) • Bitwise crossover (“uniform crossover”) • Many specialized crossover methods for specific problem types and representation choices

  15. GA: Mutation • Bitwise (“single point”) mutation

  16. GA: When to Stop? • After a fixed number of generations • When a certain fitness level is reached • When fitness variability drops below a threshold • ...

  17. GA: Parameters • Running a GA involves many parameters • Population size • Crossover rate • Mutation rate • Number of generations • Target fitness value • ...

More Related