Lecture 10

1. Lecture 10 • Annealing (final comments) • Adversary Search • Genetic Algorithms (genetic search) CS-424 Gregory Dudek

2. Comments on Assignment 2 • Who’s gotten started? • JAVA has a class for calling C functions. • See SUN’s java 1.1 web page. • It’s open ended. Give yourself time to ponder. • What are good heuristics. • Things in the URL? • Things in the page content of the “current node”? • Things in the content of the destination URL? • Tricks are important. • (This is typical of AI in practice). • Measure your performance. • This is always important! Do it scientifically. • Try easy stuff first. • A good general rule. CS-424 Gregory Dudek

3. A comment • On minimal length paths…. • The Dynamic-Programming Principle [Winston]: “The best way through a particular, intermediate place is the best way to itfrom the starting place, followed by the best way from it to the goal. There is no need to look at any other paths to or from the intermediate place.” A variation on the A* theme called dynamic programming. Use the h instead of h’ and efficiently compute true path costs. CS-424 Gregory Dudek

4. Adversary Search Basic formalism • 2 players. • Each has to beat the other (competing objective functions). • Complete information. • Alternating moves for the 2 players. CS-424 Gregory Dudek

5. The players • Players want to achieve “opposite” goals: • What’s bad for A is good for B, and vice versa. • With respect to our staticevaluation functions, one wants to maximize the function, the other wants to minimize it. • Use a game tree to describe the state space. • Approach: MINIMAX procedure. CS-424 Gregory Dudek

6. Game Tree • Nodes represent board configurations. • Edges represent allowed moves from one configuration to another. • For most 2-player games, alternating levels in the tree refer to alternating moves by the 2 players. • Static evaluation function measures the (apparent) goodness of a given configuration. • Returns a signed value, to indicate goodness for the minimizing or maximizing player. CS-424 Gregory Dudek

7. See overhead (based on RN Figure ) CS-424 Gregory Dudek

8. MINIMAX • If the limit of search has been reached, compute the static value of the current position relative to the appropriate player. Return the result • Otherwise, use MINIMAX on the children of the current position. • If the level is a minimizing level, return the minimum of the results • If the level is a maximizing layer, return the maximum of the results. CS-424 Gregory Dudek

9. MINIMAX observations • The static evaluation function is… • Crucial: all decisions are eventually based of the value it returns. • Irrelevant: if we search to the end of the game, we always know the outcome. • In practice, if same has a large search space then the static evaluation is especially important. CS-424 Gregory Dudek

10. Minimax • Divide tree into plies • Propagate information up from terminal nodes. • Storage needs grow exponentially with depth. CS-424 Gregory Dudek

11. Alpha-Beta Search • Can we avoid all the search-tree expansion implicit in MINIMAX? • Yes, with a simple observation closely related to A* : Once you have found a good path, you onlyneed to consider alternative paths that arebetter than that one. See example…. CS-424 Gregory Dudek

12. The a-b principle “If you have an idea that is surely bad, do not take time to see how truly awful it is.”” [Winston] CS-424 Gregory Dudek

13. The a-b principle. • “If you have an idea that is surely bad, do not take time to see how truly awful it is.” [Winston] CS-424 Gregory Dudek

14. Not in text a-b cutoff: How good is it? • What is the best-case improvement for alpha-beta? • What is the worst-case improvement for alpha-beta? • Best case: only examine one leftmost “pre-terminal” nodes fully. • Worst-case: • For some trees, it won’t help at all. • For some trees, even though it could help, it might not help if the nodes are encountered in the wrong order. CS-424 Gregory Dudek

15. Not in text Progressive deepening • How do you deal with (potential) time pressure? • Search progressively deeper: first depth 1, then depth 2, etc. • As before, ratio of interior to leaf nodes is bd(b-1)/(bd-1) or roughly (b-1) [so if b=30, it only takes 1/30th extra time to do the progressive part of progressive deepening] CS-424 Gregory Dudek

16. Tricks • Several techniques have proven important in practice. • Many critical techniques lead to improvements that have no importance in terms of worst-case asymptotic complexity. Transposition tables • Cache positions and their eventual outcome. • Is a position re-occurs, don’t need to reevaluate it, since the information is in cache. CS-424 Gregory Dudek

17. Not in text Issues • Horizon effect • Search-until-quiescent heuristic • Singular-extension heuristic CS-424 Gregory Dudek

18. Next major topic: LEARNING • First, a transitional problem. • How can we do hill climbing in a space of • representations ? • programs ? • entire systems ? • e.g. Find the best program for (small) problem X. • Find the best settings for the knowb on a refinery. • Find the best LOGO program for a picture. • Find the best organism encoded as DNA. CS-424 Gregory Dudek

19. GA’s Genetic Algorithms Cool, novel, controversial, costly (?), powerful (?). Inspired by real evolution. Idea: Encode the problem as a string. For example, a character string on a restricted alphabet. Define a fitness function. CS-424 Gregory Dudek

20. GA’s • Define a population of strings to encode a set of alternative solutions to a problem. • Now, we want to keep good solutions and improve them. • Add operators for: • mixing strings • randomly changing strings • Add an algorithm to repeatedly select the best, add variations, and evaluate. CS-424 Gregory Dudek

21. GA’s Genetic Algorithms Plastic transparencies & blackboard. CS-424 Gregory Dudek