Problem Solving Using Search

1. Reduce a problem to one of searching a graph. View problem solving as a process of moving through a sequence of problem states to reach a goal state Move from one state to another by taking an action A sequence of actions and states leading to a goal state is a solution to the problem. Problem Solving Using Search

3. A tree is made up of nodes and links connected so that there are no loops (cycles). Nodes are sometimes called vertices. Links are sometimes called edges. A tree has a root node. Where the tree ”starts”. Every node except the root has a single parent (aka direct ancestor). An ancestor node is a node that can be reached by repeatedly going to a parent. Each node (except a terminal, aka leaf) has one or more children (aka direct descendants). A descendant node is a node that can be reached by repeatedly going to a child. Trees

5. Set of nodes connected by links. But, unlike trees, loops are allowed. Also, unlike trees, multiple parents are allowed. Two kinds of graphs: Directed graphs. Links have a direction. Undirected graphs. Links have no direction. A tree is a special case of a graph. Graphs

7. Nodes represent cities that are connected by direct flight. Find route from city A to city B that involves the fewest hops. Nodes represent a state of the world. Which blocks are on top of what in a blocks scene. The links represent actions that result in a change from one state to the other. A path through the graph represents a plan of action. A sequence of steps that tell how to get from an initial state to a goal state. Representing Problems with Graphs

9. Assume that each state is complete. Represents all (and preferably only) relevant aspects of the problem to be solved. In the flight planning problem, the identity of the airport is sufficient. But the address of the airport is not necessary. Assume that actions are deterministic. We know exactly the state after an action has been taken. Assume that actions are discrete. We don’t have to represent what happens while the action is happening. We assume that a flight gets us to the scheduled destination without caring what happens during the flight. Problem Solving with Graphs

12. Uninformed, Any-path Depth-first Breadth-first In general, look at all nodes in a search tree in a specific order independent of the goal. Stop when the first path to a goal state is found. Informed, Any-path Exploit a task specific measure of goodness to try to reach a goal state more quickly. Classes of Search

13. Uninformed, optimal Guaranteed to find the ”best” path As measured by the sum of weights on the graph edges Does not use any information beyond what is in the graph definition Informed, optimal Guaranteed to find the best path Exploit heuristic (”rule of thumb”) information to find the path faster than uninformed methods Classes of Search

18. Assigns a numerical value to a board position The set of pieces and their locations represents a singel state in the game Represents the likelihood of winning from a given board position Typical scoring function is linear A weighted sum of features of the board position Each feature is a number that measures a specific characteristic of the position. ”Material” is some measure of which pieces one has in a given position. A number that represents the distribution of the pieces in a position. Scoring Function

21. To determine next move: Compute score for all possible next positions. Select the one with the highest score. If we had a perfect evaluation function, playing chess would be easy! Such a function exists in principle But, nobody knows how to write it or compute it directly. Scoring Function

22. Combinatorial optimization and enumeration problems are modeled by state spaces that usually lack any regular structure. Combinatorial optimization algorithms solve instances of problems that are believed to be hard in general, by exploring the usually-large discreet solution space of these instances. Goal is to find the “best” possible solution in the state space. Exhaustive search is often the only way to handle such “combinatorial chaos” Many “real” problems exhibit no regular structures to be exploited, and that leaves exhaustive enumeration as the only approach in sight. Combinatorial optimization algorithms try to reduce the effective size of the space, and explore the space efficiently. Combinatorial Optimization

23. Branch and bound is a general algorithmic method for finding optimal solutions of problems in combinatorial optimization. Solution space is discrete. The general idea: Find the minimal value of a function f(x) over a set of admissible values of the argument x called feasible region. Both f and x may be of arbitrary nature. Branch and Bound Algorithms

24. A branch-and-bound procedure requires two tools. First, a smart way of covering the feasible region by several smaller feasible subregions. This is called branching, since the procedure is repeated recursively to each of the subregions and all produced subregions naturally form a tree structure. Tree structure is called search tree or branch-and-bound-tree. Its nodes are the constructed subregions. Branch and Bound Algorithms

25. Second tool is bounding a fast way of finding upper and lower bounds for the optimal solution within a feasible subregion. Branch and Bound Algorithms

26. Based on a simple observation (for a minimization task) If the lower bound for a subregion A from the search tree is greater than the upper bound for any other (previously examined) subregion B, then A may be safely discarded from the search. This step is called pruning. It is usually implemented by maintaining a global variable m that records the minimum upper bound seen among all subregions examined so far any node whose lower bound is greater than m can be discarded. Basic Approach

27. When the upper bound equals the lower bound for a given node, the node is said to be solved. Ideally, the algorithm stops when all nodes have either been solved or pruned. In practice the procedure is often terminated after a given time at that point, the minimum lower bound and the maximum upper bound, among all non-pruned sections, define a range of values that contains the global minimum. Basic Approach

28. The efficiency of the method depends critically on the effectiveness of the branching and bounding algorithms used. Bad choices could lead to repeated branching, without any pruning, until the sub-regions become very small. In that case the method would be reduced to an exhaustive enumeration of the domain, which is often impractically large. There is no universal bounding algorithm that works for all problems. Little hope that one will ever be found. General paradigm needs to be implemented separately for each application. Branching and bounding algorithms are specially designed for it. Basic Approach

29. Limited look-ahead plus scoring I look ahead two moves (2-ply) First me – relative level 1 Then you – relative level 2 For each group of children at level 2 Check to see which has the minimum score Assign that number to the parent Represents the worst that can happen to me after your move from that parent position I pick the move that lands me in the position where you can do the least damage to me. This is the position which has the maximum value resulting from applying Step 1. Can implement this to any number (depth) of min-max level pairs. Min-Max Algorithm

31. May 1997Used Min-Max.256 specialized chess processors coupled into a 32 node supercomputer. Examined around 30 billion moves per minute. Typical search depth was 13ply - but in some dynamic situations it could go as deep as 30.

35. Pure optimization of min-max. No tradeoffs or approximations. Don’t examine more states than is necessary. ”Cutoff” moves allow us to cut off entire branches of the search tree (see following example) Only 3 states need to be examined in the following example Turns out, in general, to be very effective Alpha-Beta Pruning

38. Assumption of ordered tree is optimistic. ”Ordered” means to have the best move on the left in any set of child nodes. Node with lowest value for a min node. Node with highest value for a max node. If we could order nodes perfectly, we would notneed alpha-beta search! The good news is that in practice performance is close to optimistic limit. Move Generation

40. Goal is to produce ordered moves Needed to take advantage of alpha-beta search. Encodes a fair bit of knowledge about a game. Example order heuristic: Value of captured piece – value of attacker. E.g., ”pawn takes Queen” is the highest ranked move in this ordering Move Generator

41. Other place where substantial game knowledge is encoded In early programs, evaluation functions were complicated and buggy In time it was discovered that you could get better results by A simple reliable evaluator E.g., a weighted count of pieces on the board. Plus deeper search Static Evaluation

42. Deep Blue used static evaluation functions of medium complexity Implemented in hardware ”Cheap” PC programs rely on quite complex evaluation functions. Can’t search as deeply as Big Blue In general there is a tradeoff between Complexity of evaluation function Depth of search. Static Evalution

46. Time of Defeat: August 1994Read all about it at:http://www.cs.ualberta.ca/~chinook/

50. Neural network that is able to teach itself to play backgammon solely by playing against itself and learning from the results Based on the TD(Lambda) reinforcment learning algorithm Starts from random initial weights (and hence random initial strategy) With zero knowledge built in at the start of learning (i.e. given only a "raw" description of the board state), the network learns to play at a strong intermediate level When a set of hand crafted features is added to the network's input representation, the result is a truly staggering level of performance The latest version of TD Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players. TD-Gammon