Unit –VII Coping with limitations of algorithm power

1. Unit –VII Coping with limitations of algorithm power

2. 2 Solving NP-complete problems Engineered for Tomorrow • At present, all known algorithms for NP-complete problems require time that is super polynomial in the input size, and it is unknown whether there are any faster algorithms. • The following techniques can be applied to solve computational problems in general, and they often give rise to substantially faster algorithms: • Approximation: Instead of searching for an optimal solution, search for an "almost" optimal one.

3. Randomization: Use randomness to get a faster average running time, and allow the algorithm to fail with some small probability. Restriction: By restricting the structure of the input (e.g., to planar graphs), faster algorithms are usually possible. Parameterization: Often there are fast algorithms if certain parameters of the input are fixed. Heuristic: An algorithm that works "reasonably well" on many cases, but for which there is no proof that it is both always fast and always produces a good result. Meta-heuristic approaches are often used. Engineered for Tomorrow

4. 4 Tackling Difficult Combinatorial Problems Engineered for Tomorrow • There are two principal approaches to tackling difficult combinatorial problems (NP-hard problems). • Use a strategy that guarantees solving the problem exactly but doesn’t guarantee to find a solution in polynomial time. • Use an approximation algorithm that can find an approximate (sub-optimal) solution in polynomial time

5. 5 Exact Solution Strategies Engineered for Tomorrow Exhaustive search (brute force) • useful only for small instances Dynamic programming • applicable to some problems (e.g., the knapsack problem) Backtracking • eliminates some unnecessary cases from consideration • yields solutions in reasonable time for many instances but worst case is still exponential Branch-and-bound • further refines the backtracking idea for optimization problems

6. Backtracking Engineered for Tomorrow • The principal idea is to construct solutions one component at a time and evaluate such partially constructed candidates as follows. • If a partially constructed solution can be developed further without violating the problem’s constraints, it is done by taking the first remaining legitimate option for the next component. If there is no legitimate option for the next component, no alternatives for any remaining component need to be considered. • In this case, the algorithm backtracks to replace the last component of the partially constructed solution with its next option.

7. State-space tree Engineered for Tomorrow • This kind of processing is often implemented by constructing a tree of choices being made, called the state-space tree. • Its root represents an initial state before the search for a solution begins. • The nodes of the first level in the tree represent the choices made for the first component of a solution, • The nodes of the second level represent the choices for the second component, and so on. • A node in a state-space tree is said to be promising if it corresponds to a partially constructed solution that may still lead to a complete solution; otherwise, it is called non-promising node.

8. Engineered for Tomorrow • Leaves represent either non-promising dead ends or complete solutions found by the algorithm. • If the current node turns out to be non-promising, the algorithm backtracks to the node’s parent to consider the next possible option for its last component. • If there is no such option, it backtracks one more level up the tree, and so on.

9. Back-tracking Algorithm Engineered for Tomorrow

10. Engineered for Tomorrow • Construct the state-space tree • nodes: partial solutions • edges: choices in extending partial solutions Explore the state space tree using depth-first search • “Prune” non-promising nodes • DFS stops exploring sub-trees rooted at nodes that cannot lead to a solution and backtracks to such a node’s parent to continue the search

11. The problem is to place n queens on an n-by-n chessboard so that no two queens attack each other by being in the same row or in the same column or on the same diagonal. Example: n-Queens Problem Engineered for Tomorrow

12. N-Queens Problem Engineered for Tomorrow • We start with the empty board and then place queen 1 in the first possible position of its row, which is in column 1 of row 1. • Then we place queen 2, after trying unsuccessfully columns 1 and 2, in the first acceptable position for it, which is square (2,3), the square in row 2 and column 3. This proves to be a dead end because there is no acceptable position for queen 3. So, the algorithm backtracks and puts queen 2 in the next possible position at (2,4). • Then queen 3 is……………………….

13. Engineered for Tomorrow

14. Hamiltonian Circuit Problem Engineered for Tomorrow • We make vertex a the root of the state-space tree. The first component of our future solution, if it exists, is a first intermediate vertex of a Hamiltonian cycle to be constructed. • Using the alphabet order to break the three-way tie among the vertices adjacent to a, we select vertex b. • From b, the algorithm proceeds to c, then to d, then to e, and finally to f , which proves to be a dead end.

15. Hamiltonian Circuit Problem Engineered for Tomorrow • So the algorithm backtracks from f to e, then to d, and then to c, which provides the first alternative for the algorithm to pursue. Going from c to e eventually proves useless, and the algorithm has to backtrack from e to c and then to b. • From there, it goes to the vertices f , e, c, and d, from which it can legitimately return to a, yielding the Hamiltonian circuit a, b, f , e, c, d, a. If we wanted to find another Hamiltonian circuit, we could continue this process by backtracking from the leaf of the solution found.

16. Engineered for Tomorrow

17. Benefits and drawbacks Engineered for Tomorrow • It is typically applied to difficult combinatorial problems for which no efficient algorithms for finding exact solutions possibly exist. • Unlike the exhaustive search approach, which is doomed to be extremely slow for all instances of a problem, backtracking at least holds a hope for solving some instances of nontrivial sizes in an acceptable amount of time. This is especially true for optimization problems. • Even if backtracking does not eliminate any elements of a problem’s state space and ends up generating all its elements, it provides a specific technique for doing so, which can be of value in its own right.

18. Branch-and-Bound Engineered for Tomorrow • Branch and bound (BB) is a general algorithm for finding optimal solutions of various optimization problems, especially in discrete and combinatorial optimization. • It consists of a systematic enumeration of all candidate solutions, where large subsets of fruitless candidates are discarded, by using upper and lower estimated bounds of the quantity being optimized.

19. Branch-and-Bound Engineered for Tomorrow • In the standard terminology of optimization problems, a feasible solution is a point in the problem’s search space that satisfies all the problem’s constraints • An optimal solution is a feasible solution with the best value of the objective function

20. Branch-and-Bound Engineered for Tomorrow • 3 Reasons for terminating a search path at the current node in a state-space tree of a branch-and-bound algorithm: • The value of the node’s bound is not better than the value of the best solution seen so far. • The node represents no feasible solutions because the constraints of the problem are already violated. • The subset of feasible solutions represented by the node consists of a single point—in this case we compare the value of the objective function for this feasible solution with that of the best solution seen so far and update the latter with the former if the new solution is better.

21. Conclusion Engineered for Tomorrow • An enhancement of backtracking • Applicable to optimization problems • For each node (partial solution) of a state-space tree, computes a bound on the value of the objective function for all descendants of the node (extensions of the partial solution) • Uses the bound for: Ruling out certain nodes as “non-promising” to prune the tree – if a node’s bound is not better than the best solution seen so far guiding the search through state-space

22. Randomized Algorithms Engineered for Tomorrow • Randomized Algorithm aims to reduce both programming time and computational cost by approximating the process of calculation using randomness.

23. Randomized Algorithms Area Calculation Problem Engineered for Tomorrow Calculate the area of irregular shape (in red) in a box of size 20m x 24m 3,3 B 20,5 R 4, 15 R 6, 10 B A box of size 20 x 24 m2

24. Randomized Algorithms Engineered for Tomorrow • The randomization was made to produce a number point’s coordinate randomly with in the box. • The number of Hit & Miss will be counted. • Scaling calculation will be made the calculate the area, this case it is • Red Area = 20 x 24 x red points/all points

25. Approximation Algorithms for Knapsack Problem Engineered for Tomorrow • Greedy algorithms for the discrete knapsack problem Step 1 Compute the value-to-weight ratios ri = vi/wi , i = 1, . . . , n, for the items given. Step 2 Sort the items in non-increasing order of the ratios computed in Step 1. (Ties can be broken arbitrarily.) Step 3 Repeat the following operation until no item is left in the sorted list: if the current item on the list fits into the knapsack, place it in the knapsack; otherwise, proceed to the next item.

26. Let us consider the instance of the knapsack problem with the knapsack’s capacity equal to 10 and the item information Approximation Algorithms for Knapsack Problem Example Engineered for Tomorrow

27. The greedy algorithm will select the first item of weight 4, skip the next item of weight 7, select the next item of weight 5, and skip the last item of weight 3. The solution obtained happens to be optimal for this instance Approximation Algorithms for Knapsack Problem Example Engineered for Tomorrow • Computing the value-to-weight • ratios and sorting the items in non- • increasing order of these efficiency • ratios yields the table beside

28. Approximation Algorithms for Traveling Salesman Problem Engineered for Tomorrow • Nearest-neighbor algorithm - based on the nearest-neighbor • heuristic: the idea of always going to the nearest unvisited city • next. Step 1 Choose an arbitrary city as the start. Step 2 Repeat the following operation until all the cities have been visited: go to the unvisited city nearest the one visited last (ties can be broken arbitrarily). Step 3 Return to the starting city.

29. Approximation Algorithms for Traveling Salesman Problem Example Engineered for Tomorrow • With a as the starting vertex, the nearest-neighbor algorithm • yields the tour (Hamiltonian circuit) • Sa : a - b – c - d - a of length 10. The optimal solution, as can be easily checked by exhaustive search, is the tour • s.: a - b - d - c - a of length 8. Thus, the accuracy ratio • r(sa) = f (sa)/ f (s*) = 10/8 • = 1.25

30. Twice-Around-the-Tree Algorithm Engineered for Tomorrow Stage 1: Construct a minimum spanning tree of the graph (e.g., by Prim’s or Kruskal’s algorithm) Stage 2: Starting at an arbitrary vertex, create a path that goes twice around the tree and returns to the same vertex Stage 3: Create a tour from the circuit constructed in Stage 2 by making shortcuts to avoid visiting intermediate vertices more than once Note: RA = ∞ for general instances, but this algorithm tends to produce better tours than the nearest-neighbor algorithm

31. Examples Engineered for Tomorrow Walk: a – b – c – b – d – e – d – b – a Tour: a – b – c – d – e – a