1 / 38

CS 4413 Algorithmic Processes

CS 4413 Algorithmic Processes. Intractable Problems NP Complete. Problems Solvable by Computers. Most problems tend to become more difficult when they are made larger.

april-casey
Download Presentation

CS 4413 Algorithmic Processes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 4413 Algorithmic Processes Intractable Problems NP Complete

  2. Problems Solvable by Computers • Most problems tend to become more difficult when they are made larger. • For example, The Towers of Hanoi is fairly easy when presented with three discs on three poles, and can be solved in seven moves. • For 64 discs, the problem is much harder and needs 18 446 744 073 709 551 615 moves.

  3. Problems Solvable by Computers … • Some problems can be answered in polynomial time. • This means that the time taken to find the answer is proportional to some polynomial function of the size of the problem. • For example, the time taken to perform a naïve sort on a list is proportional to the square of the size of the list. This means that if you double the size of the list, the list may take four times as long to order. • Such problems are often called ‘P’ problems.

  4. Problems Solvable by Computers … • Other problems increase in complexity according to some exponential function of the size of the problem or according to some factorial function. • The Towers of Hanoi is an example of an exponential problem. • Polynomial problems are considered to be ‘easy’ or ‘tractable’ because they do not increase in complexity too rapidly when the size increases. • Exponential problems and factorial problems are ‘hard’ or ‘intractable’, since their complexity increases very rapidly.

  5. Problems Solvable by Computers … • A term related to ‘P’ is ‘NP’. • NP stands for non-deterministic polynomial. • If a problem is NP, it means that it is possible to guess an answer and determine whether it is correct in polynomial time. • All P problems are also NP. (why?) • P problems are a subset of NP. • It is possible that all NP problems can be reduced in some way to P problems, but this has not been either proved or disproved.

  6. Problems Solvable by Computers … • Problems that are ‘NP-hard’ are ‘hard’ in the sense that the answer cannot be found in polynomial time and are ‘NP’. • It has been demonstrated that there are certain NP problems to which all other NP problems can be reduced. • This means that if one of these problems can be solved in polynomial time, they all can. Such problems are called ‘NP-complete’.

  7. Problems Solvable by Computers • Decidable Problems • Problems solvable by Turing Machines (and computers) • How costly is the solution (xL)? • Time • # of steps of the Turing Machine • Space • # of cells used • These measurements are parameterized by the length of x

  8. Some Real Functions

  9. Time Complexity • A Turing Machine M is of time complexity T(n) if whenever M is given an input w of length n, M halts after making at most T(n) moves, regardless of whether or not M accepts • We will be interested in polynomial functions T(n)

  10. Classes P and NP • Class P collects all problems that are solvable in polynomial time by a deterministic Turing Machine • Class NP is the analogous class for non-deterministic Turing Machines • Note that PNP • Is P=NP ? NP P

  11. Sample Problems in P • Sorting • Bubblesort • Quicksort • Kruskal’s Algorithm • Minimum weight spanning tree in a graph • Encoding in a TM • Integers coded in binary, plus separators , ( ) • m, # of nodes • W, maximum edge weight • (i,j,w) for each edge from node i to node j with weight w

  12. Sample Problems in NP • Everything in P • The Traveling Salesman Problem (TSP) • Does the graph have a hamiltonian circuit of weight W or less? • Encoding in TM • Graph, as before • Threshold • A hamiltonian circuit is one that visits every node of the graph only once!

  13. Traveling Salesman Problem • The number of different possible hamiltonian circuit configurations grows as O(m!) • Larger than 2cm for any c • It will cost O(m!) to find a possible solution • Checking that a configuration is a possible solution is easy – O(m4) • DTM – seems not to be polynomial • NTM – O(m4), where m is the number of nodes

  14. It is in NP Yes! Is it in P? We have no idea We are unable to exhibit a polynomial time algorithm on a TM We cannot prove that such algorithm does not exist Traveling Salesman Problem NP P

  15. P=NP PNP Which Venn Diagram is Accurate? NP P (b) P=NP · TSP (a) · TSP NP P (c) · TSP

  16. Polynomial-Time Reduction • Remember the reductions of the previous Chapter? • We are currently studying decidable problems • How about the transformation T? • We will now require that transformations be of polynomial-time complexity (by a DTM) Machine M’ for P1 Machine M For P2 Trans formation P1 instance P2 instance Yes No

  17. Transformations, Again Machine M’ for P1 Machine M For P2 Accept Accept Trans formation w x Reject Reject

  18. NP-Complete Problems • It is not known whether P = NP • In case P NP • Wouldn’t it be nice to find problems that would lie in NP – P? • Those are the NP-complete problems • An NP-complete problem L is a problem that satisfies the following • LNP • For every problem L’NP, there is a polynomial-time reduction from L’ to L • TSP is NP-complete

  19. NP-Complete Problems • A typical NP-complete problem is one on a combinatorial space for which the best algorithm known falls in the generate-and-test category • Generate a possible solution • Test if it works • Solutions could be thought of being generated by a NTM • The test is of polynomial complexity

  20. NP-Complete Problems – Theorems • If • P1 is NP-complete • P2 is in NP • There is a polynomial-time reduction from P1 to P2 • Then • P2 is NP-complete

  21. NP-Complete Problems – Theorems • If some NP-complete problem L is in P, then P = NP

  22. Which Is the First NP-Complete Problem? • To prove that a problem is in class NP-complete, we need to reduce it, in polynomial-time, to another that we already know is NP-complete • So, how do we get the first one? • The first one is the satisfiability problem

  23. Boolean Expressions • A boolean expression is defined as • A boolean-valued variable (1=true, 0=false) • Two boolean expressions connected by a binary operator: either  (and) or  (or) • A boolean expression prefixed with the unary operator  • A parenthesized boolean expression

  24. Boolean Expressions • x  y  z • x1 (x10 x11) • (x  z) (x  z) x = 1 y = 1 z = 0 x1 = 1 x10 = 0 x11 = 0 x = 0 z = 0

  25. SAT Problem • The satisfiability problem • Given a boolean expression, is it satisfiable? • is NP-complete

  26. SAT is NP-Complete • SAT is NP • Any other NP problem is reducible to SAT in polynomial-time • Use a restricted TM • Never writes a blank symbol • Never goes past the initial leftmost character • The proof needs to show the transformation • It needs to show that an instance accepted in any other NP-complete problem is equivalent of a (yet to be designed) boolean expression being feasible

  27. How to tackle those difficult problems... There are two principal approaches to tackling NP-hard problems or other “intractable” problems: • Use a strategy that guarantees solving the problem exactly but doesn’t guarantee to find a solution in polynomial time • Use an approximation algorithm that can find an approximate (sub-optimal) solution in polynomial time

  28. Exact solutions The exact solution approach includes the strategies: • exhaustive search (brute force) • useful only for small instances • backtracking • eliminates some cases from consideration • branch-and-bound • further cuts down on the search • fast solutions for most instances • worst case is still exponential

  29. Backtracking • Construct the state space tree: • nodes: partial solutions • edges: choices in completing solutions • Explore the state space tree using depth-first search • “Prune” non-promising nodes • dfs stops exploring subtree rooted at nodes leading to no solutions and... • “backtracks” to its parent node

  30. Example: The n-Queen problem • Place n queens on an n by n chess board so that no two of them are on the same row, column, or diagonal

  31. State-space of the four-queens problem

  32. Example: The m-coloring problem • Given a graph and an integer m, color its vertices using no more than m colors so that no two adjacent vertices are the same color. a c d b e

  33. 8 17 10 8 20 17 12 13 14 11 Branch and Bound • An enhancement of backtracking • Applicable to optimization problems • Uses a lower bound for the value of the objective function for each node (partial solution) so as to: • guide the search through state-space • rule out certain branches as “unpromising”

  34. Example: The assignment problem • Select one element in each row of the cost matrix C so that: • no two selected elements are in the same column; and • the sum is minimized • For example: • Job 1 Job 2 Job 3 Job 4 • Person a 9 2 7 8 • Person b 6 4 3 7 • Person c 5 8 1 8 • Person d 7 6 9 4 • Lower bound: Any solution to this problem will have • total cost of at least:

  35. Assignment problem: lower bounds

  36. State-space levels 0, 1, 2

  37. Complete state-space

  38. Traveling salesman example:

More Related