1 / 50

Cliff Shaffer Computer Science

Computational Complexity. Cliff Shaffer Computer Science. Computer Performance. Computer Performance. Do we need to care about performance when computers keep getting faster?. Computer Performance. Do we need to care about performance when computers keep getting faster?

fifi
Download Presentation

Cliff Shaffer Computer Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computational Complexity Cliff Shaffer Computer Science

  2. Computer Performance

  3. Computer Performance • Do we need to care about performance when computers keep getting faster?

  4. Computer Performance • Do we need to care about performance when computers keep getting faster? • Our history is to do bigger problems, not the same ones faster.

  5. Computer Performance • Do we need to care about performance when computers keep getting faster? • Our history is to do bigger problems, not the same ones faster. • More complex problems are less tied to our everyday “common sense” experience

  6. Algorithm Analysis • We could compare two programs by running them side-by-side

  7. Algorithm Analysis • We could compare two programs by running them side-by-side • But that means we have to implement them!

  8. Algorithm Analysis • We could compare two programs by running them side-by-side • But that means we have to implement them! • We want a way to easily evaluate programs before they are written • Look at the algorithm, not the program

  9. Algorithm Analysis • We could compare two programs by running them side-by-side • But that means we have to implement them! • We want a way to easily evaluate programs before they are written • Look at the algorithm, not the program • Algorithm Analysis estimates problem cost as a function of growth rate

  10. Simple Search • Find the record with key value 1005. • Sequential search: Look through each record in turn. • If there are n records, we do work proportional to n (unless we are lucky).

  11. Simple Search • Find the record with key value 1005. • Sequential search: Look through each record in turn. • If there are n records, we do work proportional to n (unless we are lucky). • The growth rate of this problem (in the average or worst cases) is linear on n.

  12. Sorting: Insertion Sort • For each record • Insert it into the sorted list made from the records already seen. • Might have to look at each such record already in the (sorted) list – n work. • Since we do this for n records, this is n*n in the worst (and average) cases. • So the cost is proportional to n2 (unless we are really lucky).

  13. Sorting: Merge Sort • For a list of n records: • Split the list in half. • Merge each half (using merge sort)‏ • Merge the records together (needs n work)‏ • Total cost: proportional to n log n

  14. Sorting Demo • URL: http://www.cs.ubc.ca/spider/harrison/Java/ • Compare Insertion, Shell, Heap, Quick sorts

  15. Does It Matter? • 1000 records: • Insertion sort: 1,000,000 • Mergesort: 10,000 • Factor of 100 difference • 1,000,000 records • Insertion sort: 1,000,000,000,000 • Mergsort: 20,000,000 • Factor of 50,000 difference • Hours vs. seconds on a real computer

  16. Tractable vs. Intractable • Cost n is better than cost n log n, which is better than cost n2.

  17. Tractable vs. Intractable • Cost n is better than cost n log n, which is better than cost n2. • These are all polynomial: a faster computer gives you a bigger problem in an hour by some factor.

  18. Tractable vs. Intractable • Cost n is better than cost n log n, which is better than cost n2. • These are all polynomial: a faster computer gives you a bigger problem in an hour by some factor. • Exponential growth: 2n.

  19. Tractable vs. Intractable • Cost n is better than cost n log n, which is better than cost n2. • These are all polynomial: a faster computer gives you a bigger problem in an hour by some factor. • Exponential growth: 2n. • Making input one unit bigger doubles the cost.

  20. Tractable vs. Intractable • Cost n is better than cost n log n, which is better than cost n2. • These are all polynomial: a faster computer gives you a bigger problem in an hour by some factor. • Exponential growth: 2n. • Making input one unit bigger doubles the cost. • Running twice as fast only gives you one more problem unit.

  21. Tractable vs. Intractable • Cost n is better than cost n log n, which is better than cost n2. • These are all polynomial: a faster computer gives you a bigger problem in an hour by some factor. • Exponential growth: 2n. • Making input one unit bigger doubles the cost. • Running twice as fast only gives you one more problem unit. • Exponential-time algorithms are “intractable”.

  22. Problems • Problems have many algorithms (sorting)‏ • What does “cost of a problem” mean?

  23. Problems • Problems have many algorithms (sorting)‏ • What does “cost of a problem” mean? • We say the problem’s cost is that of the best algorithm. • But we can’t know all the algorithms!

  24. Problems • Problems have many algorithms (sorting)‏ • What does “cost of a problem” mean? • We say the problem’s cost is that of the best algorithm. • But we can’t know all the algorithms! • It is possible (though difficult) to figure out lowest cost for any algorithm to solve the problem • Sorting: n log n lower bound.

  25. Traveling Salesman Problem • Given n cities, find a tour for all the cities that is of shortest length.

  26. Traveling Salesman Problem • Given n cities, find a tour for all the cities that is of shortest length. • Nobody knows a polynomial-time algorithm, only exponential algorithms.

  27. Traveling Salesman Problem • Given n cities, find a tour for all the cities that is of shortest length. • Nobody knows a polynomial-time algorithm, only exponential algorithms. • We don’t KNOW that this problem needs exponential time.

  28. Traveling Salesman Example • URL: http://itp.nat.uni-magdeburg.de/~mertens/TSP/TSP.html • Nearest Neighbor Heuristic

  29. NP-Completeness • Many, many problems are like traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time.

  30. NP-Completeness • Many, many problems are like traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time. • It is possible to “cheaply” convert any problem from this collection into any other.

  31. NP-Completeness • Many, many problems are like traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time. • It is possible to “cheaply” convert any problem from this collection into any other. • So if we had a polynomial time algorithm for any of them, we’d have one for all.

  32. NP-Completeness • Many, many problems are like traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time. • It is possible to “cheaply” convert any problem from this collection into any other. • So if we had a polynomial time algorithm for any of them, we’d have one for all. • These are called NP-complete problems.

  33. NP-Completeness • Many, many problems are like traveling salesman – we know no polynomial algorithm, and have no proof they need exponential time. • It is possible to “cheaply” convert any problem from this collection into any other. • So if we had a polynomial time algorithm for any of them, we’d have one for all. • These are called NP-complete problems. • NP problems are those problems for which we can quickly verify that a proposed solution is correct.

  34. Examples of NP-complete problems • Find the cheapest way to wire up telephones in a city • Find the largest clique in a graph • Find a way to assign values to a boolean expression to make it true • Find the largest matching between workers and compatible jobs • Find the least number of boxes needed to pack some goods

  35. What do you do? • … when you must solve an NP-complete problem?

  36. What do you do? • … when you must solve an NP-complete problem? • Approximation • Optimization

  37. What do you do? • … when you must solve an NP-complete problem? • Approximation • Optimization • Many engineering problems are optimization problems

  38. What do you do? • … when you must solve an NP-complete problem? • Approximation • Optimization • Many engineering problems are optimization problems • Examples: Aircraft design, “best” decision

  39. Why is optimization hard? • Imagine a 2d problem – find the highest hill.

  40. Why is optimization hard? • Imagine a 2d problem – find the highest hill. • Imagine a 10-parameter problem • Just checking the “high” and “low” values would give 1024 combinations.

  41. Why is optimization hard? • Imagine a 2d problem – find the highest hill. • Imagine a 10-parameter problem • Just checking the “high” and “low” values would give 1024 combinations. • Imagine a 10d “cube”… 1024 corners. • The goal is to find the best point in the cube, for a complex function.

  42. Why is optimization hard? • Imagine a 2d problem – find the highest hill. • Imagine a 10-parameter problem • Just checking the “high” and “low” values would give 1024 combinations. • Imagine a 10d “cube”… 1024 corners. • The goal is to find the best point in the cube, for a complex function. • Many problems have higher dimension

  43. Why is optimization hard? • Imagine a 2d problem – find the highest hill. • Imagine a 10-parameter problem • Just checking the “high” and “low” values would give 1024 combinations. • Imagine a 10d “cube”… 1024 corners. • The goal is to find the best point in the cube, for a complex function. • Many problems have higher dimension • Whole branches of CS/Math/Engineering devoted to optimization

  44. Uncomputable Problems • Not all problems that we can think of can be solved. • Abstractly, not all functions can be computed.

  45. Uncomputable Problems • Not all problems that we can think of can be solved. • Abstractly, not all functions can be computed. • The number of computable programs is countably infinite. • The number of integer functions is uncountably infinite.

  46. The Halting Problem • Problem: Given a particular program P on particular input I, does P halt when run on I?

  47. The Halting Problem • Problem: Given a particular program P on particular input I, does P halt when run on I? • Can be proved that this is impossible to determine in all cases.

  48. The Halting Problem • Problem: Given a particular program P on particular input I, does P halt when run on I? • Can be proved that this is impossible to determine in all cases. • Lots of problems like this that try to determine program behavior.

  49. The Halting Problem • Problem: Given a particular program P on particular input I, does P halt when run on I? • Can be proved that this is impossible to determine in all cases. • Lots of problems like this that try to determine program behavior. • Does this program contain a virus?

  50. Does this terminate for all n? (n is an integer)‏ While n > 1 do if Odd(n) then n = 3n + 1 else n = n/2

More Related