CS 208: Computing Theory

1. CS 208: Computing Theory Assoc. Prof. Dr. Brahim Hnich Faculty of Computer Sciences Izmir University of Economics

2. Contact Info • Room 415 • Email: brahim.hnich@ieu.edu.tr • Office hours • Tuesdays: 14—16

3. Textbooks • The course lecture are based on the following books • Harry R. Lewis and C.H. Papadimitriou. "Elements of the Theory of Computation", ISBN 0-13-272741-2, Prentice Hall. (Available in library) • Michael Sipser. "Introduction to the Theory of Computation", ISBN 053494728X, PWS Publishing Company.

4. Textbooks • The course lecture are based on the following books • Michael R. Garey and David S. Johnson. “Computer and Intractability", ISBN 0-7167-1045-5, W.H. Freeman and Company. • C.H. Papadimitriou. “Computational Complexity", ISBN 0-201-53082-1, Addison-Wesley Publishing Company, Inc.

5. Grading and exams • 6 Homeworks: 30% • Midterm Exam 30% • Final Exam : 40%

6. Programme • Part A: Preliminaries • Introduction • Mathematical Notations for Computations

7. Programme • Part B: Automata Theory • Finite State Automata • Context Free Grammars

8. Programme • Part C: Computability Theory • Turing Machines • Decidability and Undecidability

9. Programme • Part D: Complexity Theory • Time Complexity • Intractability

10. Programme • Part E: Summary and Concluding Remarks

11. Good Luck!

12. Outline • Introduction to the course • Goal • Organization • Outline • Mathematical Notations for Computations

13. What is computation? • Definition: Processing of information based on a finite set of operations or rules

14. What is computation? • Paper and pencil Arithmetic • 99 - 8 = 91 • Calculators • Bar code scanner • Digital camera • Computer programs in C and Java • …

15. What do we want in a “theory” • Generality • Technology-independent • Abstraction: ignores inessential details • Precision • Mathematical, formal • Can prove theorems about computation • Positive: what can be computed • Negative: what cannot be computed

16. So you want to be a good computer scientist/ software engineer? • Learn the rules • Algorithms, data structures, … • Learn basic principles • Data abstraction, … • Understand which problems you can solve with computers and which you simply cannot • … and why!

17. So you want to be a good computer scientist/ software engineer? • Learn the rules • Algorithms, data structures, … • Learn basic principles • Data abstraction, … • Understand which problems you can solve efficiently with computers and which you cannot!

18. Course Overall Objective • Understand the fundamental capabilities and limitations of computers • Make a theory out of the idea of computation

19. Why study theory? • Pragmatic Reasons • Apply efficient algorithms to tractable problems • Avoid intractable or impossible problems • Learn to tell the difference

20. Why study theory? • Mathematical education • Automata Theory • Computability Theory • Complexity Theory

21. Why study theory? • Philosophy • What is computation? • What is provable?

22. Course of two parts

23. The Good News • You only have to take this course once • Except those of you who will not pay attention to the lectures • At the end, you can say that you understand what computers are really about • Good for postgraduate and professional life

24. 2nd half of the course

25. The Bad News • Theory course • Formal • Requires mathematics • Understanding Proofs • Analytical skills

26. The Bad News • Theory course • Formal • Requires mathematics • Understanding Proofs • Analytical skills But, we will try to make it fun because it is really interesting!

27. End of Introduction Next: Topics to be covered

28. Three major topics • What is a computer? • Automata Theory • What can (cannot) be computed? • Computability Theory • What can (cannot) be computed efficiently? • Complexity Theory

29. Three major topics • What is a computer? • Automata Theory • What can (cannot) be computed? • Computability Theory • What can (cannot) be computed efficiently? • Complexity Theory

30. Complexity Theory

31. Complexity Theory • Key notion: tractable vs. intractable problems

32. Complexity Theory • Key notion: tractable vs. intractable problems EASY HARD

33. Complexity Theory • A problem is a general question: • Description of parameters • Description of solution • E.g. • Input: Given a list of integers • Can you sort the list in ascending order?

34. Complexity Theory • An algorithm is a step-by-step procedure: • A recipe • A computer program • A mathematical object

35. Complexity Theory • We want the most efficient algorithm as a function of problem size: • Fastest (mostly) • Most economical with memory (sometimes)

36. Example: Traveling Salesman Problem • Given • A set of m cities • A set of inter-city distances • Find a permutation of cities which makes a tour of minimum length

37. Example: TSP instance a 9 10 5 c 6 3 b d 9 (not drawn to scale)

38. Example: TSP instance a 9 10 5 c 6 3 b d 9 A solution: The tour a,b,d,c,a has length 27

39. Problem Size • What is an appropriate measure of problem size? • m the number of nodes? • m(m+1)/2 distances?

40. Problem Encoding • Use an encoding of the problem • Alphabet of symbols • Strings: abcd//10/5/9//6/9//3

41. Measures • Problem size: length of the encoding • Time complexity: how long an algorithm takes, as a function of problem size

42. Time Complexity • What is tractable? • A function f(n) is O(g(n)) whenever f(n) ≤ c. g(n) for n > some constant

43. Time Complexity A polynomial-time algorithm is one whose time complexity is O(p(n)) for some polynomial p(n) e.g. O(n2)

44. Time Complexity An exponential-time algorithm is one whose time complexity cannot be bounded by a polynomial e.g. O(nlog n)

45. Tractability • Basic distinction • Polynomial time means tractable • Exponential time means intractable

46. Tractability Execution time

47. Effect of Speed-Ups Let us wait for faster hardware!

48. Back to TSP • Your boss says: • “Get me an efficient traveling-salesman algorithm, or else!” • What are you going to do?

49. Response • “Yes Ma’am, expect it this afternoon!”

50. Response • “Yes Ma’am, expect it this afternoon!” • Problem is • All known algorithms (essentially) check all possible paths • Exhaustive checking is exponential! • … so best of luck!