CS 448 Foundation of Computing

1. CS 448 Foundation of Computing Soundararajan Ezekiel Department of Computer Science Ohio Northern University ADA, Ohio 45810 e-mail: s-ezekiel@onu.edu http://www.onu.edu/user/FS/sezekiel

2. Today’s Plan • What are the fundamental capabilities and limitation of computers • Three major topics: • What is a computer? - Automata Theory • What can (cannot) be computed? - Computability Theory • What can (cannot) be computed efficiently?--Complexity Theory

3. Why Study Theory? • Pragmatic reasons • apply efficient algorithms to tractable problems • avoid intractable or impossible problems • learn to tell the difference • Mathematical Education • automata theory • complexity classes • decidability theory • Philosophy • what is computation? • what is provable?

4. Topics • Automata Theory: What is a computer? (Review) • Computability Theory: What can computers do?(Semi-review) • Complexity Theory: What makes some problems computationally hard and others easy? • Evading intractability: • Approximation • Randomization • Parallelism • ... • Lower bounds: Circuit complexity.

5. Complexity Theory • Key notion: tractable vs. intractable problems. • A problem is a general question: • description of parameters • description of solution • An algorithm is a step-by-step procedure • a recipe • a computer program • a mathematical object • We want the most efficient algorithm • fastest (mostly) • most economical with memory (sometimes) • expressed as a function of problem size

6. Example: Traveling Salesman Problem Dave’s market a 5 9 CS dept c 10 6 3 ONU Park b d 9 (not drawn to scale) • Parameters: • set of places • set of inter- places distances

7. Example: Traveling Salesman Problem Dave’s market a 5 9 CS dept c 10 6 3 ONU Park b d 9 (not drawn to scale) Solution: Want the shortest tour through the places Example: a; b; d; c; a has length 27.

8. Problem Size • What is appropriate measure of problem size? • m nodes? • m(m+1)/2distances? • Use an encoding of the problem • alphabet of symbols • strings: abcd//10/5/9//6/9//3. • Measures • Problem Size: length of encoding • Time Complexity: how long an algorithm takes, as function of problem size.

9. Time Complexity • What is tractable? • A function f(n) is O(g(n)) whenever|f(n)| c*|g(n)|for n >some constant. • A polynomial-time algorithm is one whose time complexity is O(p(n)) for some polynomial p(n). • An exponential-time algorithm is one whose time complexity cannot be bounded by a polynomial (e.g., n log n).

10. Tractability • Basic distinction: • polynomial time = tractable exponential time = intractable Execution time 10 20 30 40 50 60 n .00001 .00002 .00003 .00004 .00005 .00006 second second second second second second n2 .00001 .00004 .00009 .00016 .00025 .00036 second second second second second second n3 .00001 .00008 .027 .064 .125 .216 second second second second second second n5 .1 3.2 24.3 1.7 5.2 13.0 second seconds seconds minute minutes minutes 2n .001 1.0 17.9 12.7 35.7 366 second second minutes days years centuries 3n .059 58 6.5 3855 2*10^8 1.3*10^13 second minutes years centuries centuries centuries

11. Effect of Speed-Ups Let's wait for faster hardware! Consider maximum problem size you can solve in an hour. present 100 times faster 1000 times faster n N1 100 N1 1000 N1 n2 N2 10 N2 31.6 N2 n3 N3 4.64 N3 10 N3 n5 N4 2.5 N4 3.98 N4 2n N5 N5+6.64 N5+9.97 3n N6 N6 +4.19 N6+6.29

12. Your boss says:Get me an efficient traveling-salesmanalgorithm, or else.”What are you going to do? • Response • “Yes Ma'am, expect it this afternoon!" • Problem is • All known algorithms (essentially) check all possible paths. • Exhaustive checking is exponential. • Good luck!

13. Answer • “Hah!, I will prove that no such algorithm is possible" • Problem is, proving intractability is very hard. • Many important problems have no known tractable algorithms -- no known proof of intractability. • “I can't find an efficient algorithm, I guess I'm just a pathetic loser. " • Bad for job security. • “The problem is NP-complete. I can't find an efficient algorithm, but neither can any of these famous people . . ." • Advantage is: The problem is “just as hard" as other problems ---smart people can't solve efficiently.----So it would do no good to fire you and hire a Dartmouth graduate.

14. Further Readings • http://www.claymath.org/prizeproblems/milliondollarminesweeper.htm • http://www.claymath.org/prizeproblems/pvsnp.htm • Response • “Would you settle for a pretty good algorithm?" • Intractability isn't everything. • find an approximate solution • use randomization (“probably pretty good") • parallelism can also help • Approximation, randomization, etc. are (arguably) the hottest areas in complexity theory today.

15. Computability • In the first half of the 20th century, mathematicians such as Kurt Goedel, Alan Turing, and Alonzo Church discovered that some problems do not have a computable solution. • Example: is a mathematical statement true or false? • What could be more amenable to automation? • Results needed theoretical models for computers. • These theoretical models helped lead to the construction of real computers.

16. Computability • Hilbert's tenth problem (1900): Find an finite procedure (algorithm) that decides whether a polynomial has an integer root. • A polynomial is a sum of terms, each term is a product of variables and a constant (called coefficient). • A root of a polynomial is an assignment of values to the variables such that the value of the polynomial is 0. • x = 2; y = 1is a root of the polynomial 5x +15y= 25 • There is no such algorithm (1970)!

17. Some Other Undecidable Problems • Does a program run forever? • Is a program correct? • Are two programs equivalent? • Is a program optimal? • Is a finitely-presented group trivial?

18. Mathematical Tools • Sets---order doesn't matter • finite:{2; 4; 6} • infinite:{2, 4,6,...},{x/ x is even} • subset: UV , UV • union: UV--- intersection: UV • Sequences--order matters (1, 3, 5, ...) • finite sequence called a tuple-2 elements is pair • k elements is k-tuple • Powerset of U is set of all subsets • Cartesian product of U x V

19. Strings and Languages • an alphabet is a finite set of symbols • a string over an alphabet is a finite sequence of symbols from that alphabet. • the length of a string is the number of symbols • the empty string  • reverse: abcd is dcba. • substring: xyz in xyzzy. • concatenation of xyz and zy is xyzzy. • xkand x, ... x, k times. • a language is a set of strings.

20. Proofs • We will use the following basic kinds of proofs. • by construction • by contradiction • by induction • by reduction • Sometimes we mix them.

21. Proof by Construction • Graph:- G = (V;E), where V is set of nodes or vertices, and E is set of edges --degree of a vertex is number of edges • A graph is k-regular if every node has degree k. • Theorem: for every even n >2,there exists a 3-regular graph with n nodes Note: picture is helpful, but not a proof!

22. Proof by Contradiction • Theorem: 2 is irrational. • Proof: Suppose not. Then 2 = m/n ,where m and n are relatively prime. • n 2 = m • 2n2 = m2 • So m2is even, and so is m = 2k. • 2n2 = (2k)2= 4k2 • n2 = 2k2So n2is even, and so is n. • Thus both m and n are even, not relatively prime!

23. Induction • Prove properties of elements of an infinite set.N={1, 2, 3,....} • To prove that  holds for each element, show: • base step: show that  (1) is true. • induction step: show that if  (i) is true (the induction hypothesis), then so is  (i +1).

24. Induction Example • Theorem: All cows are the same color. • Base step A single-cow set is definitely the same color. • Induction Step: Assume all sets of i cows are the same color. Divide the set{1..., i+1} into U ={1,…, i}, and V {2,…, i+1}. • all cows in U are the same color by the induction hypothesis. • all cows in V are the same color by the induction hypothesis. • all cows in UV are the same color by the induction hypothesis. • Ergo, all cows are the same color.

25. Back to Language • alphabet  example ={x} • L1={ x^n/ n=1,2,..} L2={x^(2n+1)/ n=0,1,2...} • ={ a b} • L=PALINDROME={, set of all string x/ reverse(x)=x}={ a b aa bb aaa ....} • KLEENE CLOSURE (Kleene star/ Kleene operator) Stpehern Kleene http://burks.brighton.ac.uk/burks/foldoc/55/109.htm • given an alphabet -- define language =  + any string of letter from  and denoted by *-- language called closure

26. Example • ={x} then *={ x^n, n=1,2..} • ={0 1} then *={ 0 1 00 01 ..} • extend it to set of words-- S= set of words then S*={ plus set of all finite strings formed by concatenating words} • example: S={ a ab} then S*={ b a aa bb ... abaab.. .} • Factoring:- abaab= (ab)(a)(ab) these are in S • Factoring is not unique--- example S={xx xxx} S*={x^n, n=0,2,3,..}-- x^6= x^3 x^3 or x^2 x^2 x^2

27. Note • parentheses () are not letters --- in case if you use () are letters of alphabet -- ={x ( )} length(xxxxx)=5 but length( (xx) (xxxx))=10 • Proof by constructive algorithm • closure( closure )= closure ---- (S*)*= S**= S* • Proof: ==> S*S** Every word in S** is made up of factors from S*. Every factors from S* is made up of factors from S--- every word in S** is made up from S. therefore every word from S** is also in S* • <=== S** S* we know that S S* apply closure both side we have S* S** • Together S*=S**

28. Homework Problems • Problem # 5, 7, 9. 14,