Intro to Theory of Computation

1. LECTURE22 • Already learned • Automata theory • Computability theory • Last part: complexity theory Today: time complexity • Measuring complexity • Asymptotic notation • Relationships between models Intro to Theory of Computation CS 464 Sofya Raskhodnikova

2. How much time/memory needed to decide a language? FINITE CONTROL 0 0 .. 0 1 1 .. 1 Scan input and reject if it is not of the form Repeat while both 0s and 1s remain on the tape: Cross off one 0 and one 1 Accept if no 0s and no 1s left; otherwise reject.” Example: Consider Time needed for 1-tape TM? =“

3. Running time analysis If M is a TM and then “M runs in time ” means for every input of length , M on halts within steps S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson • Focus on worst case: • upper bound on running time for all inputs of given length • Exact time depends on computer • instead measure asymptotic growth

4. Asymptotic notation O-notation (upper bounds): f(n) = O(g(n))means there exist constants c > 0, n0 > 0 such that 0 f(n) cg(n)for all nn0. EXAMPLE:2n2 = O(n3) (c = 1, n0 = 2) functions, not values S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson L22.4

5. Asymptotic Notation • One-sided equality: T(n) = O(f(n)). • Not transitive: • ; • and • but . • Alternative notation: . S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

6. Examples O(n3) O() O(n) O(n), also O(n2) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson 106n3+ 2n2 -n +10 = + = ) = n =

7. -notation (lower bounds) O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2). f(n) = (g(n))means there exist constants c > 0, n0 > 0 such that 0 cg(n) f(n) for all nn0. EXAMPLE: (c = 1, n0 = 16) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

8. -notation (lower bounds) • Be careful:“Any comparison-based sorting algorithm requires at least O(n log n) comparisons.” • Meaningless! • Use  for lower bounds. S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

9. -notation (tight bounds) (g(n)) means both O(g(n)) and(g(n)) EXAMPLE: Polynomials are simple: ad nd + ad–1nd–1 +  + a1n + a0 = (nd) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

10. o-notation and -notation O-notation and -notation are like  and . o-notation and -notation are like < and >. f(n) = o(g(n))means for every constant c > 0, there exists a constants n0 > 0 such that 0 f(n) cg(n) for all nn0. 2n2 = o(n3) EXAMPLE: (n0 = 2/c) S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

11. Overview S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

12. Common Functions: Asymptotic Bounds • Polynomials. a0 + a1n + … + adnd is (nd) if ad > 0. • Logarithms. log a n = (log b n) for all constants a, b > 0. For every x > 0, log n = o(nx). • Exponentials. For all r >1 and all d > 0, nd = o(rn). • Factorial. . By Sterling’s formula, log grows slower than every polynomial can avoid specifying the base Every exponential grows faster than every polynomial S. Raskhodnikova; based on slides by E. Demaine, C. Leiserson

13. Sort by asymptotic order of growth • n! • n1,000,000 • n1/log(n) • log(n!) • n log n • log n • n2 • 2n • n • log log n

14. Time complexity classes TIME() is a class of languages. TIME()means that some 1-tape TM M that runs in time O() decides A.

15. How much time/memory needed to decide a language? Scan input and reject if it is not of the form Repeat while both 0s and 1s remain on the tape: Cross off one 0 and one 1 Accept if no 0s and no 1s left; otherwise reject.” Example: Consider =“ runs in time Is there a faster algorithm?

16. How much time/memory needed to decide a language? Scan input and reject if it is not of the form Repeat while both 0s and 1s remain on the tape: Reject if total number of 0s and 1s remaining is odd. Cross off every other 0 starting from the first 0 and every other 1 starting from the first 1 Accept if no 0s and no 1s left; otherwise reject.” Example: Consider =“ runs in time so Sipser, Problem 7.49: If language L can be decided in o time on a 1-tape TM then L is regular. 1-tape TM need time to decide A.

17. Two-tape TM can do it faster FINITE CONTROL 0 0 .. 0 1 1 .. 1 Scan input and reject if it is not of the form Copy 0s on tape 2. Scan tape 1. For each 1 read, cross off a 0 on tape 2. Accept if no 0s remain on tape 2; otherwise reject.” Example: Consider =“ A is decided in time (linear time) on a 2-tape TM. Unlike decidability, the complexity of the language depends on the model.

18. Complexity relationships between models: number of tapes Theorem. Let be a function, where Every time multitape TM has an equivalent time 1-tape TM. Proof: Recall: showed how to simulate multitape TMs by 1-tape TMs. Need time analysis of the simulation.

19. FINITE STATE CONTROL FINITE STATE CONTROL Theorem: Every multitapeTuring Machine can be simulated by a single tape Turing Machine 1 0 0 # # 1 0 0 #

20. SIMULATING MULTIPLE TAPES ● ● ● ● L#100#0#□#R L#100#0#1#R 100 L#100#□#□#R qi qi1 qjRSS qj qi1□□ qi1□ qj101RSS 1. “Format” tape. 2. For each move of the k-tape TM: Scan left-to-right, finding current symbols Scan left-to-right, writing new symbols Scan left-to-right, moving each tape head. 3. If a tape head goes off right end, insert blank. If tape head goes off left end, move back right.

21. Complexity relationships between models: number of tapes Theorem. Let be a function, where Every time multitape TM has an equivalent time 1-tape TM. Proof: Time analysis of the simulation. Time initialize tape: Time to simulate one step of the multitape TM: (at any point nonblank squares on each tape) Number of steps to simulate: Total time: