1 / 40

CSC 3130: Automata theory and formal languages

Fall 2008. The Chinese University of Hong Kong. CSC 3130: Automata theory and formal languages. Polynomial time. Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130. Efficient algorithms. The running time of an algorithm depends on the input For longer inputs, we allow more time

jcote
Download Presentation

CSC 3130: Automata theory and formal languages

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fall 2008 The Chinese University of Hong Kong CSC 3130: Automata theory and formal languages Polynomial time Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

  2. Efficient algorithms • The running time of an algorithm depends on the input • For longer inputs, we allow more time • Efficiency is measured as a function of input size PCP ATM decidable efficient

  3. Running time • The running time of TM M is the function tM(n): tM(n) = maximum number of steps that M takes on any input of length n L = {w$w: w∈ {a, b}} O(n) times M: On input x, until you reach $ O(n) steps Read and cross off first a or b before $ Read and cross off first a or b after $ If there is a mismatch, reject O(n) steps If all symbols but $ are crossed off, accept running time: O(n2)

  4. Running time L = {$0n1n: n ≥ 0} O(n) steps O(n) times M: On input x, Check that the input is of the form $0*1* Until everything is crossed off: Move head left to $ Cross off the leftmost 0 Cross off the following 1 If everything is crossed off, accept. O(n) steps running time: O(n2)

  5. A faster way L = {$0n1n: n ≥ 0} O(n) steps O(log n) times M: On input x, Check that the input is of the form $0*1* Until everything is crossed off: Move head left to $ Find the parities of number of 0s and 1s If one is even and other off, reject Otherwise, cross off every other 0 and every other 1 If everything is crossed off, accept. O(n) steps O(n) steps running time: O(n log n)

  6. Running time vs. model • What if we have a two-tape Turing Machine? L = {$0n1n: n ≥ 0} O(n) steps M: On input x, Check that the input is of the form $0*1* Copy 0* part of input on second tape Until ☐ is reached, Cross off next 1 from first tape and next 0 from second tape If both tapes reach ☐ at same time, accept O(n) steps running time: O(n)

  7. Running time vs. model • How about a java program? L = {$0n1n: n ≥ 0} M(string x) { n = x.len; if n % 2 == 0 reject; else for (i = 1; i <= n/2; i++) if x[i] != 0 reject; if x[n-i+1] != 1 reject; accept; } 1-tape TM 2-tape TM java running time: O(n) The running time can change depending on the model of computation! O(n log n) O(n) O(n)

  8. Measuring running time • What does it mean when we say: • One step inall mean different things! “This algorithm runs in 1000 steps” java RAM machine 1-tape TM d(q3, a) = (q7, b, R) if (x > 0) y = 5*y + x; write r3;

  9. Efficiency and the Church-Turing thesis • The Church-Turing thesis says all these models are equivalent in power… … but not in running time! java UNIVAC Turing Machine RAM machine multitape TM

  10. The Cobham-Edmonds thesis • However, there is an extension to the Church-Turing thesis that saysFor any realistic models of computation M1 and M2: • So any task that takes time T on M1 can be done in time (say) T2 or T3 on M2 M1 can be simulated on M2 with at mostpolynomial slowdown

  11. Efficient simulation • The running time of a program depends on the model of computation…… but in the grand scheme, this is irrelevant java ordinary TM multitape TM RAM machine slow fast Every reasonable model of computation can be simulated efficiently on every other

  12. Example of efficient simulation • Recall simulating multiple tapes on a single tape … … M G = {0, 1, ☐} … 1 0 0 0 1 1 0 0 S …    # 0 1 0 # 0 1 # 1 0 0 # G’ = {0, 1, ☐, 0, 1, ☐, #}   

  13. Running time of simulation • Each move of the multiple tape TM might require traversing the whole single tape 1 step of 3-tape TM O(s) steps of single tape TM s = rightmost cell ever visited s ≤ 3t + 4 after t steps t steps of 3-tape O(ts) = O(t2) single tape steps quadraticslowdown multi-tape TM single tape TM

  14. Simulation slowdown • Cobham-Edmonds Thesis: java multi-tape TM O(t) O(t) O(t2) O(t) O(t2) O(t) RAM machine single tape TM M1 can be simulated on M2 with at mostpolynomial slowdown

  15. Examples of running time (on a RAM) n = input size 0n1n parsing problem short paths matching algorithm LR(1) Edmonds CYK Dijkstra running time O(n) O(n) O(n log n) O(n3) O(n2) problem routing theorem proving scheduling running time 2O(n) 2O(n logn) 2O(n)

  16. Input representation • Since we measure efficiency in terms of input size, how the input is represented will make a difference • For us, any “reasonable” representation will be okay The number 17 17 OK (17 in base two) 10001 OK NO 11111111111111111 1 2 0000,0010,0001,0010 OK 3 4 OK {1,2,3,4}{2,3}{3,4} This graph

  17. Nondeterminism and the CE thesis • Cobham-Edmonds Thesis says: • But is nondetermistic computation realistic? Any two realistic models of computation can be simulated with polynomial slowdown

  18. Running time of nondeterministic TM • What about nondeterministic TMs? • For ordinary TMs, the running time of M on input x is the number of transitions M makes before it halts • But a nondeterministic TM can run for a different time on different “computation paths”

  19. Example • Definition of running time for nondeterministic TM qrej qacc 1/1R 10001 q0 1/1R 1/1R q1 0/0R 0/0R what is the running time? 5 computation path: any possible sequence of transitions max length of any computation path running time =

  20. Simulation of nondeterministic TM input tape x N M nondet TM multi-tape TM … 0 1 0 simulation tape z … 0 1 0 address tape a For all k > 0 For all possible strings a of length k Copy x to z. Simulate N on input z using a as choices If a specifies an invalid choice or simulation loops/rejects, abandon simulation. If N enters its accept state, accept and halt. If N rejected on all as of length k, reject and halt. … 1 2 2 1 represents possible choicesat each step each a describes a possible computation path

  21. Simulation slowdown for nondeterminism For all k > 0 For all possible strings a of length k Copy x to z. Simulate N on input z using a as choices If a specifies an invalid choice or simulation loops/rejects, abandon simulation. If N enters its accept state, accept and halt. If N rejected on all as of length k, reject and halt. running time of N is t simulation will halt when k = t running time of simulation = (running time for specific a) × (number of as of length ≤ t) = O(t)× 2O(t) = 2O(t)

  22. Simulation slowdown java multi-tape TM O(t) O(t2) O(t) O(t) O(t2) O(t) RAM machine single tape TM 2O(t) nondeterministic TM Do nondeterministic TM violate the Cobham-Edmonds thesis?

  23. Example • Recall the scheduling problem • Scheduling with nondeterminism: CSC 2110 CSC 3230 Can you schedule final exams so that there are no conflicts? schedule(int n, Edges edges) { for i := 1 to n: choose { c[i] := Y; } or { c[i] := R; } or { c[i] := B; } for all e in edges: if c[e.left] == c[e.right] reject; accept; } CSC 3130 CSC 3160 Exams → vertices Conflicts → edges Y R B Slots → colors

  24. Example In reality, programming languages don’t allow us to choose schedule(int n, Edges edges) { for i := 1 to n: choose { c[i] := Y; } or { c[i] := R; } or { c[i] := B; } for all e in edges: if c[e.left] == c[e.right] reject; accept; } ... but if we had it, we could schedule in linear time! We have to tell the computer how to make these choices Nondeterminism does not seem like a realistic feature of a programming language or computer

  25. Nondeterministic simulation • If we can do better, this would improve all known combinatorial optimization algorithms! 2O(t) slowdown nondeterministic TM multi-tape TM Is this the best we can do?

  26. Millenium prize problems • Recall how in 1900, Hilbert gave 23 problems that guided mathematics in the 20th century • In 2000, the Clay Mathematical Institute gave 7 problems for the 21st century computer science 1 P versus NP 2 The Hodge conjecture 3 The Poincaré conjecture 4 The Riemann hypothesis 5 Yang–Mills existence and mass gap 6 Navier–Stokes existence and smoothness 7 The Birch and Swinnerton-Dyer conjecture Perelman 2006 (refused money) Hilbert’s 8th problem $1,000,000

  27. The P versus NP question • Among other things, this asks: • Is nondeterminism a realistic feature of computation? • Can the choose construct be efficiently implemented? • Can we efficiently optimize any “well-posed” problem? poly(t) nondeterministic TM ordinary TM Can nondeterministic TM be simulated on ordinary TM with polynomial slowdown? Most people think not, but nobody knows for sure!

  28. The class P P is the class of all languages that can be decided on an ordinary TM whose running time is some polynomial in the length of the input decidable efficient context-free By the CE thesis, we can replace “ordinary TM” by any realisticmodel of computation regular java RAM multi-tape TM

  29. Examples of languages in P 0n1n parsing problem short paths matching algorithm LR(1) Edmonds CYK Dijkstra running time O(n) O(n) O(n log n) O(n3) O(n2) n = input size L01 = {$0n1n: n > 0} decidable LG = {x: x is generated by G} G is some CFG P (efficient) PATH = {(G, a, b, L): G is a graph with a path of length L from a to b} MATCH PATH LG MATCH = {G, a, b, L: G is a graph with a “perfect” matching} context-free L01

  30. Languages believed to be outside P problem routing thm-proving scheduling running time of best-known algorithm 2O(n) 2O(n) 2O(n) We do not know if these problems have faster algorithms, but we suspect not decidable ? SCHED ROUTE PROVE To explain why, first we need to understand what these languages have in common P (efficient) MATCH PATH LG

  31. More problems A clique is a subset of vertices that are all interconnected 2 1 {1, 4}, {2, 3, 4}, {1} are cliques An independent set is a subset of vertices so that no pair is connected 4 3 Graph G {1, 2}, {1, 3}, {4} are independent sets there is no independent set of size 3 A vertex cover is a set of vertices that touches (covers) all edges {2, 4}, {3, 4}, {1, 2, 3} are vertex covers

  32. Boolean formula satisfiability • A boolean formula is an expression made up of variables, ands, ors, and negations, like • The formula is satisfiable if one can assign values to the variables so the expression evaluates to true (x1∨x2 ) ∧ (x2∨x3∨x4) ∧ (x1) Above formula is satisfiable because this assignment makes it true: x1 = Fx2 = Fx3 = T x4 = T

  33. Status of these problems CLIQUE = {(G, k): G is a graph with a clique of k vertices} IS = {(G, k): G is a graph with an independent set of k vertices} VC = {(G, k): G is a graph with a vertex cover of k vertices} SAT = {f: f is a satisfiable Boolean formula} problem CLIQUE SAT VC IS 2O(n) 2O(n) running time of best-known algorithm 2O(n) 2O(n) What do these problems have in common?

  34. Checking solutions efficiently • We don’t know how to solve them efficiently • But if someone told us the solution, we would be able to check it very quickly Example: 12 Is (G, 5) in CLIQUE? 9 1,5,9,12,14 1 14 5 13 6 15 2 7 4 10 8 3 11

  35. Cliques via nondeterminism • Checking solutions efficiently is the same as designing efficient nondeterministic algorithms Example: Is (G, k) in CLIQUE? clique(Graph G, int k) { choose C := list of k vertices for i in C: for j in C: if i != j and G.is_edge(i,j) == false reject; accept; }

  36. Example: Formula satisfiability (x1∨x2 ) ∧ (x2∨x3∨x4) ∧ (x1) f = Checking solution: Nondeterministic algorithm: sat(Formula f) { x = new bool[f.n]; for i := 1 to n: choose { x[i] := true; } or { x[i] := false; } if f.eval(x) == true accept; else reject; } FFTT substitute x1 = F x2 = Fx3 = T x4 = T evaluate formula (F ∨T) ∧ (F∨T∨F) ∧ (T) f = can be done in linear time

  37. The class NP • The class NP: L can be solved on a nondeterministic TM in polynomial time iff its solutions can be checked in time polynomial in the input length NP is the class of all languages that can be decided on a nondeterministic TM whose running time is some polynomial in the length of the input

  38. P versus NP because an ordinary TM is only weaker than a nondeterministic one Conceptually, finding solutions can only be harder than checking them P is contained in NP decidable NP (efficiently checkable) IS SAT VC CLIQUE P (efficient) MATCH PATH LG

  39. P versus NP • The answer to the questionis not known. But one reason it is believed to be negative is because, intuitively, searching is harder than verifying • For example, solving homework problems (searching for solutions) is harder than grading (verifying the solution is correct) Is P equal to NP? $1,000,000

  40. Searching versus verifying Mathematician: Given a mathematical claim, come up with a proof for it. Scientist: Given a collection of data on some phenomena, find a theory explaining it. Engineer: Given a set of constraints (on cost, physical laws, etc.) come up with a design (of an engine, bridge, etc.) which meets them. Detective: Given the crime scene, find “who’s done it”.

More Related