INHERENT LIMITATIONS OF COMPUTER PROGRAMS

CSci 4011 INHERENT LIMITATIONS OF COMPUTER PROGRAMS

Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, , q0, qaccept, qreject), where: Q is a finite set of states Σ is the input alphabet, where   Σ Γ is the tape alphabet, where   Γ and Σ Γ  : Q Γ→ Q Γ  {L,R} q0 Q is the start state qaccept Q is the accept state qreject Q is the reject state, and qreject qaccept

r.e. languages recursive languages The language L is Turing-recognizable or recursively enumerable if some TM accepts all and only the strings in L. A language is called decidable or recursive if some TM accepts all the strings in L and rejects all the strings not in L.

THE CHURCH-TURING THESIS L is recognized by a program for some computer* , L is recognized by a TM * The computer must be “reasonable”

The Church-Turing Thesis is “consistent” with all known “reasonable” computers. SCRATCH: 1111… 1101001… R1: 1011001… R2:  #1011#1101101#1011001#...# RAM: Programs for a computer have instructions like ADD R1, R2, R3; LOAD R1, R2; STORE R1,R2; MUL R1, R2, R2; BRANCH R1, X;…

Programming languages like Java, Python, Scheme, C, … are equivalent to TMs We call such languages Turing complete. Corollary. If two programming languages are Turing complete, then they can recognize exactly the same set of languages.

We can encode a TM as a string of 0s and 1s… start state reject state n states 0n10m10k10s10t10r10u1… m tape symbols (first k are input symbols) blank symbol accept state 〈(p,a), (q,b,L)〉 = 0p10a10q10b10

Since TMs and other languages are equivalent we can also express TMs as programs. Since programs are strings, we can consider languages whose elements are programs. We let 〈M〉 denote the encoding of TM M. Theorem. We can make a Universal TM, a TM that takes any 〈M〉 and any string w as input and simulates the computation of M on w.

Similarly, we can encode DFAs, NFAs, CFGs, etc into strings of 0s and 1s So we can define the following languages: ADFA = { 〈B,w〉 | B is a DFA that accepts string w } ANFA = { 〈N,w〉 | N is an NFA that accepts string w } ACFG = { 〈G,w〉 | G is a CFG that generates string w }

ADFA = { 〈B,w〉 | B is a DFA that accepts string w } Theorem: ADFA is decidable Proof Idea: Simulate B on w Corollary: ANFA is decidable ACFG = { 〈G,w〉 | G is a CFG that generates string w } Theorem: ACFG is decidable Proof Idea: Transform G into CNF. Try all derivations of length 2|w|-1

EDFA = { 〈B〉 | B is a DFA and L(B) =Ø} Theorem: EDFA is decidable Proof Idea: Let B = (Q,,δ,q0,F) and L(B)=Ø. • Set MARK := {q0} • Repeat until MARK is unchanged: • MARK := MARK ∪ δ(MARK,Σ) • If F ∩ MARK = Ø, accept, else reject. ALLDFA = { 〈B〉 | B is a DFA and L(B) = Σ*} Theorem: ALLDFA is decidable Proof Idea: If L(B) = Σ* then L(¬B) = Ø

INFINITEDFA = { 〈B〉 | B is a DFA and L(B) is infinite} Theorem: INFINITEDFA is decidable Proof Idea: Let B = (Q,,δ,q0,F). Suppose ∃w ∈ L(B) : |w| ≥ |Q|. • Let C be a DFA for {w : |w| ≥ |Q|}. • Build a DFA D for L(C) ∩ L(B). • If D ∈ EDFAthen reject, else accept. FINITEDFA = { 〈B〉 | B is a DFA and L(B) is finite} Corollary: FINITEDFA is decidable

The language L is undecidableif there is no TM that decides L. If L is undecidable, then every TM must either: 1. Accept (infinitely many) strings s ∉ L. 2. Reject (infinitely many) strings s ∈ L. 3. Loop forever on (infinitely many) strings.

UNDECIDABILITY We will prove that there are some undecidable languages – i.e., problems that a computer can’t solve no matter how long it computes. The proof is “simple:” There are more languages than there are Turing Machines.

POP QUIZ Letℕ= {0,1,2,…} be the Natural Numbers. Let 𝔼= {0,2,4,6,…} be the Even Numbers. Let ℤ= {…,-2,-1,0,1,2,…} be the Integers. Which is biggest, |ℕ|, |𝔼|, or |ℤ|?

Are there more blue or yellow dots? ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

SET THEORY 101 A function ƒ: A ! B is: ∀x∈A: ƒ(x) is unique. 1-1 (or injective) if ∀y ∈ B : y ∈ ƒ(A) onto (or surjective) if bijective if it is 1-1 and onto. ƒcan help us count. If ƒis: 1-1 then |A| |B| ≤ onto then |A| |B| ≥ bijective then |B| |A|. =

COUNTABLE SETS We say a set Y is countable if |Y| ≤ |ℕ|. Some countable sets: Ø, {0}, {0,1}, {0,1, …, 255} 𝔼= {0,2,4,6,…} 𝕆= {1,3,5,7,…} SQUARES = {0,1,4,9,16,25…} POWERS = {1,2,4,8,16,32…} |POWERS| = |SQUARES| = |𝔼| = |𝕆| = |ℕ|

There exists a bijection between ℕand ℕ ⨉ ℕ. (0,0) (0,1) (0,2) (0,3) (0,4) … (1,0) (1,1) (1,2) (1,3) (1,4) … (2,0) (2,1) (2,2) (2,3) (2,4) … (3,0) (3,1) (3,2) (3,3) (3,4) … (4,0) (4,1) (4,2) (4,3) (4,4) …

{0,1}* is countable { 〈M〉 | M is a TM } is countable Q+= { p/q | p,q∈ ℤ+} is countable! Is any set uncountable?

Theorem: There is no onto function from the positive integers to the real interval (0,1) Proof: Suppose ƒis such a function: 0.28347279… 2 1 2 3 4 5 : 0.88388384… 8 0.77635284… 6 0.11111111… 1 0.12345678… 5 : 1 if [ n-th digit of f(n) ]  1 [ n-th digit of r ] = 2 otherwise For all n, ƒ(n)  r

The process of constructing a counterexample by “contradicting the diagonal” is called DIAGONALIZATION

Prove that |(1,1)| = |(0,1)| = |ℝ| Prove that |ℝ ⨉ ℝ| = ℝ Is there a set bigger than ℝ?

Let L be any set and 𝒫(L) be the power set of L Theorem: There is no onto map from L to 𝒫(L) Assume, for a contradiction, that there is an onto map ƒ: L 𝒫(L) Proof: We construct a set Dƒthat cannot be the image, ƒ(y) for any y L. Let Dƒ= { x  L | x  ƒ(x) } If Dƒ= ƒ(y) then y  Dƒif and only if y  Dƒ

How is that diagonalization? Y Y N Y (yi∈D) = Y iff (yi∈ f(yi)) = N

EXAMPLE Let L = {0,1,2}. Then 𝒫(L) = {Ø, {0}, {1}, {2}, {0,1}, {0,2}, {1,2}, {0,1,2}} Let ƒ(0) = {1}, ƒ(1) = Ø, ƒ(2) = {0,2}. Then: D = {0,1}

No matter what, 𝒫(L) always has more elements than L

Not all languages over {0,1} are decidable Turing Machines Languages over {0,1} Sets of strings of 0s and 1s Strings of 0s and 1s L 𝒫(L)

INHERENT LIMITATIONS OF COMPUTER PROGRAMS