1 / 13

Exploring Computability and Kolmogorov Complexity

Learn about computability theory, Kolmogorov-Chaitin complexity, and the incompressibility of strings. Explore Shannon information theory and the Berry Paradox. Delve into recursion theorems and self-printing programs.

tmcnally
Download Presentation

Exploring Computability and Kolmogorov Complexity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computability Kolmogorov-Chaitin-Solomonoff. Other topics. Homework: Prepare presentations.

  2. Information • Shannon definition: series of (binary) choices. Information in measured in bits. • Computability definition: Let x be a binary string. A minimal description of x, d(x) is the shortest string,<M,w>, where Turing Machine M on input w halts with x on the tape. The descriptive complexity (aka Kolmogorov or Kolmogorov-Chaitin complexity) is K(x) = |d(x)| • the length of this shortest string. • Note: there may be more than one.

  3. Informally • Suppose we have a string consisting of 100 groups of '0110'. This is a description and it seems like it would be shorter than writing out the whole string. • The formal description includes the TM (or program) that knows what to do with 100 groups of something. The [full] minimal description consists of this TM plus an encoding of 100 groups, 0110. • Program that takes description and produces string. • This definition requires a TM for even the definition that is the whole string.

  4. Common example • Mandelbrot fractals are very intricate patterns and yet can be produced (re-produced?) by simple computer programs.

  5. Claim • There exists a constant c, such that for all x, K(x)<= |x| + c. • intuitively: take a fixed TM that halts immediately. Then it halts with the string x on it. • There exists a constant c, such that for all x and y, K(xy)<=2K(x) + K(y)+c

  6. Incompressible strings • Definitions: a string x is c-compressible if K(x)<=|x|-c. • If x is not c-compressible, x is incompressible by c. • If x is incompressible by 1, x is incompressible.

  7. Incompressible strings exist! • The number of strings of length n is greater than the number of descriptions of length less than n. So, some string of length n is not described by any description of length less than n.

  8. K(x) is not computable! • [and not because the definition is tied to any one of several models of computing] • See http://en.wikipedia.org/wiki/Kolmogorov_complexity It is a Halting Problem type of proof.

  9. Berry Paradox • Let n be the smallest positive integer that cannot be defined in fewer than twenty English words. • oops! • Relates also to Godel incompleteness results.

  10. Recursion Theorem also called fixed point theorem. • There exists a program that prints itself. • General strategy: create constant that represents working part of program and write program that prints out constant. • See Logo example in Shai lectures. • c# example next slide:http://igoro.com/archive/how-to-write-a-self-printing-program/

  11. c# example May need to force to be in two lines class P{static void Main(){var S="class P{{static void Main(){{var S={1}{0}{1};System.Console .Write(S,S,'{1}');}}}}";System.Console.Write(S,S,'"');}}

  12. finish Breaking the code • Discussion

  13. Homework • Prepare presentations!

More Related