NECSI Summer School 2008 Week 3: Methods for the Study of Complex Systems Computation Theory

1. B B 1 0 B NECSI Summer School 2008Week 3: Methods for the Study of Complex SystemsComputation Theory Hiroki Sayama sayama@binghamton.edu

2. Four approaches to complexity Nonlinear Dynamics Complexity = No closed-form solution, Chaos Information Complexity = Length of description, Entropy Computation Complexity = Computational time/space, Algorithmic complexity Collective Behavior Complexity = Multi-scale patterns, Emergence

3. Computation? A sequence of rewritings applied to a symbolic representation of something • Rewriting rules are predefined and fixed so that: • Process of computation is efficient • Final result is accurate and useful

4. Propositional Logic

5. Proposition • A logical statement that is either true or false • “A human is an animal.” (true) • “An animal is a human.” (false) • These are both valid propositions

6. Propositional variables Propositional variables • “A human is an animal.” (true) • “An animal is a human.” (false) A = true (1), B = false (0) Propositional logic is about truth values only; no semantics involved → A → B Truth value

7. Logical operators • NOT (negation) ￢A, A • OR (disjunction) A∨B, A+B • AND (conjunction) A∧B, AB

8. A B A∨B 0 0 0 0 1 1 1 0 1 1 1 1 A B A∧B 0 0 0 0 1 0 1 0 0 1 1 1 A ￢A 0 1 1 0 Truth table

9. Logical formula • Symbolic expressions made of propositional variables and logical operators ￢B∧C (A∧B)∨(B∧￢C) • They are also propositions, and their truth values will be determined if truth values of all variables are given

10. Truth table of logical formula • X = (A∧B)∨(B∧￢C) A B C X 0 0 0 0 0 0 1 0 0 1 0 1 0 1 1 0 1 0 0 0 1 0 1 0 1 1 0 1 1 1 1 1

11. Exercise • Create truth tables of the following logical formulae: ￢(A∨B)∨￢A (A∧￢B)∧(A∨C)

12. Equivalence of logical formulae • Two logical formulae are equivalent if they have exactly the same truth table ￢(A∨B)∨￢A ⇔ ￢A

13. A B A→B 0 0 1 0 1 1 1 0 0 1 1 1 Logical operator for “implication” A → B • This expresses that “B must be true if A is true” ?

14. A B A⇔B 0 0 1 0 1 0 1 0 0 1 1 1 Logical operator for “equivalence” A ⇔ B • This expresses that “A and B are logically equivalent”

15. Complete set of operators • A set of logical operators with which any logical formula can be equivalently expressed • Examples: • { NAND } • { NOR } • { NOT, AND } • { NOT, OR }

16. Proof and Inference

17. Proof in propositional logic • Premises: A, B, C, … • Conclusion derived from premises: X • Proof in propositional logic is to show that (A∧B∧C∧…) → X is ALWAYS true

18. Method of proof (1) • Complete the truth table exhaustively and show that the formula is always true regardless of situations • Trivial, mechanistic method • Impractical if many variables are involved (due to explosion of possibility space)

19. Exercise • Prove “If A→(B→C), then (A→B)→(A→C)” by creating a truth table of this proposition

20. Method of proof (2) • Starting from unquestionably true “axioms”, keep modifying formulae based on logically sound inference rules until you reach what you want • Non-trivial, heuristic method • Potentially capable of proving complicated propositions through exploration • What humans usually do in math, physics • Reachable formulae are called “theorems”

21. Example: Hilbert-Bernays system • Made of 15 basic axioms and just 1 inference rule • The only inference rule: If A and A→B are both theorems, then B is also a theorem

22. Hilbert-Bernays axioms (1) I. Implication (1)A→(B→A) (2)(A→(A→B))→(A→B) (3)(A→B)→((B→C)→(A→C))

23. Hilbert-Bernays axioms (2) II. Conjunction (1)A∧B→A (2)A∧B→B (3)(A→B)→((A→C)→(A→B∧C))

24. Hilbert-Bernays axioms (3) III. Disjunction (1)A→A∨B (2)B→A∨B (3)(A→C)→((B→C)→(A∨B→C))

25. Hilbert-Bernays axioms (4) IV. Equivalence (1)(A⇔B)→(A→B) (2)(A⇔B)→(B→A) (3)(A→B)→((B→A)→(A⇔B))

26. Hilbert-Bernays axioms (5) V. Negation (1)(A→B)→(￢B→￢A) (2)A→￢￢A (3)　￢￢A→A

27. Hilbert-Bernays axioms (6) • Any formula that can be obtained by replacing A, B, and/or C in the basic axioms with other expressions is also an axiom • Example: A→(B→A) is an axiom, therefore (A→B)→(C→(A→B)) is also an axiom

28. Example of proof in HB system • Proving A∨A→A: Replace B in basic axiom I-(1) by A A→(A→A) Replace B in basic axiom I-(2) by A (A→(A→A))→(A→A) Apply inference rule to the above two formulae A→A Replace B and C in basic axiomIII-(3) by A (A→A)→((A→A)→(A∨A→A)) Apply inference rule to the above two formulae ((A→A)→(A∨A→A)) Apply inference rule to the above and A→A A∨A→A

29. Exercise • Prove A∧B→A∨B in the HB system

30. Consistency, soundness, completeness • An axiomatic system is: • Consistent if it doesn’t prove any theorems that contradict to each other • Sound if its inference rules never produce false formula from true premises • Complete if it can prove or disprove all possible logical formula • HB system is known to be consistent, sound and complete within prop. logic

31. Automata and Formal Languages

32. Logic vs. computation • Logic is a static description • It describes relationships between facts • It illustrates potential pathways leading to truth, but doesn’t tell where to go • Computation is a dynamic process • It executes specific operations in order • Computer (either machine or human) has its own internal states, I/Os, etc.

33. Automaton • A formal representation of dynamic, autonomous behaviors of machines • Has internal states • Changes its states and produces outputs over time according to predefined rules that refer to its own states and inputs received • States and time are usually discrete

34. Example • Parity checker • Tells whether the number of 1’s included in an input (bit string) is even or odd Initial state 1 Even Odd 1 0 0

35. Formal languages and automata • Formal language: An infinite set of strings that are grammatically valid • There are several distinct classes of grammatical complexity • Computational power of an automaton is often characterized by the class of languages it can recognize • After “hearing” a string, the automaton’s internal state tells whether that string is in the language or not

36. Chomsky’s hierarchy • Finite state automata • Can recognize regular languages • Pushdown automata • Can recognize context-free languages • Linear bounded automata • Can recognize context-sensitive languages • Turing machines • Can recognize recursively enumerable languages

37. Examples of languages • Regular languages: • { (01)n }, { bit strings in which 0’s and 1’s exist in even number each } • Context-free languages: • { 0n1n }, { bit strings in which 0’s and 1’s exist in the same number }, most programming languages • Context-sensitive languages: • { 0n1n2n }, most natural languages (weakly context-sensitive)

38. Finite state automaton • Has only finite number of states • Has no infinite memory • Therefore, all physically built computational systems are in this class in principle • Can recognize regular languages • Often used in complex systems modeling • E.g. cellular automata

39. Pushdown automaton • Finite state automaton with an infinitely long pushdown stack • Has infinite LIFO memory • Non-deterministic PDA can recognize context-free languages • Can handle nested structures inputs FSA Pushdown stack 1 0

40. Linear bounded automaton • (Non-deterministic) Turing machine whose tape length is a linear function of the length of input • Has infinite memory but its accessible range is bounded according to input size • Can recognize context-sensitive languages

41. Turing Machines

42. Turing machine (TM) • Finite state automata with an infinitely long memory tape • Has a read/write head that can move on the tape left and right B B 1 0 B FSA

43. Mathematical definition • Turing machine M: M=<S, S, f, q0, H> S: A finite set of states S: A finite set of tape symbols - Includes “B” for blank f: State-transition function - S x S→ S x S × {R, L, N} (motion of head) q0: Initial state (in S) H: A set of halting states (subset of S)

44. Rule table f: State-transition function - S x S→ S x S × {R, L, N} (motion of head) Often written as a set of “sss’s’m” • s: Current state of FSA • s: Symbol read from the tape • s’: Next state of FSA • s’: Symbol to be written to the tape • m: Direction of motion of the head

45. B B B B 1 0 0 1 B B q0 halt Computation by a Turing machine • Input: Initial contents of the tape • Output: Contents of the tape when the TM halts • TM halts when the FSA reaches one of its halting states Input Output Computation (state transition)

46. a/1, → 1/1, → q0 B/B, ← h B a 1 a B B 1 B 1 B 1 B 1 1 a B 1 a B 1 1 B 1 1 B q0 q0 q0 q0 h Example S = { q0, h },S = { B, a, 1 } f = { q0aq01R, q01q01R, q0BhBL } H = { h } Writing 1’s over non-blank symbols moving rightward until it reaches a “B”

47. Exercise • Design a Turing machine that moves its head rightward and keeps inverting bits written in its tape until its head reaches a blank symbol

48. Exercise • Figure out what the following TM does: S = { q0, e, o, h } S = { B, 1, E, O } f = { q01o1R, q0BhEN, e1o1R, eBhEN, o1e1R, oBhON } H = { h }

49. Computational Universality and Universal Turing Machines

50. Computational universality of TMs • Church–Turing thesis: “Every mathematical function that is naturally regarded as computable is computable by a Turing machine” • Not a rigorous theorem or hypothesis, but an empirical “thesis” widely accepted TMs are considered “computationally universal”