Algorithms

1. Solution exact approximate fast Speed slow Algorithms Tradeoff: Execution speed vs. solution quality “Short & sweet” “Quick & dirty” “Slowly but surely” “Too little, too late”

2. Computational Complexity Problem: Avoid getting trapped in local minima Global optimum

3. Approximation Algorithms Idea: Some intractable problems can be efficiently approximated within close to optimal! Fast: • Simple heuristics (e.g., greed) • Provably-good approximations Slower: • Branch-and-bound approaches • Integer Linear Programming relaxation

4. Approximation Algorithms Wishful: • Simulated annealing • Genetic algorithms

5. Minimum Vertex Cover Minumum vertex cover problem: Given a graph, find a minimum set of vertices such that each edge is incident to at least one vertex of these vertices. Example: Input graph Heuristic solution Optimal solution Applications: bioinformtics, communications, civil engineering, electrical engineering, etc. • One of Karp’s original NP-complete problems

6. Minimum Vertex Cover Examples

7. Approximate Vertex Cover Theorem: The minimum vertex cover problem is NP-complete (even in planar graphs of max degree 3). Theorem: The minimum vertex cover problem can be solved exactly within exponential time nO(1)2O(n). Theorem: The minimum vertex cover problem can not be approximated within £ 1.36*OPT unless P=NP. Theorem: The minimum vertex cover problem can be approximated (in linear time) within 2*OPT. Idea: pick an edge, add its endpoints, and repeat.

8. x y Approximate Vertex Cover Algorithm: Linear time 2*OPT approximation for the minimum vertex cover problem: • Pick random edge (x,y) • Add {x,y} to the heuristic solution • Eliminate x and y from graph • Repeat until graph is empty Best approximation bound known for VC! Idea: one of {x,y} must be in any optimal solution. Þ Heuristic solution is no worse than 2*OPT.

9. A B A B A B E C E E D C C D D Maximum Cut Maximum cut problem: Given a graph, find a partition of the vertices maximizing the # of crossing edges. Example: cut size = 2 cut size = 4 cut size = 5 Input graph Heuristic solution Optimal solution Applications: VLSI circuit design, statistical physics, communication networks. • One of Karp’s original NP-complete problems.

10. Maximum Cut Theorem [Karp, 1972]: The minimum vertex cover problem is NP-complete. Theorem: The maximum cut problem can be solved in polynomial time for planar graphs. Theorem: The maximum cut problem can not be approximated within £ 17/16*OPT unless P=NP. Theorem: The maximum cut problem can be approximated in polynomial time within 2*OPT. Theorem: The maximum cut problem can be approximated in polynomial time within 1.14*OPT. =1.0625*OPT

11. A B E C D A B A B E E C C D D Maximum Cut Algorithm: 2*OPT approximation for maximum cut: Start with an arbitrary node partition • If moving an arbitrary node across the partition will improve the cut, then do so • Repeat until no further improvement is possible cut size = 2 cut size = 3 cut size = 5 Input graph Heuristic solution Optimal solution Idea: final cut must contain at least half of all edges. Þ Heuristic solution is no worse than 2*OPT.

12. Approximate Traveling Salesperson Traveling salesperson problem: given a pointset, find shortest tour that visits every point exactly once. 2*OPT metric TSP heuristic: • Compute MST • T = Traverse MST • S = shortcut tour • Output S Analysis: S < T = 2* MST < OPT TSP 2* triangle inequality! T covers minimum spanning tree twice TSPminus an edge is a spanning tree

13. Non-Approximability • NP transformations typically do not preserve the approximability of the problem! • Some NP-complete problems can be approximated arbitrarily close to optimal in polynomial time. Theorem: Geometric TSP can be approximated in polynomial time within (1+e)*OPT for any e>0. • Other NP-complete problems can not be approximated within any constant in polynomial time (unless P=NP). Theorem: General TSP can not be approximated efficiently within K*OPT for any K>0 (unless P=NP).

14. Graph Isomorphism Definition: two graphs G1=(V1,E1) and G2=(V2,E2) are isomorphic iff $ bijection ƒ:V1®V2such that "vi,vjÎV1 (vi,vj)ÎE1Û (ƒ(vi),ƒ(vj))ÎE2 Isomorphism ºedge-preserving vertex permutation Problem: are two given graphs isomorphic? ≈ ≈ Note: Graph isomorphism ÎNP, but not known to be in P

15. Graph Isomorphism ≈ ≈ ≈ ≈ ≈ ≈

16. G2 G1 ≈ G Zero-Knowledge Proofs Idea: proving graph isomorphism without disclosing it! Premise: Everyone knows G1 and G2 but not ≈ ≈must remain secret! ≈ Create random G ≈ G1 Note: ≈ is≈(≈) Broadcast G Verifier asks for≈or≈ Broadcast≈or≈ Verifier checks G≈G1 or G≈G2 Repeat k times ÞProbability of cheating: 2-k ≈ ≈ Approximating a “proof”!

17. G2 G1 χ Zero-Knowledge Proofs Idea: prove graph 3-colorable without disclosing how! Premise: Everyone knows G1 but not its 3-coloring χ which must remain secret! χ' Create random G2≈ G1 Note: 3-coloring χ'(G2) is ≈(χ(G1)) Broadcast G2 Verifier asks for≈or χ' Broadcast≈or χ' Verifier checks G1≈G2 or χ'(G2) Repeat k times ÞProbability of cheating: 2-k ≈ χ Interactive proof!

18. χ Zero-Knowledge Caveats • Requires a good random number generator • Should not use the same graph twice • Graphs must be large and complex enough Applications: • Identification friend-or-foe (IFF) • Cryptography • Business transactions

19. Zero-Knowledge Proofs Idea: prove that a Boolean formula P is satisfiable without disclosing a satisfying assignment! Premise: Everyone knows P but not its secret satisfying assignment V ! P = (x+y+z)(x'+y'+z)(x'+y+z') ƒ Convert P into a graph 3-colorability instance G =ƒ(P) Publically broadcastƒ and G Use zero-knowledge protocol to show that G is 3-colorable Þ P is satisfiable iff G is 3-colorable Þ P is satisfiable with probability1-2-k G = Interactive proof!

20. Interactive Proof Systems • Prover has unbounded power and may be malicious • Verifier is honest and has limited power Completeness: If a statement is true, an honest verifier will be convinced (with high prob) by an honest prover. Soundness: If a statement is false, even an omnipotent malicious prover can not convince an honest verifier that the statement is true (except with a very low probability). • The induced complexity class depends on the verifier’s abilities and computational resources: Theorem: For a deterministic P-time verifier, class is NP. Def: For a probabilistic P-time verifier, induced class is IP. Theorem [Shamir, 1992]: IP = PSPACE

21. Concepts, Techniques, Idea & Proofs • 2-SAT • 2-Way automata • 3-colorability • 3-SAT • Abstract complexity • Acceptance • Ada Lovelace • Algebraic numbers • Algorithms • Algorithms as strings • Alice in Wonderland • Alphabets • Alternation • Ambiguity • Ambiguous grammars • Analog computing • Anisohedral tilings • Aperiodic tilings • Approximate min cut • Approximate TSP • Approximate vertex cover • Approximations • Artificial intelligence • Asimov’s laws of robotics • Asymptotics • Automatic theorem proving • Autonomous vehicles • Axiom of choice • Axiomatic method • Axiomatic system • Babbage’s analytical engine • Babbage’s difference engine • Bin packing • Binary vs. unary • Bletchley Park • Bloom axioms • Boolean algebra • Boolean functions • Bridges of Konigsberg • Brute fore • Busy beaver problem • C programs • Canonical order • Cantor dust • Cantor set • Cantor’s paradox • CAPCHA • Cardinality arguments • Cartesian coordinates • Cellular automata • Chaos • Chatterbots • Chess-playing programs • Chinese room • Chomsky hierarchy • Chomsky normal form • Chomskyan linguistics • Christofides’ heuristic • Church-Turing thesis • Clay Mathematics Institute

22. Concepts, Techniques, Ideas & Proofs • Clique problem • Cloaking devices • Closure properties • Cogito ergo sum • Colorings • Commutativity • Complementation • Completeness • Complexity classes • Complexity gaps • Complexity Zoo • Compositions • Compound pendulums • Compressibility • Computable functions • Computable numbers • Computation and physics • Computation models • Computational complexity • Computational universality • Computer viruses • Concatenation • Co-NP • Consciousness and sentience • Consistency of axioms • Constructions • Context free grammars • Context free languages • Context sensitive grammars • Context sensitive languages • Continuity • Continuum hypothesis • Contradiction • Contrapositive • Cook’s theorem • Countability • Counter automata • Counter example • Cross- product • Crossing sequences • Cross-product construction • Cryptography • DARPA Grand Challenge • DARPA Math Challenges • De Morgan’s law • Decidability • Deciders vs. recognizers • Decimal number system • Decision vs. optimization • Dedekind cut • Denseness of hierarchies • Derivations • Descriptive complexity • Diagonalization • Digital circuits • Diophantine equations • Disorder • DNA computing • Domains and ranges

23. Concepts, Techniques, Ideas & Proofs • Dovetailing • DSPACE • DTIME • EDVAC • Elegance in proof • Encodings • Enigma cipher • Entropy • Enumeration • Epsilon transitions • Equivalence relation • Euclid’s “Elements” • Euclid’s axioms • Euclidean geometry • Euler’s formula • Euler’s identity • Eulerian tour • Existence proofs • Exoskeletons • Exponential growth • Exponentiation • EXPSPACE • EXPSPACE • EXPSPACE complete • EXPTIME • EXPTIME complete • Extended Chomsky hierarchy • Fermat’s last theorem • Fibonacci numbers • Final states • Finite automata • Finite automata minimization • Fixed-point theorem • Formal languages • Formalizations • Four color problem • Fractal art • Fractals • Functional programming • Fundamental thm of Algebra • Fundamental thm of Arith. • Gadget-based proofs • Game of life • Game theory • Game trees • Gap theorems • Garey & Johnson • General grammars • Generalized colorability • Generalized finite automata • Generalized numbers • Generalized venn diagrams • Generative grammars • Genetic algorithms • Geometric / picture proofs • Godel numbering • Godel’s theorem • Goldbach’s conjecture • Golden ratio • Grammar equivalence

24. Concepts, Techniques, Ideas & Proofs • Grammars • Grammars as computers • Graph cliques • Graph colorability • Graph isomorphism • Graph theory • Graphs • Graphs as relations • Gravitational systems • Greibach normal form • “Grey goo” • Guess-and-verify • Halting problem • Hamiltonian cycle • Hardness • Heuristics • Hierarchy theorems • Hilbert’s 23 problems • Hilbert’s program • Hilbert’s tenth problem • Historical perspectives • Household robots • Hung state • Hydraulic computers • Hyper computation • Hyperbolic geometry • Hypernumbers • Identities • Immerman’s Theorem • Incompleteness • Incompressibility • Independence of axioms • Independent set problem • Induction & its drawbacks • Infinite hotels & applications • Infinite loops • Infinity hierarchy • Information theory • Inherent ambiguity • Initial state • Intelligence and mind • Interactive proofs • Intractability • Irrational numbers • JFLAP • Karp’s paper • Kissing number • Kleene closure • Knapsack problem • Lambda calculus • Language equivalence • Law of accelerating returns • Law of the excluded middle • Lego computers • Lexicographic order • Linear-bounded automata • Local minima • LOGSPACE • Low-deg graph colorability • Machine enhancements

25. Concepts, Techniques, Ideas & Proofs • Machine equivalence • Mandelbrot set • Manhattan project • Many-one reduction • Matiyasevich’s theorem • Mechanical calculator • Mechanical computers • Memes • Mental poker • Meta-mathematics • Millennium Prize • Minimal grammars • Minimum cut • Modeling • Multiple heads • Multiple tapes • Mu-recursive functions • MAD policy • Nanotechnology • Natural languages • Navier-Stokes equations • Neural networks • Newtonian mechanics • NLOGSPACE • Non-approximability • Non-closures • Non-determinism • Non-Euclidean geometry • Non-existence proofs • NP • NP completeness • NP-hard • NSPACE • NTIME • Occam’s razor • Octonions • One-to-one correspondence • Open problems • Oracles • P completeness • P vs. NP • Parallel postulate • Parallel simulation • Dovetailing simulation • Parallelism • Parity • Parsing • Partition problem • Paths in graphs • Peano arithmetic • Penrose tilings • Physics analogies • Pi formulas • Pigeon-hole principle • Pilotless planes • Pinwheel tilings • Planar graph colorability • Planarity testing • Polya’s “How to Solve It” • Polyhedral dissections

26. Concepts, Techniques, Ideas & Proofs • Polynomial hierarchy • Polynomial-time • P-time reductions • Positional # system • Power sets • Powerset construction • Predicate calculus • Predicate logic • Prime numbers • Principia Mathematica • Probabilistic TMs • Proof theory • Propositional logic • PSPACE • PSPACE completeness • Public-key cryptography • Pumping theorems • Pushdown automata • Puzzle solvers • Pythagorean theorem • Quantifiers • Quantum computing • Quantum mechanics • Quaternions • Queue automata • Quine • Ramanujan identities • Ramsey theory • Randomness • Rational numbers • Real numbers • Reality surpassing Sci-Fi • Recognition and enumeration • Recursion theorem • Recursive function theory • Recursive functions • Reducibilities • Reductions • Regular expressions • Regular languages • Rejection • Relations • Relativity theory • Relativization • Resource-bounded comput. • Respect for the definitions • Reusability of space • Reversal • Reverse Turing test • Rice’s Theorem • Riemann hypothesis • Riemann’s zeta function • Robots in fiction • Robustness of P and NP • Russell’s paradox • Satisfiability • Savitch’s theorem • Schmitt-Conway biprism • Scientific method • Sedenions

27. Concepts, Techniques, Ideas & Proofs • Self compilation • Self reproduction • Set cover problem • Set difference • Set identities • Set theory • Shannon limit • Sieve of Eratosthenes • Simulated annealing • Simulation • Skepticism • Soundness • Space filling polyhedra • Space hierarchy • Spanning trees • Speedup theorems • Sphere packing • Spherical geometry • Standard model • State minimization • Steiner tree • Stirling’s formula • Stored progam • String theory • Strings • Strong AI hypothesis • Superposition • Super-states • Surcomplex numbers • Surreal numbers • Symbolic logic • Symmetric closure • Symmetric venn diagrams • Technological singularity • Theory-reality chasms • Thermodynamics • Time hierarchy • Time/space tradeoff • Tinker Toy computers • Tractability • Tradeoffs • Transcendental numbers • Transfinite arithmetic • Transformations • Transition function • Transitive closure • Transitivity • Traveling salesperson • Triangle inequality • Turbulance • Turing complete • Turing degrees • Turing jump • Turing machines • Turing recognizable • Turing reduction • Turing test • Two-way automata • Type errors • Uncomputability

28. Concepts, Techniques, Ideas & Proofs • Uncomputable functions • Uncomputable numbers • Uncountability • Undecidability • Universal Turing machine • Venn diagrams • Vertex cover • Von Neumann architecture • Von Neumann bottleneck • Wang tiles & cubes • Zero-knowledge protocols . . . . “Make everything as simple as possible, but not simpler.” - Albert Einstein (1879-1955)