Roadmap of the lecture

1. Roadmap of the lecture • Computational complexity • polynomial vs. exponential algorithms • non-deterministic computing • N(ondetermiistic) P(olynomial) problems • Example: the Traveling Salesman Problem (TSP) • Some non-TM (?) models of computing • Social insects • (Genetic algorithms and “evolutionary computing”) • DNA • Quantum computing • Neural nets

2. Some types of problems • Ill-defined problems • Undecidable problems e.g. will an arbitrary Turing Machine program ever halt? • Decidable problems (classified by complexity) • polynomial • exponential • non-deterministic polynomial

3. Polynomial vs. exponential time complexity (one algorithm step = 1 microsecond) (Figure 1.2 from Garvey & Johnson 1979)

4. The effect of technological improvements: size of the largest problem solvable in one hour... (Figure 1.3 from Garvey & Johnson 1979)

5. Time vs. task size in human cognitive processing: The Sternberg Item Memory test

6. Non-deterministic algorithms • Non-determinism in TMs and similar automata • one next state... or more? • guessing and backtracking • “determinization”: eliminating non-determinism • Complexity of non-deterministic algorithms • interesting class of problems: • guessing the answer seems hard (exponential or worse) • checking a guess is easy (polynomial) • called “non-deterministic polynomial” problems • abbreviated “NP” • open question: P = NP? • i.e. is guessing really, truly hard for these problems?

7. Some key equivalence classesof problem complexity • “NP complete” A decision problem is NP-complete if it is in NP and if every other problem in NP is “quickly” reducible to it Examples: traveling salesman, boolean satisfiability, knapsack, subgraph isomorphism, . . . • “NP hard” A problem is NP-hard if every other problem in NP is “quickly” reducible to it. I.e. “NP complete or even harder than that”

8. Practical approaches to NP problems • Approximation efficient solution within some bound • Probabilisticefficient solution to most expected problem instances • Special caseefficient solution for a certain class of cases • Heuristica method that sometimes sort of works

9. The traveling salesman

10. On beyond Turing... • Plenty of pre-Turing specialized computer designs • Samples from the post-Turing bestiary: • Genetic algorithms / evolutionary computing • Social insects, markets etc. • DNA computing / biomolecular computing • Neural nets / connectionism / PDP • Quantum computing • Themes: • Parallelism / interconnection • Gradient vs. discrete values • Chance

11. Collective Intelligence in Social Insects Self-organization was originally introduced in the context of physics and chemistry to describe how microscopic processes give rise to macroscopic structures in out-of-equilibrium systems. Recent research that extends this concept to ethology, suggests that it provides a concise description of a wide rage of collective phenomena in animals, especially in social insects. This description does not rely on individual complexity to account for complex spatiotemporal features which emerge at the colony level, but rather assumes that interactions among simple individuals can produce highly structured collective behaviors. E. Bonabeau et al., Self-Organization in Social Insects, 1997

12. Buridan’s Ants make a decision Percentage of Iridomyrex Humulis workers passing each (equal) arm of bridge per 3-minute period

13. Finding the shortest path: (from Dorigo et al. 1997)

14. More complex emergent structure: termite mounds…

15. Termite Theory: Bruinsma (1979): positive feedback mechanisms, involving responses to a short-lived pheromone in deposited soil pellets, a long-lived pheromone along travel paths, and a general tendency to orient pellet deposition to spatial heterogeneities; these lead to the construction of pillars and roofed lamellae around the queen. Deneubourg (1977): a simple model with parameters for the random walk of the termites and the diffusion and attractivity of the pellet pheronome, producing a regular array of pillars. Bonabeau et al. (1997): air convection, pheromone trails along walkways, and pheromones emitted by the queen; "under certain conditions, pillars are transformed into walls or galleries or chambers", with different outcomes depending not on changes in behavioral dispositions but on environmental changes caused by previous building. Thus "nest complexity can result from the unfolding of a morphogenetic process that progressively generates a diversity of history-dependent structures." Similar to models of embryological morphogenesis.

16. Why neural nets? • Computers: • fast circuits (109 state changes per second) • small number of interconnected processors (1-100) • digital flow of information • determinate, exactly repeatable behavior • designed and replicated exactly • small flaw or damage is catastrophic • mostly programmed • good at arithmetic, logic; bad at pattern recognition, abduction • Brains: • slow circuits (103 state changes per second) • large number of interconnected processors (1010) • quasi analog flow of information • stochastic behavior • grow organically with lots of case-to-case variation • small flaw or damage usually irrelevant • some “programming” (by evolution), some learning (from individual experience) • bad at arithmetic, logic; good at pattern recognition, abduction • The appeal of “neural net” models: a family of mathematical abstraction that share many of the differential properties of brains vs. computers