Emergence & The Information Processing Paradigm (alt: Ants vs Clocks!)

1. Emergence & The Information Processing Paradigm(alt: Ants vs Clocks!) Deepak Kumar Computer Science Bryn Mawr College

2. Emergence

3. Information Processing

4. Problem Solving(Investigation of Processes) • Given a dictionary of english words, what are all the anagram classes? (e.g. earthling, haltering, lathering) • Given the details of a tropical depression, can you predict if it will become a hurricane? What path will it take? • Can you play the game of chess in a way that guarantees a win or a draw? • How does the mind work? • What is the most optimal way to get from here to there? • What is the square root of 42? • If the Fed. Raises the short term interest rates? What impact will it have on international currencies? • What is the meaning of life?

5. Investigation: Asking Questions Given: A problem/process. • Does it have a model? • Is the model solvable? • Is it computable? • What is the best algorithm for it? • Write a computer program that implements the algorithm. • Is the program equivalent to the model? • Does the model lend any new insights into the problem/process?

6. Given: A Problem Does it have a model? Is the model solvable? Is it computable? What is the best algorithm? Write a program that implements the algorithm that models the problem Is the program correct? Do the solutions offered by the program work in the real world? Does the model/program offer any new insights into the problem?

7. Information Processing Program that implements the model that solves the problem.

8. Program that implements the model that solves the problem. Information Processing

9. Computers & Clocks • 1 flop = 1 floating-point operation (+, -, *, /) per second. • 1 megaflop = 1 MF = 106 flops • 1 gigaflop = 1 GF = 109 flops • 1 teraflop = 1 TF = 1012 flops • 1 petaflop = 1 PF = 1015 flops • The fastest PC can give a peak performance of 1GF/second • World’s fastest computer operates at approximately 35 TF/sec(as of 11/16/2003): NEC’s Earth Simulator at the Earth Simulator Center in Yokohama, Japan • Examining a single move of a chess game is roughly equivalent to 1000 FLOPS. • To make one move the computer will have to examine approximately 1065 board situations. • This will take the Earth Simulator approximately 1050 years to make one move!

10. Does it have a model? Is the model solvable? Is it computable? What is the best algorithm? Write a program that implements the algorithm that models the problem Is the program correct? Do the solutions offered by the program work in the real world? Does the model/program offer any new insights into the problem? Computability: Problems that can be solved by algorithms (Turing Machines)

11. Does it have a model? Is the model solvable? Is it computable? What is the best algorithm? Write a program that implements the algorithm that models the problem Is the program correct? Do the solutions offered by the program work in the real world? Does the model/program offer any new insights into the problem? Complexity Theory: Resources required during computation to solve a given problem (time & space)

12. What is an algorithm? A set of instructions arranged in a specific order is a procedure. Similar to a recipe, process, method, technique, procedure, routine, rigmarole, except the word “algorithm” connotes just a little something different. An algorithm is a finite, definite, effective procedure, with some output. Donald Knuth: The Art of Computer Programming, Volume 1: Fundamental Algorithms, 3rd edition, 1997.

13. Algorithm (origins) Abu ‘Abd Allah Muhammad ibn Musa al-Khwarizmi(Father of Abdullah, Mohammad, son of Moses, native of Khwarizm) Kitab al jabr wa’l-muqabala (Rules of restoring and equating)

14. Algorithm (properties) Finitethere must be an end to it within a reasonable time DefinitePrecisely definable in clearly understood terms, no “pinch of salt” type vagaries, or possible ambiguities EffectiveIt must be possible to actually carry out the steps ProcedureThe sequence of specific steps Outputunless there is something coming out of the computation, the result will be unknown!

15. Program that implements the model that solves the problem In f(n) steps. Problem Size • Time complexity of a problem is the number of steps that it takes to solve an instance of the problem as a function of the size of the input. i.e. if the input is of size, n, it will take f(n) steps to solve it. Problem size = 4

16. Complexity Classes(# steps it takes to solve a problem) • Constant time: O(1) • Logarithmic time: O(log n) • Linear time: O(n) • Quadratic Time: O(n2) • Polynomial time (P): O(nk) • Non-deterministic Polynomial Time (NP): O(nk) on some inputs. • Exponential time: O(2n) • Exponential time (in general): O(2p(n))

17. # time steps

18. Solving hard problems: Options • solve small problem sizes • use heuristics • get a close enough answer! • Emergence!

19. Emergence as a methodology for solving problems • What kinds of problems?no models exist, unsolvable models, uncomputable models, exponential problems! • How? “many ants are better than one”Train ‘em!

20. Problem Instance “ANTS” Solution The Emergence Approach to Problem Solving Training Regime Key Idea: Known solutions to problem instances are used in training.

21. Problem Instance “ANTS” Solution On “Graduation” The system will generalize to solve other problem instances!

22. Examples: Neural Networks Problem Instance Solution Supervised Training

23. Examples: Self-Organizing Maps Problem Instance Solution Unsupervised Training!

24. Examples: Genetic Programming Problem Instances Solutions Fitness Function

25. Emergence & Problem Solving Are solutions to several problem instances known? Given a solution, is it possible to quantify the “quality” of the solution? Identify the training regime and implement it. Does the solution offered by the trained program work in the real world? Does the resulting program offer any insights into the problem?

26. Some characteristics of emergent problem solving • Rather than starting from a model, start with a set of known problem-solution instances • Solutions may not be exact, but characterized as “best” solutions under the circumstances • Training takes time • Once trained, all solutions take the same amount of time, independent of the size of the problem! • Emergent solutions can beat the clock! • Emergent Computation = Surreal computation?