Disciplines in Distress: Artificial Intelligence and Connectionism

1. Disciplines in Distress: Artificial Intelligence and Connectionism Ath. Kehagias School of Engineering Aristotle Univ.

2. Some Remarks Not a philosophical talk (since I am not a philosopher but a philo-philosopher). Not a thesis, just an interesting (?) story and some questions. Twenty-five slides, roughly one quote per slide. Feel free to interrupt at any point.

3. First Definitions 01 Artificial Intelligence:The science of making machines do things that would require intelligence if done by people. Also known as Symbolicism, Symbolic Artif. Int. (SAI), Good Old-Fashioned Artif. Int. (GOFAI) etc. (Because of the extensive use of symbol-manipulating approaches by the practitioners.) Connectionism: A computational approach to modeling the brain which relies on the interconnection of many simple units to produce complex behavior Also known as Neural Networks (from the Dictionary of Philosophy of Mind, www.artsci.wustl.edu/~philos/MindDict)

4. An Example of AI (GPS) (define *school-ops* (list ;; If your son is at home and your car works, it is ;; possible to drive him to school. (Then he'll be at ;; school and will no longer be at home.) (make-op "drive son to school" '(son-at-home car-works) '(son-at-school) '(son-at-home)) ;; If your car needs a new battery, and the mechanic ;; knows the problem ;; and has been paid, it is possible him to install the ;; new battery. Then the car will work. (make-op "have the mechanic install a new battery" '(car-needs-battery mechanic-knows-problem mechanic-has-money) '(car-works) '(car-needs-battery)) 02a

5. Here, then, are a couple of problems that GPS can solve, using these operations: > (GPS '(son-at-home car-works) '(son-at-school) *school-ops*)drive son to school> (GPS '(son-at-home car-needs-battery have-phone-book have-money) '(son-at-school) *school-ops*)look up the telephone numbertelephone the mechanictell the mechanic what the problem ispay the mechanichave the mechanic install a new batterydrive son to school 02b An Example of AI (GPS)

6. An Example of NN The training procedure for TD-Gammon is as follows: the network observes a sequence of board positions starting at the opening position and ending in a terminal position characterized by one side having removed all its checkers. The board positions are fed as input vectors x[1], x[2], . . . , x[f] to the neural network. Each time step in the sequence corresponds to a move made by one side. For each input pattern x[t] there is a neural network output vector Y[t] indicating the neural network's estimate of expected outcome for pattern x[t]. At each time step, the TD(lambda) algorithm is applied to change the network's weights. The formula for the weight change is as follows: 03

7. Second “Definitions” 04 There is a community of people who do (S)AI. They attempt to create intelligent entities (usually software). To this end they use computer programs which manipulate symbolic data structures (lists, trees, graphs etc.). There is another community of people who do NN. They attempt to create intelligent entities and/or model human intelligence. To this end they use computer programs which manipulate numbers. (from Thanasis’ Own Dictionary)

8. Timeline 05

9. Early AI Goals “It is not my aim to surprise or shock you … But the simplest way I can summarize is to say that there are now in the world machines that think, that learn and that create. Moreover, their ability to do these things is going to increase rapidly, until --in a visible future-- the range of problems they can handle will be coextensive with the range to which the human mind has been applied.” H. Simon and A. Newell, “Heuristic problem solving: the next advance in operations research”, Op. Res., vol.6, p.6, 1958. 06

10. Early Critique of AI What Computers Can’t Do, H.L. Dreyfus, 1972. Four Assumptions Biological Assumption: Psychological Assumption Epistemological Assumption Ontological Assumption Two Important (and Missing) Factors: The Body (Embodied Intelligence) The Situation 07

11. Current AI Goals “Douglas Lenat has a ten-year program of building a huge semantic memory (CYC). Then we will see … When people start to build programs at that magnitude and they still cannot do what they are supposed to, then we will start worrying.” . (H.A. Simon, “Technology is not the problem” In P.Baumgartner and S.Payr, Speaking Mind, Princeton UP, 1995.) “AI no longer does Coginitive Modeling. It is a bunch of techniques in search of practical problems.” (J. Feldman cited in H.L. Dreyfus, Artif. Intelligence, vol.80, p.171-191, 1996) 08

12. Current Critique of AI What Computers Still Can’t Do, H.L. Dreyfus, 1993. 09

13. Early NN Goals "Perceptrons are not intended to serve as detailed copies of any actual nervous system. They're simplified networks, designed to permit the study of lawful relationships between the organization of a nerve net, the organization of its environment, and the 'psychological' performances of which it is capable. Perceptrons might actually correspond to parts of more extended networks and biological systems; in this case, the results obtained will be directly applicable.” (F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, Spartan Books, 1962) 10

14. Perceptrons (the book) “The final episode in this era was a campaign led by Marvin Minsky and Seymour Papert to discredit neural network research and divert neural network research funding to the field of “artificial intelligence” … The campaign was waged by means of personal persuasion by Minsky and Papert and their allies, as well as by limited circulation of a technical manuscript (which was later de-venomized and, after further refinement and expansion, published in 1969 by Minsky and Papert as the book Perceptorns.” (R. Hecht-Nielsen, Neurocomputation,1990) 11

15. NN Resurgence (AI Stagnation) “PDP models...hold out the hope of offering computationally sufficient and psychologically accurate mechanistic accounts of the phenomena of human cognition which have eluded successful explication in conventional computational formalisms…” (D.E. Rumelhart, J.L. McClelland, and the PDP Research Group,Parallel Distributed Processing: Explorations in the Microstructure of Cognition, MIT Press, 1987) 12

16. The BandWagon Effect “Undoubtedly, the emergence of 'new' connectionism was accompanied by a certain amount of jumping on the proverbial connectionist bandwagon.” Istvan S. N. Berkeley, “A Revisionist History of Connectionism”, 1997, http://www.ucs.louisiana.edu/~isb9112/dept/phil341/histconn.html 13

17. NN Stagnation 1 “I think and have thought for the last twenty years that the future consist of, just in a few days time, discovering efficient unsupervised learning algorithms that find a suitable representation. And I still believe that. I think this can be a huge technological payoff to making this work well. And I think the talk I gave this morning is a small amount of progress in that direction and on the technological front I think that is one of the major things that can happen in the next five years. For the last twenty years I’ve been saying "it’s gonna happen in the next five years" and I keep believing that.” G. Hinton, The Future and Prospects of Neural Networks: The Workshop in Edinburgh (Sep 8, 1999) 14

18. NN Stagnation 2 “ The maturing of neural networks presents an interesting study in hyperbole and substance. Those of us who jumped on the bandwagon early (in the heady days of "connectionism") foretold of a revolution that has not materialized as yet; looking back, I think we thought neural nets would have the same scale of impact as the World Wide Web has had.” From the Editor, Control Systems Magazine, October 2000 15

19. NN Stagnation? “Given the multiple relationships and interdependencies that now exist between financial markets, neural nets have a natural role to play. Their capacity to process and detect relationships and patterns in huge quantities of data goes far beyond that of a human trader. `Neural networks can find patterns in what would otherwise be disparate data that a human being would not visually be able to discern,’ says Mendelsohn. `From the standpoint of performing intermarket analysis, it is the right tool for the job’. ” Vantage Point: Intermarket Analysis Software. At their WebSite (http://www.profittaker.com/futures_options_new.asp) Andrew Webb reports on the latest resurgence of interest in neural network technology (2000). 16

20. AI/NN: Similarities They both have attempted to perform some form of cognitive modeling. They both have claimed that they can produce intelligent behavior. 17

21. AI/NN: Differences AI: works at the symbol manipulation level. NN works at the parallel distributed computation (sub-symbolic) level. 18

22. AI/NN: Sociohistorical Comparison In both cases grandiose claims were made at the start. In both cases the claims were not realized. In both cases a sub-product was a toolbox of (very) useful agorithms In both cases we have a degenerating research program (?) 19

23. AI/NN: Interactions A story from the 1988 Connectionist Models Summer School “Hybrid” Systems (e.g. “Connectionist Symbol Processing”). 20

24. Beyond AI/NN: Computational Intelligence “Flash forward to the late 1990s, and you'll find a hauntingly familiar atmosphere surrounding the evolutionary computation field (also referred to in some circles as genetic algorithms, after the technology that has been successfully embodied into software tools, or the loftier catch-all term artificial life). … Neural networks and evolutionary computing and fuzzy logic can all be lumped under the general phrase "computational intelligence," and for only the second time in this decade an entire conference was devoted to the research efforts of all three groups.” Intelligent Systems Report, May 1998, Vol. 15, No. 5, www.lionhrtpub.com/ISR/isr-5-98/wcci98.html 21

25. Philosophical Issues Which of the two (AI vs. NN) is more succesful? Why? 22

26. Sociohistorical Issues The ebb and flow of each field’s popularity? Is there some kind of vacuum which must be filled by one theory of intelligence? 23

27. “Religious” Issues Why did the debate between Symbolicists and Connectionists become so emotional? Why is there so strong resistance to the idea of Artificial Intelligence? (Dreyfus, Searle, Fodor) Some Possible Explanations Scientific Conflict Financial Conflict The Frankenstein Syndrome 24

28. Generalizations and Extensions Mathematics and Computer Science Anthropology and Sociology Philology and PostModern Studies To what extend is Academia / University changing and how? 25